8/12/2019 BGP Multihome
1/185
1 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
BGP Multihoming Techniques
Philip SmithPhilip Smith
NANOG 23, OaklandNANOG 23, Oakland
8/12/2019 BGP Multihome
2/185
2001, Cisco S ystems, I nc. All rights reserved. 2NANOG 23
Presentation Slides
Available on NANOG Web site
www.nanog.org/mtg-0110/smith.html
Available on
www.cisco.com/public/cons/seminars/NANOG23
8/12/2019 BGP Multihome
3/185
2001, Cisco S ystems, I nc. All rights reserved. 3NANOG 23
Introduction
Presentation has many configuration examples
Uses Cisco IOS CLI Aimed at Service Providers
Techniques can be used by many enterprises too
Feel free to ask questions
8/12/2019 BGP Multihome
4/185
2001, Cisco S ystems, I nc. All rights reserved. 4NANOG 23
BGP Multihoming Techniques
Definition & Options
Connecting to the same ISP
Connecting to different ISPs
Service Provider Multihoming
Using Communities
Case Study
8/12/2019 BGP Multihome
5/185
5 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Multihoming Definition &Options
8/12/2019 BGP Multihome
6/185
2001, Cisco S ystems, I nc. All rights reserved. 6NANOG 23
Multihoming Definition
More than one link external to the localnetwork
two or more links to the same ISP
two or more links to different ISPs
Usually two external facing routers
one router gives link and provider redundancy
only
8/12/2019 BGP Multihome
7/185
2001, Cisco S ystems, I nc. All rights reserved. 7NANOG 23
AS Numbers
An Autonomous System Number isrequired by BGP
Obtained from upstream ISP or RegionalRegistry
Necessary when you have links to more
than one ISP or exchange point
8/12/2019 BGP Multihome
8/185
2001, Cisco S ystems, I nc. All rights reserved. 8NANOG 23
Configuring Policy
Three BASIC Principles
prefix-lists to filter prefixesfilter-lists to filterASNs
route-maps to apply policy
Avoids confusion!
8/12/2019 BGP Multihome
9/185
2001, Cisco S ystems, I nc. All rights reserved. 9NANOG 23
Policy Tools
Local preference
outbound traffic flows
Metric (MED)
inbound traffic flows (local scope)
AS-PATH prepend
inbound traffic flows (Internet scope)
Communities
specific inter-provider peering
8/12/2019 BGP Multihome
10/185
2001, Cisco S ystems, I nc. All rights reserved. 10NANOG 23
Originating Prefixes
Basic Assumptions
MUST announce assigned address block to Internet
MAY also announce subprefixes reachability is notguaranteed
RIR minimum allocation is /20
several ISPs filter RIR blocks on this boundary
called Net Police by some
8/12/2019 BGP Multihome
11/185
2001, Cisco S ystems, I nc. All rights reserved. 11NANOG 23
Part of the Net Police prefix list!! APNIC
ip prefix-list FILTER permit 61.0.0.0/8 ge 9 le 20
ip prefix-list FILTER permit 202.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 210.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 218.0.0.0/7 ge 9 le 20
!! ARIN
ip prefix-list FILTER permit 63.0.0.0/8 ge 9 le 20
ip prefix-list FILTER permit 64.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 66.0.0.0/8 ge 9 le 20ip prefix-list FILTER permit 199.0.0.0/8 ge 9 le 20
ip prefix-list FILTER permit 200.0.0.0/8 ge 9 le 20
ip prefix-list FILTER permit 204.0.0.0/6 ge 9 le 20
ip prefix-list FILTER permit 208.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 216.0.0.0/8 ge 9 le 20
!! RIPE NCC
ip prefix-list FILTER permit 62.0.0.0/8 ge 9 le 20ip prefix-list FILTER permit 80.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 193.0.0.0/8 ge 9 le 20
ip prefix-list FILTER permit 194.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 212.0.0.0/7 ge 9 le 20
ip prefix-list FILTER permit 217.0.0.0/8 ge 9 le 20
8/12/2019 BGP Multihome
12/185
2001, Cisco S ystems, I nc. All rights reserved. 12NANOG 23
Net Police prefix list issues
meant to punish ISPs who pollute the routing tablewith specifics rather than announcing aggregates
impacts legitimate multihoming especially at theInternets edge
impacts regions where domestic backbone isunavailable or costs $$$ compared with internationalbandwidth
hard to maintain requires updating when RIRs startallocating from new address blocks
dont do it unless consequences understood and youare prepared to keep the list current
8/12/2019 BGP Multihome
13/185
2001, Cisco S ystems, I nc. All rights reserved. 13NANOG 23
Multihoming Scenarios
Stub network
Multi-homed stub network
Multi-homed network
Configuration Options
8/12/2019 BGP Multihome
14/185
2001, Cisco S ystems, I nc. All rights reserved. 14NANOG 23
Stub Network
No need for BGP
Point static default to upstream ISP
Upstream ISP advertises stub network
Policy confined within upstream ISPs policy
AS100
AS101
8/12/2019 BGP Multihome
15/185
2001, Cisco S ystems, I nc. All rights reserved. 15NANOG 23
Multi-homed Stub Network
Use BGP (not IGP or static) to loadshare
Use private AS (ASN > 64511)
Upstream ISP advertises stub network
Policy confined within upstream ISPs policy
AS100
AS65530
8/12/2019 BGP Multihome
16/185
2001, Cisco S ystems, I nc. All rights reserved. 16NANOG 23
Multi-Homed Network
Many situations possible
multiple sessions to same ISP
secondary for backup only
load-share between primary and secondary
selectively use different ISPs
AS300 AS200
AS100
Global Internet
8/12/2019 BGP Multihome
17/185
2001, Cisco S ystems, I nc. All rights reserved. 17NANOG 23
Multiple Sessions to an ISP Example One
eBGP multihop
eBGP to loopback addresses
eBGP prefixes learned withloopback address as next hop
router bgp 201
neighbor 1.1.1.1 remote-as 200
neighbor 1.1.1.1 ebgp-multihop 5
!
ip route 1.1.1.1 255.255.255.255 serial 1/0
ip route 1.1.1.1 255.255.255.255 serial 1/1
ip route 1.1.1.1 255.255.255.255 serial 1/2
AS 201
1.1.1.1
AS 200
8/12/2019 BGP Multihome
18/185
2001, Cisco S ystems, I nc. All rights reserved. 18NANOG 23
Multiple Sessions to an ISP Example Two
BGP multi-path
Three BGP sessionsrequired
limit of 6 parallel pathsrouter bgp 201
neighbor 1.1.2.1 remote-as 200
neighbor 1.1.2.5 remote-as 200
neighbor 1.1.2.9 remote-as 200
maximum-paths 3AS 201
AS 200
8/12/2019 BGP Multihome
19/185
2001, Cisco S ystems, I nc. All rights reserved. 19NANOG 23
Multiple Sessions to an ISP
Simplest scheme is to usedefaults
Learn/advertise prefixesfor better control
Planning and some workrequired to achieveloadsharing
No magic solution
AS 201
AS200
CC DD
AA BB
8/12/2019 BGP Multihome
20/185
2001, Cisco S ystems, I nc. All rights reserved. 20NANOG 23
Private-AS Application
Applications
ISP with single-homed customers
(RFC2270)
corporate networkwith severalregions and
connections to theInternet only in thecore
1880193.1.34.0/24 65003
193.2.35.0/24
65002193.0.33.0/24
65001193.0.32.0/24
A
193.1.32.0/22 1880
B
C
8/12/2019 BGP Multihome
21/185
2001, Cisco S ystems, I nc. All rights reserved. 21NANOG 23
Private-AS Removal
neighbor x.x.x.x remove-private-AS
Rules:
available for eBGP neighbors only
if the update has AS_PATH made up of private-AS numbers, theprivate-AS will be dropped
if the AS_PATH includes private and public AS numbers, private ASnumber will not be removedit is a configuration error!
if AS_PATH contains the AS number of the eBGP neighbor, theprivate-AS numbers will not be removed
if used with confederations, it will work as long as the private ASnumbers are after the confederation portion of the AS_PATH
8/12/2019 BGP Multihome
22/185
2001, Cisco S ystems, I nc. All rights reserved. 22NANOG 23
BGP Multihoming Techniques
Definition & Options
Connecting to the same ISP
Connecting to different ISPs Service Provider Multihoming
Using Communities
Case Study
8/12/2019 BGP Multihome
23/185
23 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Multihoming to the same ISP
8/12/2019 BGP Multihome
24/185
2001, Cisco S ystems, I nc. All rights reserved. 24NANOG 23
Multihoming to the same ISP
Use BGP for this type of multihoming
use a private AS (ASN > 64511)
upstream ISP proxy aggregates
in other words, announces only your addressblock to the Internet (as would be done if you
had one statically routed connection)
8/12/2019 BGP Multihome
25/185
25 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to the same ISP
One link primary, the other link backup only
8/12/2019 BGP Multihome
26/185
2001, Cisco S ystems, I nc. All rights reserved. 26NANOG 23
Two links to the same ISP
AS 109 AS 65534
AACC
AS109 removes private AS and any customer
subprefixes from Internet announcement
DDEEBB
primary
backup
8/12/2019 BGP Multihome
27/185
2001, Cisco S ystems, I nc. All rights reserved. 27NANOG 23
Two links to the same ISP(one as backup only)
Announce /19 aggregate on each link
primary link makes standard announcement
backup link increases metric on outbound, and reduceslocal-pref on inbound
When one link fails, the announcement of the /19aggregate via the other link ensures continued
connectivity
8/12/2019 BGP Multihome
28/185
2001, Cisco S ystems, I nc. All rights reserved. 28NANOG 23
Two links to the same ISP(one as backup only)
Router A Configuration
router bgp 65534
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.2 remote-as 109
neighbor 222.222.10.2 description RouterC
neighbor 222.222.10.2 prefix-list aggregate out
neighbor 222.222.10.2 prefix-list default in
!
ip prefix-list aggregatepermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
8/12/2019 BGP Multihome
29/185
2001, Cisco S ystems, I nc. All rights reserved. 29NANOG 23
Two links to the same ISP(one as backup only)
Router B Configuration
router bgp 65534
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.6 remote-as 109
neighbor 222.222.10.6 description RouterD
neighbor 222.222.10.6 prefix-list aggregate out
neighbor 222.222.10.6 route-map routerD-out out
neighbor 222.222.10.6 prefix-list default in
neighbor 222.222.10.6 route-map routerD-in in
!
..next slide
8/12/2019 BGP Multihome
30/185
2001, Cisco S ystems, I nc. All rights reserved. 30NANOG 23
Two links to the same ISP(one as backup only)
ip prefix-list aggregatepermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
route-map routerD-outpermit 10
match ip address prefix-list aggregate
set metric 10
route-map routerD-outpermit 20
!
route-map routerD-inpermit 10
set local-preference 90
!
8/12/2019 BGP Multihome
31/185
2001, Cisco S ystems, I nc. All rights reserved. 31NANOG 23
Two links to the same ISP(one as backup only)
Router C Configuration (main link)
router bgp 109
neighbor 222.222.10.1 remote-as 65534
neighbor 222.222.10.1 default-originate
neighbor 222.222.10.1 prefix-list Customer in
neighbor 222.222.10.1 prefix-list default out
!
ip prefix-list Customerpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
8/12/2019 BGP Multihome
32/185
2001, Cisco S ystems, I nc. All rights reserved. 32NANOG 23
Two links to the same ISP(one as backup only)
Router D Configuration (backup link)
router bgp 109
neighbor 222.222.10.5 remote-as 65534
neighbor 222.222.10.5 default-originate
neighbor 222.222.10.5 prefix-list Customer in
neighbor 222.222.10.5 prefix-list default out
!
ip prefix-list Customerpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
T li k t th ISP
8/12/2019 BGP Multihome
33/185
2001, Cisco S ystems, I nc. All rights reserved. 33NANOG 23
Two links to the same ISP(one as backup only)
Router E Configurationrouter bgp 109
neighbor 222.222.10.17 remote-as 110
neighbor 222.222.10.17 remove-private-AS
neighbor 222.222.10.17 prefix-list Customer out!
ip prefix-list Customerpermit 221.10.0.0/19
Router E removes the private AS andcustomers subprefixes from externalannouncements
Private AS still visible inside AS109
8/12/2019 BGP Multihome
34/185
34 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to the same ISP
With Loadsharing
8/12/2019 BGP Multihome
35/185
2001, Cisco S ystems, I nc. All rights reserved. 35NANOG 23
Loadsharing to the same ISP
AS 109 AS 65534
AACC
AS109 removes private AS and any customer
subprefixes from Internet announcement
DDEEBB
Link one
Link two
8/12/2019 BGP Multihome
36/185
2001, Cisco S ystems, I nc. All rights reserved. 36NANOG 23
Loadsharing to the same ISP
Announce /19 aggregate on each link
Split /19 and announce as two /20s, one on each link
basic inbound loadsharing
assumes equal circuit capacity and even spread of traffic acrossaddress block
Vary the split until perfect loadsharing achieved
Accept the default from upstream
basic outbound loadsharing by nearest exit
okay in first approx as most ISP and end-site traffic is inbound
8/12/2019 BGP Multihome
37/185
2001, Cisco S ystems, I nc. All rights reserved. 37NANOG 23
Loadsharing to the same ISP
Router A Configurationrouter bgp 65534
network 221.10.0.0 mask 255.255.224.0
network 221.10.0.0 mask 255.255.240.0
neighbor 222.222.10.2 remote-as 109
neighbor 222.222.10.2 prefix-list routerC out
neighbor 222.222.10.2 prefix-list default in
!
ip prefix-list defaultpermit 0.0.0.0/0
ip prefix-list routerCpermit 221.10.0.0/20
ip prefix-list routerCpermit 221.10.0.0/19
!
ip route 221.10.0.0 255.255.240.0 null0ip route 221.10.0.0 255.255.224.0 null0
Router B configuration is similar but with the other /20
8/12/2019 BGP Multihome
38/185
2001, Cisco S ystems, I nc. All rights reserved. 38NANOG 23
Loadsharing to the same ISP
Router C Configurationrouter bgp 109
neighbor 222.222.10.1 remote-as 65534
neighbor 222.222.10.1 default-originate
neighbor 222.222.10.1 prefix-list Customer in
neighbor 222.222.10.1 prefix-list default out
!
ip prefix-list Customerpermit 221.10.0.0/19 le 20
ip prefix-list defaultpermit 0.0.0.0/0
Router C only allows in /19 and /20 prefixes fromcustomer block
Router D configuration is identical
8/12/2019 BGP Multihome
39/185
2001, Cisco S ystems, I nc. All rights reserved. 39NANOG 23
Loadsharing to the same ISP
Loadsharing configuration is only on customerrouter
Upstream ISP has to
remove customer subprefixes from externalannouncements
remove private AS from external announcements
Could also use BGP communities
8/12/2019 BGP Multihome
40/185
40 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to the same ISP
Multiple Dualhomed Customers
(RFC2270)
Multiple Dualhomed Customers
8/12/2019 BGP Multihome
41/185
2001, Cisco S ystems, I nc. All rights reserved. 41NANOG 23
Multiple Dualhomed Customers(RFC2270)
AS 109AS 65534A1A1
CC
AS109 removes private AS and
any customer subprefixes fromInternet announcement
DDEE
B1B1
AS 65534A2A2
B2B2
AS 65534A3A3
B3B3
8/12/2019 BGP Multihome
42/185
2001, Cisco S ystems, I nc. All rights reserved. 42NANOG 23
Multiple Dualhomed Customers
Customer announcements as per previousexample
Use the sameprivate AS for each customer
documented in RFC2270
address space is not overlapping
each customer hears default only
Router Anand Bnconfiguration same as RouterA and B previously
8/12/2019 BGP Multihome
43/185
2001, Cisco S ystems, I nc. All rights reserved. 43NANOG 23
Two links to the same ISP
Router A1 Configurationrouter bgp 65534
network 221.10.0.0 mask 255.255.224.0
network 221.10.0.0 mask 255.255.240.0
neighbor 222.222.10.2 remote-as 109
neighbor 222.222.10.2 prefix-list routerC out
neighbor 222.222.10.2 prefix-list default in
!
ip prefix-list defaultpermit 0.0.0.0/0
ip prefix-list routerCpermit 221.10.0.0/20
ip prefix-list routerCpermit 221.10.0.0/19
!
ip route 221.10.0.0 255.255.240.0 null0
ip route 221.10.0.0 255.255.224.0 null0
Router B1 configuration is similar but for the other /20
8/12/2019 BGP Multihome
44/185
2001, Cisco S ystems, I nc. All rights reserved. 44NANOG 23
Multiple Dualhomed Customers
Router C Configurationrouter bgp 109
neighbor bgp-customers peer-group
neighbor bgp-customers remote-as 65534
neighbor bgp-customers default-originate
neighbor bgp-customers prefix-list default out
neighbor 222.222.10.1 peer-groupbgp-customers
neighbor 222.222.10.1 description Customer One
neighbor 222.222.10.1 prefix-list Customer1 in
neighbor 222.222.10.9 peer-groupbgp-customersneighbor 222.222.10.9 description Customer Two
neighbor 222.222.10.9 prefix-list Customer2 in
8/12/2019 BGP Multihome
45/185
2001, Cisco S ystems, I nc. All rights reserved. 45NANOG 23
Multiple Dualhomed Customers
neighbor 222.222.10.17 peer-groupbgp-customers
neighbor 222.222.10.17 description Customer Three
neighbor 222.222.10.17 prefix-list Customer3 in
!
ip prefix-list Customer1permit 221.10.0.0/19 le 20ip prefix-list Customer2permit 221.16.64.0/19 le 20
ip prefix-list Customer3permit 221.14.192.0/19 le 20
ip prefix-list defaultpermit 0.0.0.0/0
Router C only allows in /19 and /20 prefixesfrom customer block
Router D configuration is almost identical
8/12/2019 BGP Multihome
46/185
2001, Cisco S ystems, I nc. All rights reserved. 46NANOG 23
Multiple Dualhomed Customers
Router E Configurationassumes customer address space is not part ofupstreams address block
router bgp 109
neighbor 222.222.10.17 remote-as 110
neighbor 222.222.10.17 remove-private-AS
neighbor 222.222.10.17 prefix-list Customers out
!
ip prefix-list Customerspermit 221.10.0.0/19
ip prefix-list Customerspermit 221.16.64.0/19
ip prefix-list Customerspermit 221.14.192.0/19
Private AS still visible inside AS109
8/12/2019 BGP Multihome
47/185
2001, Cisco S ystems, I nc. All rights reserved. 47NANOG 23
Multiple Dualhomed Customers
If customers prefixes come fromISPs address block
do NOT announce them to the Internet
announce ISP aggregate only Router E configuration:router bgp 109
neighbor 222.222.10.17 remote-as 110
neighbor 222.222.10.17 prefix-listmy-aggregate out!
ip prefix-listmy-aggregatepermit 221.8.0.0/13
8/12/2019 BGP Multihome
48/185
2001, Cisco S ystems, I nc. All rights reserved. 48NANOG 23
BGP Multihoming Techniques
Definition & Options
Connecting to the same ISP
Connecting to different ISPs
Service Provider Multihoming
Using Communities
Case Study
8/12/2019 BGP Multihome
49/185
49 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Multihoming to different ISPs
8/12/2019 BGP Multihome
50/185
2001, Cisco S ystems, I nc. All rights reserved. 50NANOG 23
Two links to different ISPs
Use a Public AS
or use private AS if agreed with the other ISP
Address space comes from
both upstreams or
Regional Internet Registry
Configuration concepts very similar
8/12/2019 BGP Multihome
51/185
51 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to different ISPs
One link primary, the other link backup only
8/12/2019 BGP Multihome
52/185
Two links to different ISPs (one as
8/12/2019 BGP Multihome
53/185
2001, Cisco S ystems, I nc. All rights reserved. 53NANOG 23
AS 109 AS 108
AS 107
CC DD
backup only)
Announce /19 block
with longer AS PATH
Internet
Announce /19 block
BBAA
Two links to different ISPs (one as
8/12/2019 BGP Multihome
54/185
2001, Cisco S ystems, I nc. All rights reserved. 54NANOG 23
backup only)
Router A Configurationrouter bgp 107
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 109
neighbor 222.222.10.1 prefix-list aggregate out
neighbor 222.222.10.1 prefix-list default in
!
ip prefix-list aggregatepermit 221.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
Two links to different ISPs (one as
8/12/2019 BGP Multihome
55/185
2001, Cisco S ystems, I nc. All rights reserved. 55NANOG 23
backup only)
Router B Configurationrouter bgp 107
network 221.10.0.0 mask 255.255.224.0
neighbor 220.1.5.1 remote-as 108
neighbor 220.1.5.1 prefix-list aggregate out
neighbor 220.1.5.1 route-map routerD-out out
neighbor 220.1.5.1 prefix-list default in
neighbor 220.1.5.1 route-map routerD-in in
!
ip prefix-list aggregatepermit 221.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
route-map routerD-outpermit 10
set as-path prepend 107 107 107
!
route-map routerD-inpermit 10
set local-preference 80
8/12/2019 BGP Multihome
56/185
56 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to different ISPs
With Loadsharing
Two links to different ISPs (withl d h i )
8/12/2019 BGP Multihome
57/185
2001, Cisco S ystems, I nc. All rights reserved. 57NANOG 23
loadsharing)
Announce /19 aggregate on each link
Split /19 and announce as two /20s, one on
each linkbasic inbound loadsharing
When one link fails, the announcement of the/19 aggregate via the other ISP ensures
continued connectivity
Two links to different ISPs (with
8/12/2019 BGP Multihome
58/185
2001, Cisco S ystems, I nc. All rights reserved. 58NANOG 23
AS 109 AS 108
AS 107
CC DD
loadsharing)
Announce second
/20 and /19 block
Internet
Announce first
/20 and /19 blockBBAA
Two links to different ISPs (withl d h i )
8/12/2019 BGP Multihome
59/185
2001, Cisco S ystems, I nc. All rights reserved. 59NANOG 23
loadsharing)
Router A Configurationrouter bgp 107
network 221.10.0.0 mask 255.255.224.0
network 221.10.0.0 mask 255.255.240.0
neighbor 222.222.10.1 remote-as 109
neighbor 222.222.10.1 prefix-list firstblock out
neighbor 222.222.10.1 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
!ip prefix-list firstblockpermit 221.10.0.0/20
ip prefix-list firstblockpermit 221.10.0.0/19
Two links to different ISPs (withloadsharing)
8/12/2019 BGP Multihome
60/185
2001, Cisco S ystems, I nc. All rights reserved. 60NANOG 23
loadsharing)
Router B Configurationrouter bgp 107
network 221.10.0.0 mask 255.255.224.0
network 221.10.16.0 mask 255.255.240.0
neighbor 220.1.5.1 remote-as 108
neighbor 220.1.5.1 prefix-list secondblock out
neighbor 220.1.5.1 prefix-list default in
!
ip prefix-list defaultpermit 0.0.0.0/0
!ip prefix-list secondblockpermit 221.10.16.0/20
ip prefix-list secondblockpermit 221.10.0.0/19
8/12/2019 BGP Multihome
61/185
61 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to different ISPs
More Controlled Loadsharing
Loadsharing with different ISPs
8/12/2019 BGP Multihome
62/185
2001, Cisco S ystems, I nc. All rights reserved. 62NANOG 23
Loadsharing with different ISPs
Announce /19 aggregate on each linkOn first link, announce /19 as normal
On second link, announce /19 with longer ASPATH, and announce one /20 subprefix
controls loadsharing between upstreams and theInternet
Vary the subprefix size and AS PATH length
until perfect loadsharing achieved
Still require redundancy!
Loadsharing with different ISPs
8/12/2019 BGP Multihome
63/185
2001, Cisco S ystems, I nc. All rights reserved. 63NANOG 23
AS 109 AS 108
AS 107
CC DD
Loadsharing with different ISPs
Announce /20 subprefix, and
/19 block with longer AS path
Internet
Announce /19 block
BBAA
Loadsharing with different ISPs
8/12/2019 BGP Multihome
64/185
2001, Cisco S ystems, I nc. All rights reserved. 64NANOG 23
Loadsharing with different ISPs
Router A Configurationrouter bgp 107
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 109
neighbor 222.222.10.1 prefix-list default in
neighbor 222.222.10.1 prefix-list aggregate out
!
ip prefix-list aggregatepermit 221.10.0.0/19
Loadsharing with different ISPs
8/12/2019 BGP Multihome
65/185
2001, Cisco S ystems, I nc. All rights reserved. 65NANOG 23
Loadsharing with different ISPs
Router B Configurationrouter bgp 107network 221.10.0.0 mask 255.255.224.0
network 221.10.16.0 mask 255.255.240.0
neighbor 220.1.5.1 remote-as 108
neighbor 220.1.5.1 prefix-list default in
neighbor 220.1.5.1 prefix-list subblocks outneighbor 220.1.5.1 route-map routerD out
!
route-map routerDpermit 10
match ip address prefix-list aggregate
set as-path prepend 107 107
route-map routerDpermit 20
!
ip prefix-list subblockspermit 221.10.0.0/19 le 20
ip prefix-list aggregatepermit 221.10.0.0/19
BGP Multihoming Techniques
8/12/2019 BGP Multihome
66/185
2001, Cisco S ystems, I nc. All rights reserved. 66NANOG 23
BGP Multihoming Techniques
Definition & Options
Connecting to the same ISP
Connecting to different ISPs
Service Provider Multihoming
Using Communities
Case Study
8/12/2019 BGP Multihome
67/185
67 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
Service Provider Multihoming
8/12/2019 BGP Multihome
68/185
2001, Cisco S ystems, I nc. All rights reserved. 68NANOG 23
Service Provider Multihoming
Previous examples dealt with loadsharinginbound traffic
Of primary concern at Internet edge
What about outbound traffic?
Transit ISPs strive to balance traffic flows in bothdirections
Balance link utilisation
Try and keep most traffic flows symmetric
Service Provider Multihoming
8/12/2019 BGP Multihome
69/185
2001, Cisco S ystems, I nc. All rights reserved. 69NANOG 23
Service Provider Multihoming
Balancing outbound traffic requires inboundrouting information
Common solution is full routing table
Rarely necessaryWhy use the routing mallet to try solve loadsharingproblems?
Keep It Simple is often easier (and $$$ cheaper) thancarrying N-copies of the full routing table
Service Provider Multihoming
8/12/2019 BGP Multihome
70/185
2001, Cisco S ystems, I nc. All rights reserved. 70NANOG 23
Service Provider Multihoming
Examples
One upstream, one local peer
One upstream, local exchange point
Two upstreams, one local peer
Tier-1 and regional upstreams, with local peers
Disconnected Backbone
IDC Multihoming
All examples require BGP and a public ASN
8/12/2019 BGP Multihome
71/185
71 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
One Upstream, One local peerOne Upstream, One local peer
One Upstream, One Local Peer
8/12/2019 BGP Multihome
72/185
2001, Cisco S ystems, I nc. All rights reserved. 72NANOG 23
p ,
Announce /19 aggregate on each link
Accept default route only from upstream
Either 0.0.0.0/0 or a network which can be used asdefault
Accept all routes from local peer
One Upstream, One Local Peer
8/12/2019 BGP Multihome
73/185
2001, Cisco S ystems, I nc. All rights reserved. 73NANOG 23
p ,
AS 109
CC
AA
Upstream ISP
AS107
Local Peer
AS108
One Upstream, One Local Peer
8/12/2019 BGP Multihome
74/185
2001, Cisco S ystems, I nc. All rights reserved. 74NANOG 23
p ,
Router A Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.2 remote-as 108
neighbor 222.222.10.2 prefix-listmy-block out
neighbor 222.222.10.2 prefix-listAS108-peer in
!
ip prefix-listAS108-peerpermit 222.5.16.0/19
ip prefix-listAS108-peerpermit 221.240.0.0/20
ip prefix-listmy-blockpermit 221.10.0.0/19
!
ip route 221.10.0.0 255.255.224.0 null0
One Upstream, One Local Peer
8/12/2019 BGP Multihome
75/185
2001, Cisco S ystems, I nc. All rights reserved. 75NANOG 23
p
Router A Alternative Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.2 remote-as 108
neighbor 222.222.10.2 prefix-listmy-block out
neighbor 222.222.10.2 filter-list 10 in
!
ip as-path access-list 10permit ^(108_)+$
!
ip prefix-listmy-blockpermit 221.10.0.0/19
!
ip route 221.10.0.0 255.255.224.0 null0
One Upstream, One Local Peer
8/12/2019 BGP Multihome
76/185
2001, Cisco S ystems, I nc. All rights reserved. 76NANOG 23
Router C Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list default inneighbor 222.222.10.1 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
One Upstream, One Local Peer
8/12/2019 BGP Multihome
77/185
2001, Cisco S ystems, I nc. All rights reserved. 77NANOG 23
Two configurations possible for Router A
Filter-lists assume peer knows what they aredoing
Prefix-list higher maintenance, but saferSome ISPs use both
Local traffic goes to and from local peer,everything else goes to upstream
8/12/2019 BGP Multihome
78/185
78 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
78
One Upstream, Local Exchange PointOne Upstream, Local Exchange Point
8/12/2019 BGP Multihome
79/185
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
80/185
2001, Cisco S ystems, I nc. All rights reserved. 80NANOG 23
AS 109
CC
AA
Upstream ISP
AS107
IXP
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
81/185
2001, Cisco S ystems, I nc. All rights reserved. 81NANOG 23
Router A Configurationinterface fastethernet 0/0
description Exchange Point LAN
ip address 220.5.10.1 mask 255.255.255.224
ip verify unicast reverse-path
no ip directed-broadcast
no ip proxy-arpno ip redirects
!
router bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor ixp-peers peer-group
neighbor ixp-peers soft-reconfiguration in
neighbor ixp-peers prefix-listmy-block out
..next slide
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
82/185
2001, Cisco S ystems, I nc. All rights reserved. 82NANOG 23
neighbor 220.5.10.2 remote-as 100
neighbor 222.5.10.2 peer-group ixp-peers
neighbor 222.5.10.2 prefix-listpeer100 in
neighbor 220.5.10.3 remote-as 101
neighbor 222.5.10.3 peer-group ixp-peers
neighbor 222.5.10.3 prefix-listpeer101 inneighbor 220.5.10.4 remote-as 102
neighbor 222.5.10.4 peer-group ixp-peers
neighbor 222.5.10.4 prefix-listpeer102 in
neighbor 220.5.10.5 remote-as 103
neighbor 222.5.10.5 peer-group ixp-peers
neighbor 222.5.10.5 prefix-listpeer103 in
..next slide
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
83/185
2001, Cisco S ystems, I nc. All rights reserved. 83NANOG 23
ip route 221.10.0.0 255.255.224.0 null0
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-listpeer100permit 222.0.0.0/19
ip prefix-listpeer101permit 222.30.0.0/19
ip prefix-listpeer102permit 222.12.0.0/19
ip prefix-listpeer103permit 222.18.128.0/19
!
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
84/185
2001, Cisco S ystems, I nc. All rights reserved. 84NANOG 23
Router C Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list default in
neighbor 222.222.10.1 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
One Upstream, Local Exchange Point
8/12/2019 BGP Multihome
85/185
2001, Cisco S ystems, I nc. All rights reserved. 85NANOG 23
Note Router A configuration
Prefix-list higher maintenance, but safer
uRPF on the FastEthernet interface
IXP traffic goes to and from local IXP,everything else goes to upstream
8/12/2019 BGP Multihome
86/185
86 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
86
Two Upstreams, One local peerTwo Upstreams, One local peer
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
87/185
2001, Cisco S ystems, I nc. All rights reserved. 87NANOG 23
Announce /19 aggregate on each link
Accept default route only from upstreams
Either 0.0.0.0/0 or a network which can be used asdefault
Accept all routes from local peer
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
88/185
2001, Cisco S ystems, I nc. All rights reserved. 88NANOG 23
AS 109
CC
AA
Upstream ISP
AS106
Local PeerAS108 DD
Upstream ISP
AS107
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
89/185
2001, Cisco S ystems, I nc. All rights reserved. 89NANOG 23
Router A
Same routing configuration as in example withone upstream and one local peer
Same hardware configuration
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
90/185
2001, Cisco S ystems, I nc. All rights reserved. 90NANOG 23
Router C Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list default in
neighbor 222.222.10.1 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
91/185
2001, Cisco S ystems, I nc. All rights reserved. 91NANOG 23
Router D Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.5 remote-as 106
neighbor 222.222.10.5 prefix-list default in
neighbor 222.222.10.5 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
92/185
2001, Cisco S ystems, I nc. All rights reserved. 92NANOG 23
This is the simple configuration for Router C andD
Traffic out to the two upstreams will take nearestexit
Inexpensive routers required
This is not useful in practice especially for internationallinks
Loadsharing needs to be better
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
93/185
2001, Cisco S ystems, I nc. All rights reserved. 93NANOG 23
Better configuration options:
Accept full routing from both upstreams
Expensive & unnecessary!
Accept default from one upstream and someroutes from the other upstream
The way to go!
Two Upstreams, One Local Peer FullRoutes
8/12/2019 BGP Multihome
94/185
2001, Cisco S ystems, I nc. All rights reserved. 94NANOG 23
Router C Configuration
router bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list rfc1918-deny in
neighbor 222.222.10.1 prefix-listmy-block out
neighbor 222.222.10.1 route-mapAS107-loadshare in
!
ip prefix-listmy-blockpermit 221.10.0.0/19
! See earlier presentation for RFC1918 list
..next slide
Two Upstreams, One Local Peer FullRoutes
8/12/2019 BGP Multihome
95/185
2001, Cisco S ystems, I nc. All rights reserved. 95NANOG 23
ip route 221.10.0.0 255.255.224.0 null0
!
ip as-path access-list 10 permit ^(107_)+$
ip as-path access-list 10 permit ^(107_)+_[0-9]+$
!
route-mapAS107-loadsharepermit 10
match ip as-path 10
set local-preference 120
route-mapAS107-loadsharepermit 20
set local-preference 80
!
Two Upstreams, One Local Peer FullRoutes
8/12/2019 BGP Multihome
96/185
2001, Cisco S ystems, I nc. All rights reserved. 96NANOG 23
Router D Configuration
router bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.5 remote-as 106
neighbor 222.222.10.5 prefix-list rfc1918-deny in
neighbor 222.222.10.5 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
! See earlier in presentation for RFC1918 list
Two Upstreams, One Local Peer FullRoutes
8/12/2019 BGP Multihome
97/185
2001, Cisco S ystems, I nc. All rights reserved. 97NANOG 23
Router C configuration:Accept full routes from AS107
Tag prefixes originated by AS107 and AS107s neighbouringASes with local preference 120
Traffic to those ASes will go over AS107 link
Remaining prefixes tagged with local preference of 80
Traffic to other all other ASes will go over the link toAS106
Router D configuration same as Router C withoutthe route-map
Two Upstreams, One Local Peer FullRoutes
8/12/2019 BGP Multihome
98/185
2001, Cisco S ystems, I nc. All rights reserved. 98NANOG 23
Full routes from upstreams
Expensive needs 128Mbytes RAM today
Need to play preference games
Previous example is only an example real lifewill need improved fine-tuning!
Previous example doesnt consider inboundtraffic see earlier presentation for examples
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
99/185
2001, Cisco S ystems, I nc. All rights reserved. 99NANOG 23
Router C Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list rfc1918-nodef-deny inneighbor 222.222.10.1 prefix-listmy-block out
neighbor 222.222.10.1 filter-list 10 in
neighbor 222.222.10.1 route-map tag-default-low in
!
..next slide
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
100/185
2001, Cisco S ystems, I nc. All rights reserved. 100NANOG 23
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
!
ip as-path access-list 10permit ^(107_)+$
ip as-path access-list 10permit ^(107_)+_[0-9]+$
!
route-map tag-default-low permit 10
match ip address prefix-list default
set local-preference 80
route-map tag-default-low permit 20
!
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
101/185
2001, Cisco S ystems, I nc. All rights reserved. 101NANOG 23
Router D Configurationrouter bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.5 remote-as 106
neighbor 222.222.10.5 prefix-list default in
neighbor 222.222.10.5 prefix-listmy-block out
!
ip prefix-listmy-blockpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
ip route 221.10.0.0 255.255.224.0 null0
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
102/185
2001, Cisco S ystems, I nc. All rights reserved. 102NANOG 23
Router C configuration:
Accept full routes from AS107
(or get them to send less)
Filter ASNs so only AS107 and AS107s neighbouring ASesare accepted
Allow default, and set it to local preference 80
Traffic to those ASes will go over AS107 link
Traffic to other all other ASes will go over the link to AS106
If AS106 link fails, backup via AS107 and vice-versa
8/12/2019 BGP Multihome
103/185
Two Upstreams, One Local Peer
8/12/2019 BGP Multihome
104/185
2001, Cisco S ystems, I nc. All rights reserved. 104NANOG 23
When upstreams cannot or will notannounce default route
Because of operational policy against using
default-originate on BGP peeringSolution is to use IGP to propagate defaultfrom the edge/peering routers
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
105/185
2001, Cisco S ystems, I nc. All rights reserved. 105NANOG 23
Router C Configurationrouter ospf 109default-information originate metric 30
passive-interface Serial 0/0
!
router bgp 109
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.1 remote-as 107
neighbor 222.222.10.1 prefix-list rfc1918-deny in
neighbor 222.222.10.1 prefix-listmy-block out
neighbor 222.222.10.1 filter-list 10 in
!..next slide
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
106/185
2001, Cisco S ystems, I nc. All rights reserved. 106NANOG 23
ip prefix-listmy-blockpermit 221.10.0.0/19
! See earlier for RFC1918 list
!
ip route 221.10.0.0 255.255.224.0 null0
ip route 0.0.0.0 0.0.0.0 serial 0/0 254
!
ip as-path access-list 10permit ^(107_)+$
ip as-path access-list 10permit ^(107_)+_[0-9]+$
!
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
107/185
2001, Cisco S ystems, I nc. All rights reserved. 107NANOG 23
Router D Configurationrouter ospf 109default-information originate metric 10
passive-interface Serial 0/0
!
router bgp 109network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.5 remote-as 106
neighbor 222.222.10.5 prefix-list deny-all in
neighbor 222.222.10.5 prefix-listmy-block out
!
..next slide
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
108/185
2001, Cisco S ystems, I nc. All rights reserved. 108NANOG 23
ip prefix-list deny-all deny 0.0.0.0/0 le 32
ip prefix-listmy-blockpermit 221.10.0.0/19
! See earlier presentation for RFC1918 list
!
ip route 221.10.0.0 255.255.224.0 null0
ip route 0.0.0.0 0.0.0.0 serial 0/0 254
!
Two Upstreams, One Local Peer Partial Routes
8/12/2019 BGP Multihome
109/185
2001, Cisco S ystems, I nc. All rights reserved. 109NANOG 23
Partial routes from upstreams
Use OSPF to determine outbound path
Router D default has metric 10 primary outbound path
Router C default has metric 30 backup outbound path
Serial interface goes down, static default is removedfrom routing table, OSPF default withdrawn
8/12/2019 BGP Multihome
110/185
110 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
110
Two Tier-1 upstreams, two regional upstreams, and local peersTwo Tier-1 upstreams, two regional upstreams, and local peers
Tier-1 & Regional Upstreams, LocalPeers
8/12/2019 BGP Multihome
111/185
2001, Cisco S ystems, I nc. All rights reserved. 111NANOG 23
Announce /19 aggregate on each link Accept partial/default routes from upstreams
For default, use 0.0.0.0/0 or a network which can be usedas default
Accept all routes from local peer
Accept all partial routes from regionalupstreams
This is more complex, but a very typicalscenario
Tier-1 & Regional Upstreams, LocalPeers
8/12/2019 BGP Multihome
112/185
2001, Cisco S ystems, I nc. All rights reserved. 112NANOG 23
AS 109
CC
AA
Upstream ISP
AS106
Local Peer
AS108 DD
Upstream ISP
AS107Regional Upstream
AS110
BB
EEFFRegional Upstream
AS111Local Peers
IXP
Tier-1 & Regional Upstreams, LocalPeers Detail
8/12/2019 BGP Multihome
113/185
2001, Cisco S ystems, I nc. All rights reserved. 113NANOG 23
Router A local private peer
Accept all (local) routes
Local traffic stays local
Use prefix and/or AS-path filters
Use local preference (if needed)
Router F local IXP peering
Accept all (local) routes
Local traffic stays local
Use prefix and/or AS-path filters
Tier-1 & Regional Upstreams, LocalPeers Detail
8/12/2019 BGP Multihome
114/185
2001, Cisco S ystems, I nc. All rights reserved. 114NANOG 23
Router B regional upstreamThey provide transit to Internet, but longer AS paththan Tier-1s
Accept all regional routes from them
e.g. 110_[0-9]+$
Ask them to send default, or send a network you canuse as default
Set local pref on default to 60
Will provide backup to Internet only when direct Tier-1links go down
Tier-1 & Regional Upstreams, LocalPeers Detail
8/12/2019 BGP Multihome
115/185
2001, Cisco S ystems, I nc. All rights reserved. 115NANOG 23
Router E regional upstreamThey provide transit to Internet, but longer AS paththan Tier-1s
Accept all regional routes from them
e.g. 111_[0-9]+$
Ask them to send default, or send a network you canuse as default
Set local pref on default to 70
Will provide backup to Internet only when direct Tier-1links go down
Tier-1 & Regional Upstreams, LocalPeers Detail
8/12/2019 BGP Multihome
116/185
2001, Cisco S ystems, I nc. All rights reserved. 116NANOG 23
Router C first Tier-1
Accept all their customer and AS neighbour routesfrom them
e.g. 107_[0-9]+$
Ask them to send default, or send a network you canuse as default
Set local pref on default to 80
Will provide backup to Internet only when link tosecond Tier-1 goes down
Tier-1 & Regional Upstreams, LocalPeers Detail
8/12/2019 BGP Multihome
117/185
2001, Cisco S ystems, I nc. All rights reserved. 117NANOG 23
Router D second Tier-1
Ask them to send default, or send a networkyou can use as default
This has local preference 100 by default
All traffic without any more specific path willgo out this way
Tier-1 & Regional Upstreams, LocalPeers Summary
8/12/2019 BGP Multihome
118/185
2001, Cisco S ystems, I nc. All rights reserved. 118NANOG 23
Local traffic goes to local peer and IXP
Regional traffic goes to two regionalupstreams
Everything else is shared between the two
Tier-1s
To modify loadsharing tweak what is heardfrom the two regionals and the first Tier-1
Best way is through modifying the AS-path filter
Tier-1 & Regional Upstreams, LocalPeers
8/12/2019 BGP Multihome
119/185
2001, Cisco S ystems, I nc. All rights reserved. 119NANOG 23
What about outbound announcement strategy?
This is to determine incoming traffic flows
/19 aggregate must be announced to everyone!
/20 or /21 more specifics can be used to improve ormodify loadsharing
See earlier for hints and ideas
Tier-1 & Regional Upstreams, LocalPeers
8/12/2019 BGP Multihome
120/185
2001, Cisco S ystems, I nc. All rights reserved. 120NANOG 23
What about unequal circuit capacity?
AS-path filters are very useful
What if upstream will only give me fullrouting table or nothing
AS-path and prefix filters are very useful
8/12/2019 BGP Multihome
121/185
121 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider Multihoming
Disconnected BackboneDisconnected Backbone
Disconnected Backbone
8/12/2019 BGP Multihome
122/185
2001, Cisco S ystems, I nc. All rights reserved. 122NANOG 23
ISP runs large network
Network has no backbone, only large PoPs ineach location
Each PoP multihomes to upstreams
Common in some countries where backbonecircuits are hard to obtain
Disconnected Backbone
8/12/2019 BGP Multihome
123/185
2001, Cisco S ystems, I nc. All rights reserved. 123NANOG 23
CCUpstream
AS108
BB
AA
DD
UpstreamAS110
City One
City Two
City Three
City Four
IXP
IXP
IXP
IXP
8/12/2019 BGP Multihome
124/185
Disconnected Backbone
Router A Configuration
8/12/2019 BGP Multihome
125/185
2001, Cisco S ystems, I nc. All rights reserved. 125NANOG 23
router bgp 109network 221.10.0.0 mask 255.255.248.0
neighbor 222.200.0.1 remote-as 108
neighbor 222.200.0.1 description AS108 Serial 0/0
neighbor 222.200.0.1 prefix-list default in
neighbor 222.222.0.1 prefix-listmy-block out
neighbor 222.222.10.1 remote-as 110
neighbor 222.222.10.1 description AS110 Serial 1/0
neighbor 222.222.10.1 prefix-list rfc1918-sua in
neighbor 222.222.10.1 prefix-listmy-block out
neighbor 222.222.10.1 filter-list 10 in
!
continued on next page
Disconnected Backbone
8/12/2019 BGP Multihome
126/185
2001, Cisco S ystems, I nc. All rights reserved. 126NANOG 23
ip prefix-listmy-blockpermit 221.10.0.0/21
ip prefix-list defaultpermit 0.0.0.0/0
!
ip as-path access-list 10 permit ^(110_)+$
ip as-path access-list 10 permit ^(110_)+_[0-9]+$
!etc to achieve outbound loadsharing
!
ip route 0.0.0.0 0.0.0.0 Serial 1/0 250
ip route 221.10.0.0 255.255.248.0 null0
!
Disconnected Backbone
8/12/2019 BGP Multihome
127/185
2001, Cisco S ystems, I nc. All rights reserved. 127NANOG 23
Peer with AS108Receive just default route
Announce /22 address
Peer with AS110
Receive full routing table filter with AS-path filter
Announce /22 address
Point backup static default distance 252 in caseAS108 goes down
Disconnected Backbone
8/12/2019 BGP Multihome
128/185
2001, Cisco S ystems, I nc. All rights reserved. 128NANOG 23
Default ensures that disconnected parts ofAS109 are reachable
Static route backs up AS108 default
No BGP loop detection relying on default route
Do not announce /19 aggregate
No advantage in announcing /19 and could lead toproblems
8/12/2019 BGP Multihome
129/185
129 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
IDC Multihoming
8/12/2019 BGP Multihome
130/185
Two Upstreams, Two Local Peers IDC
8/12/2019 BGP Multihome
131/185
2001, Cisco S ystems, I nc. All rights reserved. 131NANOG 23
AS 109
CC
AA
Upstream ISP
AS106
Local Peer
AS108 DD
Upstream ISP
AS107
BB
Local Peer
AS110
IDC core
Assigned /24 from AS107 and /23 from AS106.
Circuit to AS107 is 2Mbps, circuit to AS106 is 4Mbps
IDC Multihoming
8/12/2019 BGP Multihome
132/185
2001, Cisco S ystems, I nc. All rights reserved. 132NANOG 23
Router A and B configuration
In: Should accept all routes from AS108 andAS110
Out: Should announce all address space toAS108 and AS110
Straightforward
IDC Multihoming
8/12/2019 BGP Multihome
133/185
2001, Cisco S ystems, I nc. All rights reserved. 133NANOG 23
Router C configurationIn: Accept partial routes from AS107
e.g. ^107_[0-9]+$
In: Ask for a route to use as defaultset local preference on default to 80
Out: Send /24, and send /23 with AS-PATHprepend of one AS
IDC Multihoming
8/12/2019 BGP Multihome
134/185
2001, Cisco S ystems, I nc. All rights reserved. 134NANOG 23
Router D configuration
In: Ask for a route to use as default
Leave local preference of default at 100
Out: Send /23, and send /24 with AS-PATHprepend of one AS
IDC Multihoming Fine Tuning
F l l fi t i i i it it
8/12/2019 BGP Multihome
135/185
2001, Cisco S ystems, I nc. All rights reserved. 135NANOG 23
For local fine tuning, increase circuit capacityLocal circuits usually are cheap
Otherwise
For longer distance fine tuning
In: Modify as-path filter on Router C
Out: Modify as-path prepend on Routers C and D
Outbound traffic flow is usual critical for an IDC soinbound policies need to be carefully thought out
IDC Multihoming Other Details
R d d
8/12/2019 BGP Multihome
136/185
2001, Cisco S ystems, I nc. All rights reserved. 136NANOG 23
RedundancyCircuits are terminated on separate routers
Apply thought to address space use
Request from both upstreams
Utilise address space evenly across IDC
Dont start with /23 then move to /24 use bothblocks at the same time in the same proportion
Helps with loadsharing yes, really!
IDC Multihoming Other Details
8/12/2019 BGP Multihome
137/185
2001, Cisco S ystems, I nc. All rights reserved. 137NANOG 23
What about failover?
/24 and /23 from upstreams blocks announcedto the Internet routing table all the time
No obvious alternative at the momentConditional advertisement can help in steady state,but subprefixes still need to be announced infailover condition
BGP Multihoming Techniques
D fi iti & O ti
8/12/2019 BGP Multihome
138/185
2001, Cisco S ystems, I nc. All rights reserved. 138NANOG 23
Definition & Options
Connecting to the same ISP
Connecting to different ISPs
Service Provider Multihoming Using Communities
Case Study
8/12/2019 BGP Multihome
139/185
139 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Communities
Community usage
8/12/2019 BGP Multihome
140/185
2001, Cisco S ystems, I nc. All rights reserved. 140NANOG 23
RFC1998
Examples of SP applications
RFC1998
8/12/2019 BGP Multihome
141/185
2001, Cisco S ystems, I nc. All rights reserved. 141NANOG 23
Informational RFC
Describes how to implement loadsharing andbackup on multiple inter-AS links
BGP communities used to determine local preference inupstreams network
Gives control to the customer
Simplifies upstreams configuration
simplifies network operation!
RFC1998
8/12/2019 BGP Multihome
142/185
2001, Cisco S ystems, I nc. All rights reserved. 142NANOG 23
Community values defined to haveparticular meanings:
ASx:100 set local pref 100 preferred route
ASx:90 set local pref 90 backup route if dualhomed on ASx
ASx:80 set local pref 80 main link is to another ISP withsame AS path length
ASx:70 set local pref 70 main link is to another ISP
RFC1998
Sample Customer Router Configuration
router bgp 107
8/12/2019 BGP Multihome
143/185
2001, Cisco S ystems, I nc. All rights reserved. 143NANOG 23
router bgp 107
neighbor x.x.x.x remote-as 109
neighbor x.x.x.x description Backup ISP
neighbor x.x.x.x route-map config-community out
neighbor x.x.x.x send-community
!ip as-path access-list 20permit ^$
ip as-path access-list 20 deny .*
!
route-map config-communitypermit 10
match as-path 20set community 109:90
RFC1998
Sample ISP Router Configuration
! Homed to another ISP
8/12/2019 BGP Multihome
144/185
2001, Cisco S ystems, I nc. All rights reserved. 144NANOG 23
! Homed to another ISP
ip community-list 70 permit 109:70
! Homed to another ISP with equal ASPATH length
ip community-list 80 permit 109:80
! Customer backup routes
ip community-list 90 permit 109:90
!
route-map set-customer-local-pref permit 10
match community 70set local-preference 70
RFC1998
Sample ISP Router Configuration
route-map set-customer-local-pref permit 20
8/12/2019 BGP Multihome
145/185
2001, Cisco S ystems, I nc. All rights reserved. 145NANOG 23
route map set customer local pref permit 20
match community 80
set local-preference 80
!
route-map set-customer-local-pref permit 30
match community 90
set local-preference 90
!
route-map set-customer-local-pref permit 40set local-preference 100
RFC1998
S ti RFC1998
8/12/2019 BGP Multihome
146/185
2001, Cisco S ystems, I nc. All rights reserved. 146NANOG 23
Supporting RFC1998
many ISPs do, more should
check AS object in the Internet Routing
Registry
if you do, insert comment in AS object in theIRR
8/12/2019 BGP Multihome
147/185
147 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Two links to the same ISPOne link primary, the other link backup onlyOne link primary, the other link backup only
Two links to the same ISP
CC
primary
8/12/2019 BGP Multihome
148/185
2001, Cisco S ystems, I nc. All rights reserved. 148NANOG 23
AS 109 AS 65534
AACC
AS109 proxy aggregates for AS 65534
DDEEBB
backup
8/12/2019 BGP Multihome
149/185
Two links to the same ISP(one as backup only)
Router A Configuration
8/12/2019 BGP Multihome
150/185
2001, Cisco S ystems, I nc. All rights reserved. 150NANOG 23
grouter bgp 65534
network 221.10.0.0 mask 255.255.224.0
neighbor 222.222.10.2 remote-as 109
neighbor 222.222.10.2 description RouterC
neighbor 222.222.10.2 prefix-list aggregate outneighbor 222.222.10.2 prefix-list default in
!
ip prefix-list aggregatepermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
8/12/2019 BGP Multihome
151/185
Two links to the same ISP(one as backup only)
ip prefix-list aggregate permit 221.10.0.0/19ip prefix-list default permit 0 0 0 0/0
8/12/2019 BGP Multihome
152/185
2001, Cisco S ystems, I nc. All rights reserved. 152NANOG 23
ip prefix-list defaultpermit 0.0.0.0/0
!
route-map routerD-outpermit 10
match ip address prefix-list aggregate
set community 109:90route-map routerD-outpermit 20
!
route-map routerD-inpermit 10
set local-preference 90
!
8/12/2019 BGP Multihome
153/185
Two links to the same ISP(one as backup only)
Router D Configuration (backup link)
router bgp 109
8/12/2019 BGP Multihome
154/185
2001, Cisco S ystems, I nc. All rights reserved. 154NANOG 23
neighbor 222.222.10.5 remote-as 65534
neighbor 222.222.10.5 default-originate
neighbor 222.222.10.5 prefix-list Customer in
neighbor 222.222.10.5 route-mapbgp-cust-in in
neighbor 222.222.10.5 prefix-list default out
!
ip prefix-list Customerpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
!
..next slide
Two links to the same ISP(one as backup only)
ip prefix-list Customerpermit 221.10.0.0/19
ip prefix-list defaultpermit 0.0.0.0/0
8/12/2019 BGP Multihome
155/185
2001, Cisco S ystems, I nc. All rights reserved. 155NANOG 23
p p p
!
ip community-list 90permit 109:90
!
route-mapbgp-cust-inpermit 30
match community 90
set local-preference 90
route-mapbgp-cust-inpermit 40
set local-preference 100
8/12/2019 BGP Multihome
156/185
156 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Service Provider use ofCommunitiesSome working examplesSome working examples
Background
RFC1998 is okay for simple multihomed
8/12/2019 BGP Multihome
157/185
2001, Cisco S ystems, I nc. All rights reserved. 157NANOG 23
RFC1998 is okay for simple multihomedcustomers
assumes that upstreams are interconnected
ISPs create many other communities tohandle more complex situations
Simplify ISP BGP configuration
Give customer more policy control
Some ISP Examples
Public policy is usually listed in the IRR
8/12/2019 BGP Multihome
158/185
2001, Cisco S ystems, I nc. All rights reserved. 158NANOG 23
Following examples are all in the IRR or referencedfrom the AS Object in the IRR
Consider creating communities to give policy
control to customersReduces technical support burden
Reduces the amount of router reconfiguration, and thechance of mistakes
Some ISP ExamplesConnect.com.au
aut-num: AS2764
as-name: ASN-CONNECT-NET
descr: connect.com.au pty ltd
admin-c: CC89tech-c: MP151
8/12/2019 BGP Multihome
159/185
2001, Cisco S ystems, I nc. All rights reserved. 159NANOG 23
tech c: MP151
remarks: Community Definition
remarks: ------------------------------------------------
remarks: 2764:1 Announce to "domestic" rate ASes only
remarks: 2764:2 Don't announce outside local POP
remarks: 2764:3 Lower local preference by 25
remarks: 2764:4 Lower local preference by 15
remarks: 2764:5 Lower local preference by 5
remarks: 2764:6 Announce to non customers with "no-export"
remarks: 2764:7 Only announce route to customers
remarks: 2764:8 Announce route over satellite link
notify: [email protected]
mnt-by: CONNECT-AUchanged: [email protected] 19990506
source: CCAIR
Some ISP ExamplesUUNET Europe
aut-num: AS702
as-name: AS702
descr: UUNET - Commercial IP service provider in Europe
remarks: -------------------------------------------------------------remarks: UUNET uses the following communities with its customers:
8/12/2019 BGP Multihome
160/185
2001, Cisco S ystems, I nc. All rights reserved. 160NANOG 23
g
remarks: 702:80 Set Local Pref 80 within AS702
remarks: 702:120 Set Local Pref 120 within AS702
remarks: 702:20 Announce only to UUNET AS'es and UUNET customers
remarks: 702:30 Keep within Europe, don't announce to other UUNET AS's
remarks: 702:1 Prepend AS702 once at edges of UUNET to Peers
remarks: 702:2 Prepend AS702 twice at edges of UUNET to Peers
remarks: 702:3 Prepend AS702 thrice at edges of UUNET to Peers
remarks: Details of UUNET's peering policy and how to get in touch with
remarks: UUNET regarding peering policy matters can be found at:
remarks: http://www.uu.net/peering/
remarks: --------------------------------------------------------------
mnt-by: UUNET-MNTchanged: [email protected] 20010928
source: RIPE
Some ISP ExamplesConcert Europe
aut-num: AS5400
as-name: CIPCORE
descr: Concert European Core Network
remarks: Communities scheme:remarks: The following BGP communities can be set by Concert BGP
8/12/2019 BGP Multihome
161/185
2001, Cisco S ystems, I nc. All rights reserved. 161NANOG 23
g y
remarks: customers to affect announcements to major peerings.
remarks:
remarks: Community to Community to
remarks: Not announce To peer: AS prepend 5400
remarks:
remarks: 5400:1000 European peers 5400:2000
remarks: 5400:1001 Ebone (AS1755) 5400:2001
remarks: 5400:1002 Eunet (AS286) 5400:2002
remarks: 5400:1003 Unisource (AS3300) 5400:2003
remarks: 5400:1100 US peers 5400:2100
notify: [email protected]: CIP-MNT
source: RIPE
8/12/2019 BGP Multihome
162/185
ISP Examples
Several more
8/12/2019 BGP Multihome
163/185
2001, Cisco S ystems, I nc. All rights reserved. 163NANOG 23
Tiscali is very detailed
Consult their website for more information
Includes IOS configuration examples
Many ISP support communities formultihoming preferences
BGP Multihoming Techniques
Definition & OptionsConnecting to the same ISP
8/12/2019 BGP Multihome
164/185
2001, Cisco S ystems, I nc. All rights reserved. 164NANOG 23
Connecting to the same ISP
Connecting to different ISPs
Service Provider Multihoming Using Communities
Case Study
8/12/2019 BGP Multihome
165/185
165 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Case Study
Case Study Requirements (1)
ISP needs to multihome:
8/12/2019 BGP Multihome
166/185
2001, Cisco S ystems, I nc. All rights reserved. 166NANOG 23
To AS5400 in Europe
To AS2516 in Japan
/19 allocated by APNIC
AS 17660 assigned by APNIC
1Mbps circuits to both upstreams
Case Study Requirements (2)
ISP wants:
8/12/2019 BGP Multihome
167/185
2001, Cisco S ystems, I nc. All rights reserved. 167NANOG 23
Symmetric routing and equal link utilisation in and out(as close as possible)
international circuits are expensive
Has two Cisco 2600 border routers with 64Mbytesmemory
Cannot afford to upgrade memory or hardware on borderrouters or internal routers
Philip, make it work, please
Case Study
Upstream ISP
AS2516
Upstream ISPAS5400
8/12/2019 BGP Multihome
168/185
2001, Cisco S ystems, I nc. All rights reserved. 168NANOG 23
AS 17660
AA
AS2516
BB
ISP Core
Allocated /19 from APNIC
Circuit to AS5400 is 1Mbps, circuit to AS2516 is 1Mbps
Case Study
Both providers stated that routers with128Mbytes memory required for AS17660 toltih
8/12/2019 BGP Multihome
169/185
2001, Cisco S ystems, I nc. All rights reserved. 169NANOG 23
multihome
Wrong!
Full routing table is rarely required or desired
Solution:
Accept default from one upstream
Accept partial prefixes from the other
Case Study Inbound Loadsharing
First cut: Went to a few US LookingGlasses
8/12/2019 BGP Multihome
170/185
2001, Cisco S ystems, I nc. All rights reserved. 170NANOG 23
Checked the AS path to AS5400
Checked the AS path to AS2516
AS2516 was one hop closer
Sent AS-PATH prepend of one AS on AS2516peering
Case Study Inbound Loadsharing
Refinement
Did d
8/12/2019 BGP Multihome
171/185
2001, Cisco S ystems, I nc. All rights reserved. 171NANOG 23
Did not need any
First cut worked, seeing on average 600kbps inboundon each circuit
Does vary according to time of day, but this is asbalanced as it can get, given customer profile
J
Case Study Outbound Loadsharing
First cut:Requested default from AS2516
8/12/2019 BGP Multihome
172/185
2001, Cisco S ystems, I nc. All rights reserved. 172NANOG 23
q
Requested full routes from AS5400
Then looked at my Routing Report
Picked the top 5 ASNs and created a filter-listIf 701, 1, 7018, 1239 or 7046 are in AS-PATH, prefixes arediscarded
Allowed prefixes originated by AS5400 and up to two AS hopsaway
Resulted in 32000 prefixes being accepted in AS17660
8/12/2019 BGP Multihome
173/185
Case StudyConfiguration Router A
router ospf 100
log-adjacency-changes
passive-interface default
no passive-interface Ethernet0/0
8/12/2019 BGP Multihome
174/185
2001, Cisco S ystems, I nc. All rights reserved. 174NANOG 23
p /
default-information originate metric 20
!
router bgp 17660
no synchronization
no bgp fast-external-fallover
bgp log-neighbor-changes
bgp deterministic-med
...next slide
8/12/2019 BGP Multihome
175/185
Case StudyConfiguration Router A
ip as-path access-list 1 deny _701_
ip as-path access-list 1 deny _1_
ip as-path access-list 1 deny _7018_ip as-path access-list 1 deny _1239_
ip as-path access-list 1 deny 7046
8/12/2019 BGP Multihome
176/185
2001, Cisco S ystems, I nc. All rights reserved. 176NANOG 23
p as pat access st de y _ 0 6_
ip as-path access-list 1 deny _209_
ip as-path access-list 1 deny _2914_
ip as-path access-list 1 deny _3549_
ip as-path access-list 1 permit _5400$
ip as-path access-list 1 permit _5400_[0-9]+$
ip as-path access-list 1 permit _5400_[0-9]+_[0-9]+$
ip as-path access-list 1 deny .*
ip as-path access-list 3 permit ^$
!
Case StudyConfiguration Router B
router ospf 100
log-adjacency-changes
passive-interface default
8/12/2019 BGP Multihome
177/185
2001, Cisco S ystems, I nc. All rights reserved. 177NANOG 23
no passive-interface Ethernet0/0
default-information originate
!
router bgp 17660
no synchronization
no auto-summary
no bgp fast-external-fallover
...next slide
Case StudyConfiguration Router B
bgp log-neighbor-changes
bgp deterministic-med
neighbor 210.132.92.165 remote-as 2516
neighbor 210.132.92.165 description eBGP peering
8/12/2019 BGP Multihome
178/185
2001, Cisco S ystems, I nc. All rights reserved. 178NANOG 23
neighbor 210.132.92.165 soft-reconfiguration inbound
neighbor 210.132.92.165 prefix-list default-route in
neighbor 210.132.92.165 prefix-list out-filter out
neighbor 210.132.92.165 route-map as2516-out out
neighbor 210.132.92.165 maximum-prefix 100
neighbor 210.132.92.165 filter-list 2 in
neighbor 210.132.92.165 filter-list 3 out
!
...next slide
Case StudyConfiguration Router B
!
prefix-list default-route permit 0.0.0.0/0
prefix-list out-filter permit 202.144.128.0/19
!
8/12/2019 BGP Multihome
179/185
2001, Cisco S ystems, I nc. All rights reserved. 179NANOG 23
!
ip as-path access-list 2 permit _2516$
ip as-path access-list 2 deny .*
ip as-path access-list 3 permit ^$
!
route-map as2516-out permit 10
set as-path prepend 17660
!
Configuration Summary
Router A
Hears full routing table throws away most of it
AS5400 BGP options are all or nothing
8/12/2019 BGP Multihome
180/185
2001, Cisco S ystems, I nc. All rights reserved. 180NANOG 23
AS5400 BGP options are all or nothing
Static default pointing to serial interface if link goesdown, OSPF default removed
Router BHears default from AS2516
If default disappears (BGP goes down or link goesdown), OSPF default is removed
Case StudyMRTG Graphs
8/12/2019 BGP Multihome
181/185
2001, Cisco S ystems, I nc. All rights reserved. 181NANOG 23
Router B to AS2516
Router A to AS5400
Case Study Summary
Multihoming is not hard, really!
8/12/2019 BGP Multihome
182/185
2001, Cisco S ystems, I nc. All rights reserved. 182NANOG 23
Needs a bit of thought, a bit of planning
Use this case study as an example strategy
Does not require sophisticated equipment, bigmemory, fast CPUs
8/12/2019 BGP Multihome
183/185
183 2001, Cisco S ystems, I nc. All rights reserved.NANOG 23
Summary
Summary
Multihoming is not hard, reallyKeep It Simple!is a very good principle tof ll
8/12/2019 BGP Multihome
184/185
2001, Cisco S ystems, I nc. All rights reserved. 184NANOG 23
follow
Full routing table is rarely required
A default is just as good
If customers want 109k prefixes, charge themmoney for it
8/12/2019 BGP Multihome
185/185
Top Related