Post on 13-Jul-2018
IWAN - Implementing Performance Routing
(PfRv3) Jaromír Pilař– Consulting Systems Engineer, CCIE #2910
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
• IWAN Introduction • IWAN Domain
• Transport Independent Design • IWAN Sites • Components and Roles
• Performance Routing Principles • Policies, Site Discovery, Site Prefix Learning, WAN Interface Discovery • Channels, Traffic Class • Path Selection
• Enterprise Deployment • Conclusion
Agenda
2
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKRST-2362
IWAN Introduction
3
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hybrid WAN: Leveraging the Internet Secure WAN Transport and Internet Access
Branch
MPLS (IP-VPN)
Internet
Private Cloud
Virtual Private Cloud
Public Cloud
Secure WAN transport for private and virtual private cloud access
Leverage local Internet path for public cloud and Internet access
Increased WAN transport capacity, cost effectively!
Improve application performance (right flows to right places)
Secure WAN Transport
Direct Internet Access
• Secure WAN transport for private and virtual private cloud access
• Leverage local Internet path for public cloud and Internet access
• Increased WAN transport capacity; and cost effectively!
• Improve application performance (right flows to right places)
4
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Intelligent WAN Solution Components
MPLS
Unified Branch
3G/4G-LTE
Internet
Private Cloud
Virtual Private Cloud
Public Cloud
Application Optimization
Enhanced Application Visibility and Performance
Secure Connectivity
Comprehensive Threat Defense
Intelligent Path Control
Application Aware Routing
Transport Independent
Simplified Hybrid WAN
Enterprise IWAN - IWAN-App/APIC-EM SP-IWAN - vMS/NSO ORCHESTRATION
5
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN: Intelligent Path Control Performance Routing
Branch
MPLS
Internet
Virtual Private Cloud
Private Cloud
• PfR monitors network performance and routes applications based on application performance policies
• PfR load balances traffic based upon link utilization levels to efficiently utilize all available WAN bandwidth
Other traffic is load balanced to maximize bandwidth
Voice/Video/Critical will be rerouted if the current path degrades below policy thresholds
Voice/Video/Critical take the best delay, jitter, and/or loss path
6
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Master Controller commands path changes based on traffic class policy definitions
Best Path
MC+BR MC+BR BR MC+BR
Path Enforcement
BR BR
MC
Measure the traffic flow and network performance and report metrics to the Master Controller
Performance Measurements
MC+BR MC+BR MC+BR MC+BR
MC
Measurement
MC
BR BR
Border Routers learn current traffic classes going to the WAN based on classifier definitions
Learning Active TCs
MC+BR MC+BR MC+BR MC+BR
Traffic Classes
Learn the Traffic
BR BR
MC
How PfR Works – Key Operations
Define Traffic Classes and service level Policies based on Applications or DSCP
Define Your Traffic Policy
7
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Overlay routing over tunnels
Overlay tunnels (DMVPN)
Internet Routing
Transport routing
Perimeter Security
Perimeter Security
MPLS-VPN Routing
PfR path selection policies
PfR intelligent routing
• CPE-to-CPE overlay enables separation of transport (underlay) and VPN service (overlay)
• Point to multipoint WAN connections with secure tunnel overlay architecture
• Intelligent policy routing to provide cost optimization and dynamic load balancing
AVC/QoS AVC/QoS
IWAN Layered Solution
9
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Components • The Decision Maker: Master Controller (MC)
• Apply policy, verification, reporting • No packet forwarding/ inspection required • Standalone of combined with a BR • VRF Aware • IPv4 only (IPv6 Future)
• The Forwarding Path: Border Router (BR) • Gain network visibility in forwarding path (Learn, measure) • Enforce MC’s decision (path enforcement) • VRF aware • IPv4 only (IPv6 Future)
MC1
BR1 BR2
MC/BR
MC/BR BR
10
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Domain
Site ID 10.3.0.31
MC/BR MC/BR
BR1 BR2 BR3 BR4
MC1 MC2
PATH1 PATH2
DCI WAN Core
DC1 DCn
• Collection of sites that share the same set of policies • An IWAN domain includes:
– A mandatory Hub site, – Optional Transit sites, – As well as Branch sites.
• Each site has a unique identifier (Site-Id) – Derived from the loopback address of the local MC
• Central and headquarter sites play a significant role in PfR and are called an IWAN Point of Presence (POP). – Each of these sites will have a unique identifier called a
POP-ID
• Each site runs PfR and gets its path control configuration and policies from the logical IWAN domain controller through the IWAN Peering Service
Transit Hub
POP1 - HUB Site ID = 10.1.0.10
POP2 - TRANSIT Site ID = 10.2.0.20
Site ID 10.4.0.41
Site ID 10.5.0.51
11
MC/BR BR
IWAN Peering
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hub Site • Located in an enterprise central site or
headquarter location. • Can act as a transit site to access servers in the
datacenters or for spoke-to-spoke traffic • A POP Identifier (POP-ID) 0 is automatically
assigned to a Hub site. • Only one Hub site exists per IWAN domain.
• The logical domain controller functionality resides on this site’s master controller (MC). • The master controller (MC) for this site is known as
the Hub master controller (Hub MC, HMC)
• MCs from all other sites (transit or branch) connect to the Hub MC for PfR configuration and policies.
BR1 BR2 BR3 BR4
MC1
POP1 - HUB Site ID = 10.1.0.10
POP-ID 0
POP2 - TRANSIT Site ID = 10.2.0.20
POP-ID 1
DMVPN MPLS
DMVPN INET
MC2
MC/BR MC/BR MC/BR BR
Branch Branch Branch
Policies Monitors
12
Path MPLS Id 1
Path INET Id 2
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Transit Site • Located in an enterprise central site or
headquarter location. • Can act as a transit site to access servers in
the datacenters or for spoke-to-spoke traffic • A POP Identifier (POP-ID) is configured for
each transit site. This POP-ID has to be unique in the domain.
• The master controller (MC) for this site is known as a Transit Master Controller (Transit MC, TMC)
• The local MC peers with the Hub MC to get its policies, monitor, configuration and timers
BR1 BR2 BR3 BR4
MC1
DMVPN MPLS
DMVPN INET
MC2
MC/BR MC/BR MC/BR BR
Branch Branch Branch
POP1 - HUB Site ID = 10.1.0.10
POP-ID 0
POP2 - TRANSIT Site ID = 10.2.0.20
POP-ID 1
IWAN Peering
13
Path MPLS Id 1
Path INET Id 2
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Branch Site • These will always be a DMVPN spoke,
and are a stub sites where traffic transit is not allowed.
• The local MC peers with the logical domain controller (aka Hub MC) to get its policies, and monitoring guidelines.
BR1 BR2 BR3 BR4
MC1
POP1 - HUB Site ID = 10.1.0.10
POP2 - TRANSIT Site ID = 10.2.0.20
DMVPN MPLS
DMVPN INET
MC2
MC/BR MC/BR MC/BR BR
Branch Branch Branch
IWAN Peering
14
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
WAN Interface Discovery
BR1
MC/BR MC/BR MC/BR BR
Hub MC
BR1 BR3 BR4
MC1 Transit MC MC2 • Hub and Transit BRs have path names
and path identifier manually defined – Path name identifies a Transport – Path Identifier (Path-id) is unique per site
• Hub and Transit BRs send Discovery Packet with path names from to all discovered sites
• Path Discovery from the Hub Border Routers
Path MPLS Path-id 1
Path INET Path-id 2
Path INET Path-id 2
Path MPLS Path-id 1
DMVPN MPLS
DMVPN INET
POP-ID 0 POP-ID 1
WAN Path is detected on the branch - Path Name - POP-ID - Path-Id - DSCP
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
10.3.1.0/24 10.4.1.0/24 10.5.1.0/24
15
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
WAN Interface – Performance Monitors • PfR automatically configures 3 Performance
Monitors instances (PMI) over every external interface • Monitor1 – Site Prefix Learning (egress direction) • Monitor2 – Aggregate Bandwidth per Traffic Class
(egress direction) • Monitor3 – Performance measurements (ingress
direction) BR
2 3 1 2 3 1
16
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Site Prefix Discovery • Every MC in the domain owns a Site Prefix
database • Gives the mapping between site and prefixes • 2 options:
– Static (Hub and Transit sites) – Automatic Learning (Branch sites)
DMVPN INET
DMVPN MPLS
10.3.3.0/24
R31
1 Site 3
10.3.3.0/24 1
17
IWAN Peering
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
10.3.3.0/24
MC/BR
BR1 BR2 BR3 BR4
MC1 MC2
Shared Prefixes (M) • Prefix (10.1.0.0/16 in this example) can
belong to multiple Sites. • Prefix associated with a list of site-ids • Flags:
• S – Learned from SAF (IWAN Peering) • C – Configured • M – Shared
• A TC may be associated with more than 1 site
DMVPN MPLS
DMVPN INET
SITE-ID PREFIXES FLAGS
10.1.0.10 10.1.0.0/16 S,C,M
10.2.0.20 10.2.0.0/16 S,C,M
10.4.0.41 10.4.4.0/24 S
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
R31
Hub MC Transit MC
10.1.0.0/16 10.2.0.0/16
IOS-XE 3.15 IOS 15.5(2)T
10.4.4.0/24
MC/BR
R41
18
10.1.0.0/16 10.2.0.0/16
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Define PfR Traffic Policies
Define your Traffic Policy § Identify Traffic Classes based on Application
or DSCP § Performance thresholds (loss, delay and
Jitter), Preferred Path § Centralized on a Domain Controller
CLASS MATCH ADMIN PERFORMANCE
Voice DSCP Application
Preferred: MPLS Fallback: INET Next Fallback: 4G
Delay threshold Loss threshold Jitter threshold
Interactive Video DSCP Application
Preferred: MPLS Fallback: INET
Delay threshold Loss threshold Jitter threshold
Critical Data DSCP Application
Preferred: MPLS Fallback: INET
Delay threshold Loss threshold Jitter threshold
Best Effort DSCP Application
- Delay threshold Loss threshold Jitter threshold
20
Hub MC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfRv3 works on Traffic Class – DSCP Based
INET MPLS
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
IWAN POP
R10
R11 R12
R31 R41 R51 R52
Traffic with EF, AF41, AF31 and 0
DSCP Based Policies Prefix DSCP AppID Dest Site Next-Hop
10.3.3.0/24 EF N/A Site 3 ? 10.3.3.0/24 AF41 N/A Site 3 ? 10.3.3.0/24 AF31 N/A Site 3 ? 10.3.3.0/24 0 N/A Site 3 ? 10.4.4.0/24 EF N/A Site 4 ? 10.4.4.0/24 AF41 N/A Site 4 ? 10.4.4.0/24 AF31 N/A Site 4 ? 10.4.4.0/24 0 N/A Site 4 ? 10.5.5.0/24 EF N/A Site 5 ? 10.5.5.0/24 AF41 N/A Site 5 ? 10.5.5.0/24 AF31 N/A Site 5 ? 10.5.5.0/24 0 N/A Site 5 ?
Traffic Class § Destination Prefix § DSCP Value § Application (N/A when DSCP policies used)
21
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfRv3 works on Traffic Class– Application Based
INET MPLS
10.3.3.0/24 10.4.4.0/24
IWAN POP
R10
R11 R12
R31 R41 R51 R52
Traffic with EF, AF41, AF31 and 0 App1, App2, etc
Application based Policies Prefix DSCP AppID Dest Site Next-Hop
10.3.3.0/24 EF N/A Site 3 ? 10.3.3.0/24 AF41 App1 Site 3 ? 10.3.3.0/24 AF41 App2 Site 3 ? 10.3.3.0/24 AF41 N/A Site 3 ? 10.3.3.0/24 AF31 N/A Site 3 ? 10.3.3.0/24 0 N/A Site 3 ? 10.4.4.0/24 EF N/A Site 4 ? 10.4.4.0/24 AF41 App1 Site 4 ? 10.4.4.0/24 AF31 N/A Site 4 ? 10.4.4.0/24 0 N/A Site 4 ? 10.5.5.0/24 EF N/A Site 5 ? 10.5.5.0/24 AF41 App2 Site 5 ? 10.5.5.0/24 AF31 N/A Site 5 ? 10.5.5.0/24 0 N/A Site 5 ?
10.5.5.0/24 Traffic Class § Destination Prefix § DSCP Value § Application (N/A when DSCP policies used)
22
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
SITE1
Performance Monitoring
MPLS
INET
Bandwidth on egress Per Traffic Class
(dest-prefix, DSCP, AppName)
2
3 2
User traffic
CPE1 CPE11
CPE12
CPE10
CPE2
Passive Monitoring
Performance Monitor • Collect Performance Metrics • Per Channel - Per DSCP - Per Source and Destination Site - Per Interface
3
SITE3 Single CPE
SITE2 Dual CPE
23
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Performance Monitoring
MPLS
INET
User traffic
Integrated Smart Probes • Traffic driven – intelligent on/off • Site to site and per DSCP
Performance Monitor • Collect Performance Metrics • Per Channel - Per DSCP - Per Source and Destination Site - Per Interface
CPE1 CPE11
CPE12
CPE10
CPE2
Smart Probing
SITE3 Single CPE
SITE2 Dual CPE
SITE1
24
2
3 2
3
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Channels Between Central and Branch Sites
INET MPLS
MC1
BR1 BR3
R10 R11 R12 R13
Hub MC 10.1.0.10/32
Present Channel 10 • Site 1 • DSCP AF41 • MPLS • Path 1
Backup Channel 12 • Site 1 • DSCP AF41 • INET • Path 3
IWAN POP
BR2
Present Channel 11 • Site 1 • DSCP AF41 • MPLS • Path 2
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
25
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Channel Between Branch Sites
INET MPLS
MC1
BR1 BR2
R31 R41 R51 R52
Hub MC 10.8.3.3/32
Present Channel 13 • Site 4 • DSCP EF • MPLS
Backup Channel 14 • Site 4 • DSCP EF • INET
IWAN POP
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
26
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
SITE1
Performance Violation
MPLS
INET
ALERT – Threshold Crossing Alert (TCA) • From Destination site • Sent to source site • Loss, delay, jitter, unreachable
CPE1 CPE11
CPE12
CPE10
CPE2 SITE3
Single CPE
SITE2 Dual CPE
27
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
SITE1
Policy Decision
MPLS
INET
User traffic
User traffic
CPE1 CPE11
CPE12
CPE10
CPE2
• Reroute Traffic to a Secondary Path
• PfR Dataplane Route control
SITE3 Single CPE
SITE2 Dual CPE
28
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Deploying IWAN Intelligent Path Control
29
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco IWAN Enterprise Management Portfolio
• Customer wants advanced provisioning, life cycle management, and customized policies
• System-wide network consistency assurance
• Lean IT OR IT Network team
Cisco
Prime Infrastructure
• Customer needs customizable IWAN with end-to-end monitoring
• One Assurance across Cisco portfolio from Branch to Datacenter
• IT Network team
Enterprise Network Mgmt and Monitoring
Ecosystem Partners
IWAN App
• Customer wants considerable automation and operational simplicity
• Requirements consistent
with prescriptive IWAN Validated Design
• Lean IT organization
Prescriptive Policy Automation
• Customer looking for advanced monitoring and visualization
• QoS/ PfR/ AVC configuration,
Real-time analytics and network troubleshooting
• IT Network team
Application Aware Performance Mgmt
Advanced Orchestration
30
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – DMVPN • IWAN Prescriptive Design – Transport
Independent Design based on DMVPN • Branch spoke sites establish an IPsec tunnel to
and register with the hub site • Data traffic flows over the DMVPN tunnels • WAN interface IP address used for the tunnel
source address (in a Front-door VRF) • One tunnel per user inside VRF
• Per-tunnel QOS is applied to prevent hub site oversubscription to spoke sites
R31 R41 R51 R52
R11 R12 R21 R22
R10
Site1 Site2
R20
MPLS INET
DCI WAN Core
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
10.1.0.0/16 10.2.0.0/16
31
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
FVRF_SP1 (SP1 routing context)
FVRF_SP2 (SP2 routing context)
Customer routing context (Global table)
Using Front Door VRF Keeping the Default Routes in Separate VRFs vrf definition FVRF_SP1
! address-family ipv4
exit-address-family !
! crypto keyring DMVPN vrf FVRF_SP1
pre-shared-key address 0.0.0.0 0.0.0.0 key cisco123
! Interface Tunnel0
ip address 172.50.1.1 255.255.255.0 ip nhrp authentication HBfR3lpl
ip nhrp map multicast 3.3.3.3 ip nhrp map 172.50.1.254 3.3.3.3
ip nhrp network-id 1 ip nhrp nhs 172.50.1.254
ip nhrp shortcut tunnel source GigabitEthernet0/0
tunnel mode gre multipoint tunnel vrf FVRF_SP1
tunnel protection ipsec profile dmvpn !
Interface GigabitEthernet 0/0 description WAN interface to ISP in vrf
ip address dhcp ip vrf forwarding FVRF_SP1
! Interface GigabitEthernet 0/1
description LAN interface In Global Table
• Different default routes possible within global table and towards SP infrastructure
• Configuration towards SP simplified, allows for simple swap
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Overlay Routing - Which protocol should I use? • IWAN Profiles are based upon BGP and EIGRP for scalability and optimal Intelligent Path
Control
• Intelligent Path Control: • PfR can be used with any routing protocols by relying on the routing table (RIB).
• Requires all valid WAN paths be ECMP so that each valid path is in the RIB. • For BGP and EIGRP, PfR can look into protocol’s topology information to determine both best paths
and secondary paths thus, ECMP is not required.
• PfRv3 always checks for a parent route before being able to control a Traffic Class. Parent route check is done as follows: • Check to see if there is an NHRP shortcut route • If not – Check in the order of BGP, EIGRP, Static and RIB • Make sure that all Border Routers have a route over each external path to the destination sites PfR
will NOT be able to effectively control traffic otherwise.
33
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – EIGRP • Single EIGRP process for Branch, WAN and
POP/hub sites • Extend Hello/Hold timers for WAN • Adjust tunnel interface “delay” to ensure WAN path
preference (MPLS primary, INET secondary) • Hubs
• Disable Split-Horizon • Advertise Site summary, enterprise summary,
default route to spokes • Summary metrics: A summary-metric is used to
reduce computational load on the DMVPN hubs. • Ingress filter on tunnels.
• Spokes • EIGRP Stub-Site functionality builds on stub
functionality that allows a router to advertise itself as a stub to peers on specified WAN interfaces, but allows for it to exchange routes learned on LAN interface 10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
R31 R41
R10
Site1 Site2
R20
MPLS INET
DCI WAN Core
Delay 1000
Delay 25000 Delay 25000 Delay 25000 Delay 25000
Set Tunnel Delay to
influence best path
EIGRP Stub Site
Delay 2000
R11 R12 R21 R22
Delay 24000 Delay 24000
Delay 20000 Delay 1000 Delay 1000
Delay 20000
R51 R52 Delay 24000
Delay 25000 Delay 25000
34
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN Deployment – BGP • A single iBGP routing domain is used • Appropriate Hello/Hold timers for WAN • Hub
• DMVPN hub routers function as BGP route-reflectors for the spokes.
• No BGP peering between RR. • BGP dynamic peer feature configured on the route-reflectors • Site specific prefixes, Enterprise summary prefix and default route
advertised to spokes • Set local preference for all prefixes • Redistribute BGP into local IGP with a defined metric cost to
attract traffic from the central sites to the spokes across MPLS. • Spokes
• Peer to Hub/Transit BRs in each DMVPN cloud • Mutual redistribution OSPF/BGP • Set a route tag to identify routes redistributed from BGP • Preferred path is MPLS due to highest Local Preference
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
R31 R41 R51 R52
R11 R12 R21 R22
R10
Site1 Site2
R20
MPLS INET
DCI WAN Core
LP 100000 LP 3000 LP 20000 LP 400
OSPF Metric: 1000 Metric: 2000
OSPF
OSPF
Metric: 1000 Metric: 2000
35
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Deployment – Hub domain IWAN vrf default master hub source-interface Loopback0 enterprise-prefix prefix-list ENTERPRISE_PREFIX site-prefixes prefix-list SITE_PREFIX
domain IWAN vrf default border master 10.1.0.10 source-interface Loopback0 ! interface Tunnel100 description -- Primary Path -- domain IWAN path MPLS path-id 1 <zero-sla> <path-last-resort>
MC R10
BR R11
Hub MC
R31 R41 R51 R52
R11 R12 R21 R22
R10
Path MPLS Id 1
Path INET Id 2
R20
DMVPN MPLS
DMVPN INET
POP ID 0
HUB SITE Site ID = 10.1.0.10
• Enterprise Prefix: summary prefix for the entire domain • Site Prefix: Disable automatic learning – Mandatory • POP Id is 0 • Path ID unique per Site
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
36
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Deployment – Transit Site domain IWAN vrf default master transit 1 source-interface Loopback0 site-prefixes prefix-list SITE_PREFIX hub 10.1.0.10
domain IWAN vrf default border master 10.2.0.20 source-interface Loopback0 ! interface Tunnel100 description -- Primary Path -- domain IWAN path MPLS path-id 1 <zero-sla> <path-last-resort>
MC R20
BR R21
R31 R41 R51 R52
R11 R12 R21 R22
R10
Path MPLS Id 1
Path INET Id 2
Transit MC R20
DMVPN MPLS
DMVPN INET
POP ID 1
TRANSIT SITE Site ID = 10.2.0.20
• Site Prefix: Disable automatic learning – Mandatory • POP Id unique per domain • Path ID unique per Site • Peering with Hub MC
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
HUB SITE Site ID = 10.1.0.10
37
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
R31 R41 R51 R52
R11 R12 R21 R22
R10 R20
DMVPN MPLS
DMVPN INET
TRANSIT SITE Site ID = 10.2.0.20
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
HUB SITE Site ID = 10.1.0.10
PfR Deployment – Single CPE Branch
• MC/BR colocated • Branch MCs connect to the Hub
domain IWAN vrf default master branch source-interface Loopback0 hub 10.1.0.10 border master local source-interface Loopback0
R31 R41
38
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
R31 R41
R11 R12 R21 R22
R10 R20
DMVPN MPLS
DMVPN INET
TRANSIT SITE Site ID = 10.2.0.20
10.3.3.0/24 10.4.4.0/24 10.5.5.0/24
HUB SITE Site ID = 10.1.0.10
PfR Deployment – Dual CPE Branch
• Branch MCs connect to the Hub • Make sure there is a direct connection
between BRs
domain IWAN vrf default border master 10.5.0.51 source-interface Loopback0
R52
domain IWAN vrf default master branch source-interface Loopback0 hub 10.1.0.10 border master local source-interface Loopback0
R51
39
R51 R52
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Policies – DSCP or App Based domain IWAN vrf default
master hub load-balance
class MEDIA sequence 10 match application <APP-NAME1> policy real-time-video
match application <APP-NAME2> policy custom
priority 1 one-way-delay threshold 200 priority 2 loss threshold 1
path-preference MPLS fallback INET
class VOICE sequence 20 match dscp <DSCP-VALUE> policy voice
path-preference MPLS fallback INET class CRITICAL sequence 30
match dscp af31 policy low-latency-data
R83
• Pre-defined thresholds
• Custom thresholds
• When load balancing is enabled, PfRv3 adds a “default class for match all DSCP (lowest priority compared to all the other classes)” and PfRv3 controls this traffic.
• When load balancing is disabled, PfRv3 deletes this “default class” and as a part of that frees up the TCs that was learnt as a part of LB – they follow the routing table
40
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Policies – Built-in Policy Templates Pre-defined
Template Threshold Definition
Voice priority 1 one-way-delay threshold 150 threshold 150 (msec) priority 2 packet-loss-rate threshold 1 (%) priority 2 byte-loss-rate threshold 1 (%) priority 3 jitter 30 (msec)
Real-time-video priority 1 packet-loss-rate threshold 1 (%) priority 1 byte-loss-rate threshold 1 (%) priority 2 one-way-delay threshold 150 (msec) priority 3 jitter 20 (msec)
Low-latency-data priority 1 one-way-delay threshold 100 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%)
Pre-defined
Template
Threshold Definition
Bulk-data priority 1 one-way-delay threshold 300 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%)
Best-effort priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 10 (%) priority 2 packet-loss-rate threshold 10 (%)
scavenger priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 50 (%) priority 2 packet-loss-rate threshold 50 (%)
For Your Reference
41
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Policies – Transit Site Preference • Transit Site Preference is used in the context of a Multiple Transit Site
deployment with the same set of prefixes advertised from all central sites. • A specific Transit site is preferred for a specific prefix, as long as there are available ‘in
policy’ channels for this site. • Based on routing metrics and advertised mask length in routing
• Transit Site preference is a higher priority filter and takes precedence over path-preference.
• Transit Site Affinity introduced in 15.5(3)M1 and XE 3.16.1
domain IWAN vrf default master hub advanced no transit-site-affinity
Transit Site Affinity is enabled by default. To disable use:
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfR Policies – Path Preference • With Path Preference configured, PfR will then first consider all the links
belonging to the preferred path preference (i.e it will include the active and the standby links belonging to the preferred path) and will then use the fallback provider links.
• Without Path Preference configured PfR will give preference to the active channels and then the standby channels (active/standby will be per prefix) with respect to the performance and policy decisions • Note that the Active and Standby channels per prefix will span across the POP’s. • Spoke will randomly (hash) choose the active channel
43
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Load Balancing • Current Situation - Load balancing works on physical links - Load sharing on NH on the same DMVPN network
(XE 3.16.1 and IOS 15.5(3)M1) : - between R11 and R21 - between R12 and R22
• Default Classes TCs - Load balancing at any time (not only at creation
time). - TC will be moved to ensure bandwidth on all links is
within the defined range
• Performance TCs - Initial load-balancing while placing the TCs, on a per
TC basis. PfR does not account for the TCs getting fatter.
Hub MC
R11 R12 R21 R22
R10
Path MPLS Id 1
Path INET Id 2
R20 POP ID 0
Transit MC
POP ID 1
Path MPLS Id 1
Path INET Id 2
R31
MPLS INET
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
10.1.0.0/16 10.2.0.0/16
10.1.0.0/16 10.2.0.0/16
44
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Unreachable Timer
10.3.3.0/24
Hub MC
R31
R11 R12 R21 R22
R10
Path MPLS Id 1
Path MPLS Id 1
Transit MC R20
DMVPN MPLS
DMVPN INET
POP ID 0 POP ID 1
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
Path INET Id 2
Path INET Id 2
• Channel Unreachable • PfRv3 considers a channel reachable as long as
the site receives a PACKET on that channel • A channel is declared unreachable in both
direction if • There is NO traffic on the Channel, probes are the only way
of detecting unreachability. So if no probe is received within 1 sec, PfR detects unreachability.
• When there IS traffic on the channel, if PfR does not see any packet for more than a second on a channel PfR detects unreachability.
Default: 1 Sec Recommended: 4 sec Advanced options – with 3.16 15.5(3)S / 15.5(3)M channel-unreachable-timer 4
45
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Failover Time Hub MC
R11 R12 R21 R22
R10
Path MPLS Id 1
Path MPLS Id 1
Transit MC R20
DMVPN MPLS
DMVPN INET
POP ID 0 POP ID 1
Path INET Id 2
Path INET Id 2
• Ingress Performance Violation detected • Delay, loss or jitter thresholds • Based on Monitor-interval
domain IWAN vrf default master hub monitor-interval 4 dscp ef monitor-interval 4 dscp af41 monitor-interval 4 dscp cs4 monitor-interval 4 dscp af31
R31
10.3.3.0/24
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
46
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Path Selection Direction from POPs to Spokes
• Each POP is a unique site by itself and so it will only control traffic towards the spoke on the WAN’s that belong to that POP.
• PfRv3 will NOT be redirecting traffic between POP across the DCI or WAN Core. If it is required that all the links are considered from POP to spoke, then the customer will need to use a single MC.
• Only one next hop (on branch) per DMVPN network
• No PfR control between Transit Sites
R11 R12 R21 R22
R10
Path MPLS Id 1
Path MPLS Id 1
R20
DMVPN MPLS
DMVPN INET
Path INET Id 2
Path INET Id 2
R31
10.3.3.0/24
HUB SITE Site ID = 10.1.0.10
TRANSIT SITE Site ID = 10.2.0.20
Hub MC POP-ID 0
Transit MC POP-ID 1
47
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Path Selection Direction from Spokes to POPs
• The spoke considers all the paths (multiple NH’s) towards the POPs
• The concept of "active" and "standby" next hops based on routing metrics and advertised mask length in routing is used to gather information about the preferred POP for a given prefix.
• Example: If the best metric for a given prefix is on DC1 then all the next hops on that DC for all the ISPs are tagged as active (only for that prefix).
R11 R12 R21 R22
R10
Path MPLS Id 1
Path MPLS Id 1
R20
DMVPN MPLS
DMVPN INET
Path INET Id 2
Path INET Id 2
R31
10.3.3.0/24
DC1 Site ID = 10.1.0.10
DC2 Site ID = 10.2.0.20
LP 100000 LP 3000 LP 20000 LP 400
10.1.0.0/24 10.1.0.0/24
48
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
NetFlow – PfRv3 Exporter Configuration
• Enable exporter on the Hub MC • Distributed through SAF to all MCs and BRs in
the domain • Cisco Prime Infrastructure 3.0 • LiveAction 4.3 • All records available at:
• http://docwiki.cisco.com/wiki/PfRv3:Reporting
domain IWAN vrf default master hub collector 10.151.1.95 port 2055
MC1
INET MPLS
Hub MC 10.1.0.10/32
IWAN POP
R10
R11 R12
R31 R41 R51 R52
50
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
PfRv3 Syslogs • Syslog messages for all major PfRv3 events
• Use cisco standard format (Facility-Severity-Mnemonic) for all syslogs with common Facility name 'DOMAIN’
• Add TCA-ID to all syslog to allow correlation of TCA syslog to PFR reaction syslog. If PFR action is not related to TCA then TCA-ID will be 0
• Command '[no] logging' in domain submode default is syslog on
• Distributed through SAF to all MCs and BRs in the domain • http://docwiki.cisco.com/wiki/PfRv3:Syslogs • DOMAIN-2-IME • DOMAIN-2-IME_DETAILS • DOMAIN-4-MC_SHUTDOWN • DOMAIN-5-TCA • DOMAIN-6-TC_CTRL • DOMAIN-5-TC_PATH_CHG • DOMAIN-3-PLR_INT_CFG • DOMAIN-5-MC_STATUS
IOS-XE 3.16 IOS 15.5(3)M
*Jun 1 18:50:41.104: %DOMAIN-5-TC_PATH_CHG: Traffic class Path Changed. Details: Instance=0: VRF=default: Source Site ID=10.8.3.3: Destination Site ID=10.2.11.11: Reason=Delay: TCA-ID=4: Policy Violated=VOICE: TC=[Site id=10.2.11.11, TC ID=6, Site prefix=10.1.11.0/24, DSCP=ef(46), App ID=0]: Original Exit=[CHAN-ID=14, BR-IP=10.8.4.4, DSCP=ef[46], Interface=Tunnel100, Path=MPLS[label=0:0 | 0:1 [0x1]]]: New Exit=[CHAN-ID=13, BR-IP=10.8.5.5, DSCP=ef[46], Interface=Tunnel200, Path=INET[label=0:0 | 0:2 [0x2]]]
51
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Performance Routing – Platform Support
Cisco ISR G2 family 3900-AX 2900-AX 1900-AX
890
Cisco ISR 4000 4400 4300
Cisco ASR-1000
Cisco CSR-1000
MC BR
MC BR
MC BR
MC BR(1)
(1) XE 3.18
53
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
IWAN 2.1 – HUB-MC Scaling
ISR 4431 50 sites
ASR 1001-X 1000 sites
ISR 4451 200 sites
ASR 1002-X 2000 sites
CSR1000v 1 vCPU
200 sites
CSR1000v 2 vCPU
500 sites
54
CSR1000v 4 vCPU
2000 sites
XE 3.16.2
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Performance Routing v3 – Phases IOS 15.4(3)M IOS-XE 3.13
IOS 15.5(1)T IOS-XE 3.14
IOS 15.5(2)T IOS-XE 3.15
IOS 15.5(3)M IOS-XE 3.16
IOS 15.5(3)M1 IOS-XE 3.16.1
• PfR Domain • One touch provisioning • Auto Discovery of sites • NBAR2 support • Passive Monitoring
(performance monitor) • Smart Probing • VRF Awareness • IPv4/IPv6 (Future) • <10 lines of configuration
and centralized
• Zero SLA • WCCP Support
• Transit Sites • Multiple Next Hop per
DMVPN • Multiple POPs • Syslog (TCA) • Show last 5 TCA
• Path of Last Resort • EIGRP IWAN
Simplification (Stub site)
• Transit Site Affinity
• Blackout ~ sub second • Brownout ~ 2 sec • Scale 2000 sites
55
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Key Takeaways • IWAN Intelligent Path Control pillar is based upon Performance
Routing (PfR) • Maximizes WAN bandwidth utilization • Protects applications from performance degradation • Enables the Internet as a viable WAN transport • Provides multisite coordination to simplify network wide provisioning. • Application-based policy driven framework and is tightly integrated with
existing AVC components. • Smart and Scalable multi-sites solution to enforce application SLAs while
optimizing network resources utilization.
• PfRv3 is the 3rd generation Multi-Site aware Bandwidth and Path Control/Optimization solution for WAN/Cloud based applications. • Available now on ISR-G2, ISR-4000, CSR1000v, ASR1k
56