Routing for an Anycast CDN Martin J Levy @ CloudFlare MENOG14 - Dubai - March 2014.
MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol...
Transcript of MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol...
![Page 1: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/1.jpg)
1
MENOG14, Dubai, 2014
Abdelrahman Ibrahim
Sr. Solution Architect – Cisco
![Page 2: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/2.jpg)
2
• Current L2 and DC deployment limitations.
• Need for TRILL and Goal
• What is TRILL
• TRILL and RBirdges Key Features
• Use Cases for TRLL
• Unicast and Multicast Handling
• TRILL Control Plane Overview
• TRILL Data Plane Overview
• TRILL Standardization Status
![Page 3: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/3.jpg)
3
Why Layer 2 in the Data Center?
• Because customers request it!
• Some protocols rely on the functionality
• Simple, almost plug and play
• No addressing
• Required for implementing subnets
• Allows easy server provisioning
• Allows virtual machine mobility
3
![Page 4: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/4.jpg)
4
POD POD POD
• L2 benefits limited to a POD
4
L3
L2
Current Data Center Design
![Page 5: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/5.jpg)
5
• Just extend STP to the whole network
5
STP
L3
L2
Possible Solution for End-to-End L2?
![Page 6: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/6.jpg)
6
• Local STP problems have network-wide impact, troubleshooting is difficult
• STP provides limited bandwidth (no load balancing)
• STP convergence is disruptive
• Tree topologies introduce sub-optimal paths
• MAC address tables don’t scale
• Flooding impacts the whole network
6
Typical Limitations of L2
![Page 7: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/7.jpg)
7
• Branches of trees never interconnect (no loop!!!)
Spanning Tree Protocol (STP) uses the same approach to build loop-free L2 logical topology
Over-subscription ratio exacerbated by STP algorithm
11 Physical Links
5 Logical Links
![Page 8: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/8.jpg)
8
“TRILL brings Layer 3 routing benefits to
flexible Layer 2 bridged Ethernet networks”
Easy Configuration
Plug & Play
Provisioning Flexibility
Multi-pathing (ECMP)
Fast Convergence
Highly Scalable
Switching Routing
TRILL Fabric
![Page 9: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/9.jpg)
9
![Page 10: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/10.jpg)
10
Externally, a Fabric looks like a single switch
Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides in a plug-and-play fashion:
Optimal, low latency connectivity any to any
High bandwidth, high resiliency
Open management and troubleshooting
TRILL provides additional capabilities in term of scalability and L3 integration
TRILL TRILL
Learning of MAC addresses takes place at the Edge
![Page 11: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/11.jpg)
11
1
1
• TRILL (TRansparent Interconnect of Lots of Links)
http://www.ietf.org/html.charters/trill-charter.html
• Main areas addressed by TRILL:
Provide Shortest Path and Equal Cost Multi-Pathing for traffic
Be Plug-n-Play
![Page 12: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/12.jpg)
12
![Page 13: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/13.jpg)
13
![Page 14: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/14.jpg)
14
![Page 15: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/15.jpg)
15
![Page 16: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/16.jpg)
16
![Page 17: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/17.jpg)
17
![Page 18: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/18.jpg)
18
![Page 19: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/19.jpg)
19
![Page 20: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/20.jpg)
20
![Page 21: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/21.jpg)
21
![Page 22: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/22.jpg)
22
![Page 23: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/23.jpg)
23
![Page 24: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/24.jpg)
24
![Page 25: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/25.jpg)
25
![Page 26: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/26.jpg)
26 26
RBridge
IEEE Bridge
A TRILL Network is comprised of Routing Bridges (RBridges/RBs).
Each RBridge is uniquely identified by a ‘nickname’ or rbridge-id (auto-created from ISIS system id)
RBs can be connected by 802.1 LANs or
RBs can be connected by simple P2P links (incl. PPP – see RFC 6361)
Architecturally, RBridges run “on top” of an 802.1 bridged network similarly to Routers
RBridges may be interconnected by classical 802.1Q bridges: Allows for gradual migration of existing networks
RBridges do not participate in xSTP, and drop BPDUs if they are received
![Page 27: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/27.jpg)
27
TRILL Principles of Operation
27
Frames are encapsulated with the RBridge addresses and further encapsulated with originating rbridge and next hop rbridge MAC address
Header fields differ from 802.1ah
Headers are swapped hop by hop (similar to routing)
RBridges learn what MAC addresses are on their edge ports using general dataplane learning and MAY advertise them other RBridges
Remote mac-address-to-rbridge binding learning: hardware or control plane
Unknown unicast /multicast/broadcast frames flooded along pre-calculated distribution tree(s)
RBridge
IEEE Bridge
AD
E
CA C.1Q frame E
OuterMAC
RBridgeHeader
A
.1Q frame
C D.1Q frame EA
D E.1Q frame EA
.1Q frame
B
![Page 28: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/28.jpg)
28
TRILL Forwarding
28
• RBridges use ISIS for discovery and to synchronize Link State Databases
• TRILL uses these Link State Database to
Compute pair wise bidirectional paths for unicast (per node and/or per VLAN) between all Rbridges
For multicast, distribution trees are calculated rooted at (potentially) every rbridge ; trees are given an rbridge-id/nickname as well
• TRILL adds to standard IS-IS
Ships in the night with other protocols using ISIS
TRILL Hellos
Find out whether nodes are on a LAN or P2P link
Designated Rbridge (DRB) Election
Root-Bridge-IDs
See also: RFC 6165 (Extensions to IS-IS for Layer-2 Systems)
![Page 29: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/29.jpg)
29
TRILL Forwarding (Cont.)
29
• Edge RBridges learn End Station MAC addresses in the data plane and associate them with RBridge nicknames
End Station Address Distribution Information (ESADI)
Optional ESADI capability allows RBridges to distribute End Station MAC addresses in IS-IS.
• TRILL Addressing and Forwarding
Outer header resembles 802.1ah—MAC-SA and MAC-DA interpreted differently: used to send frame to the next Rbridge, much like a router
Provider shim header with TTL field, Ingress and Egress Rbridge Nicknames, and Multicast Flag
Rbridges will use the RB-Nicknames (16-bit RBridge-IDs) as well as a Multicast Flag (M-bit) to forward frames
If M = 0 : egress rbridge-id is used for unicast forwarding
If M = 1 : egress rbridge-id denotes the pre-calculated distribution tree to forward the multicast frame
![Page 30: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/30.jpg)
30
TRILL—Ethernet Data Encapsulation
V: Version
M: Multi-destination; indicates if the frame is to be delivered to a single or multiple end stations
Opt-Length: >0 if an Option field is present
Hop Limit: Similar to TTL
RBridge Nickname: Not the MAC address of the Rbridge, but the a TRILL ID for the RBridge (Egress Nickname used differently if M = 1)
Outer-VLAN Tag Information: This is used only if two RBridges communicate across a standard 802.1Qnetwork
See also: RFC 6325 and RFC 6327
Outer Ethernet Header (link specific):
Outer Destination MAC Address (RB2)
Outer Destination MAC Address Outer Source MAC Address
Outer Source MAC Address (RB1)
Ethertype = IEEE 802.1Q Outer.VLAN Tag Information
TRILL Header:
Ethertype = TRILL V R M Op-Length Hop Count
Egress (RB2) Nickname Ingress (RB1) Nickname
Inner Ethernet Header:
Inner Destination MAC Address
Inner Destination MAC Address Inner Source MAC Address
Inner Source MAC Address
Ethertype = IEEE 0x8100 Inner.VLAN Tag Information
Multicast tree pruning:Requires inspection of customer Destination MAC Addressand customer VLAN
Ethertype = IEEE 0x893B Inner.VLAN second part
In case of Fine Grain Labeling: Second VLAN tag(see draft-ietf-trill-fine-labeling)
![Page 31: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/31.jpg)
31
Packet Flow — Known Unicast
31
RB1
MR1
RB2
RB3
MR3
RB4
RB5
MR5
RB6
MR6
RB7
RB8
MR8
RB9
802.1Q
Cloud
802.1Q
Cloud
TRILL
Network
M1
M2
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q Outer VLAN
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q Inner VLAN
Payload ….
MR8
MR1
RB3
M2
M1
RB1
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q Outer VLAN
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q Inner VLAN
Payload ….
MR3
MR5
RB1
• Perform MAC lookup on M2 to determine Egress RBridge RB3 • Encapsulate in TRILL header & forward to next-hop RBridge
•Perform Egress RBridge nickname lookup on RB3 to determine next hop RBridge
• Decapsulate TRILL header •Perform MAC lookup on M2 to determine egress port
Unchanged From
Ingress to Egress
Changes
Hop-to-Hop (MACs, VLAN, TTL)
Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL
B1
B2
B3
B4 B5
B6 B7
![Page 32: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/32.jpg)
32
Packet Flow — Multicast/Broadcast/Unknown Unicast
32
RB1
MR1
RB2
RB4
RB5
MR5
RB6
RB7
RB8
MR8
RB9
MR9
802.1Q
Cloud
802.1Q
Cloud
TRILL
Network
M1
G1
G1
G1
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q
Outer VLAN
Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q
Inner VLAN
Payload ….
MR1
RB9
G1
M1
RB1
M=1
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q
Outer VLAN
Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q
Inner VLAN
Payload ….
MR5
RB1
Perform MAC lookup on G1 Encapsulate in TRILL
header, set M bit and tree id (egress rbridge id) &
forward to all-RB’s mcast address
Perform lookup on egress rbridge-id to
determine distribution tree
Decapsulate TRILL header Perform MAC lookup on G1 to determine egress ports
M=1
Unchanged From
Ingress to Egress
Changes
Hop-to-Hop (MACs, VLAN, TTL)
All-RB-MCAST
(or MR9) All-RB-MCAST
RB3
MR3
Note: All-RB=All-Rbridges = 01-80-c2-00-00-40
![Page 33: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/33.jpg)
33
TRILL Benefits
33
• Shortest path delivery of unicast
• Layer 2 multi-pathing (ECMP) of unicast
• Optimal multicast delivery over shared trees
Load-balancing over multiple trees.
Per-VLAN/c-group pruning of trees via IGMP/PIM snooping.
• Fast convergence times, Minimal configuration
• Support for Shared Media and P2P links
• Loop Prevention and Mitigation (adds a TTL)
• Support for multi-homing (DRB election)
• Confines MAC Address learning to edge nodes, providing MAC address scalability similar to IEEE 802.1ah (MAC-in-MAC)
![Page 34: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/34.jpg)
34
• Single address lookup at the ingress edge identifies the exit port across the fabric
• Traffic is then switched using the shortest path available
• Reliable L2 and L3 connectivity any to any (L2 as if it was within the same switch, no STP inside)
Shortest path any-to-any
TRILL
A B
s3 s8
MAC IF
A e1/1
… …
B s8, e1/2
e1/1 e1/2
![Page 35: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/35.jpg)
35
• Multi-pathing (up to 256 links active between any 2 devices)
• Traffic is redistributed across remaining links in case of failure, providing fast convergence
Equal Cost Multi-Pathing
TRILL Fabric
A B
s3 s8
![Page 36: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/36.jpg)
36
• Per-port MAC address table only needs to learn the peers that are reached across the fabric
A virtually unlimited number of hosts can be attached to the fabric
Conversational Learning
TRILL Fabric
A B
s3 s8
MAC IF
A s1,e1/1
… …
B e1/2
MAC IF
… …
s5
MAC IF
A e1/1
… …
B s8, e1/2
e1/1 e1/2
![Page 37: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/37.jpg)
37
• Allows extending VLANs with no limitation (no risks of loop)
• Devices can be attached active/active to the fabric using IEEE standard port channels and without resorting to STP
TRILL Fabric
A
s3 s8 s7
B
s4
VLAN X
VLAN Y
VLAN Z
![Page 38: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/38.jpg)
38
![Page 39: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/39.jpg)
39
![Page 40: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/40.jpg)
40
![Page 41: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/41.jpg)
41
![Page 42: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/42.jpg)
42
![Page 43: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/43.jpg)
43
![Page 44: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/44.jpg)
44
TRILL Silicon
• Here are six publicly known independent silicon
implementations of the TRILL Fast Path. In some cases the
vendor has multiple different chips supporting TRILL.
•Broadcom – merchant silicon
•Brocade – products
•Cisco – products
•Fulcrum – merchant silicon
•Marvell – merchant silicon
•Mellanox – merchant silicon
![Page 45: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/45.jpg)
45
TRILL Open Source
Three Open Source Implementations
1. Oracle: TRILL for Solaris
TRILL ships as part of Solaris 11
2. VirtuOR: www.virtuor.fr
http://sourceforge.net/p/opentrill/wiki/Home/
3. TRILL Port to Linux: National University of Sciences and Technology (NUST), Islamabad, Pakistan
Muhammad Mohsin Sardar [email protected]
http://wisnet.seecs.nust.edu.pk/projects/trill/index.html
![Page 46: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/46.jpg)
46
![Page 47: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/47.jpg)
47
![Page 48: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/48.jpg)
48
![Page 49: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/49.jpg)
49
![Page 50: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/50.jpg)
50
![Page 51: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/51.jpg)
51
![Page 52: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/52.jpg)
52
IXP Requirements
• Layer 2 Peering
• 10GE non-blocking Fabric
• Scale to thousands of ports
TRILL Benefits for IXP
• Layer 2 Fabric
• Non-blocking up to thousands 10GE ports
• Simple to manage
• No design constraint, easy to grow
TRILL/ TRILL
Provider A Provider B
Provider C
Provider D
Provider E
Provider F
![Page 53: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/53.jpg)
53
![Page 54: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/54.jpg)
54
• IS-IS assigns addresses to all TRILL switches automatically
• Compute shortest, pair-wise paths • Support equal-cost paths between any TRILL
switch pairs
L1
TRILL
Routing Table
L2 L3
L4
TRILL
Switch IF
S10 L1
S20 L2
S30 L3
S40 L4
S200 L1, L2, L3, L4
… …
S400 L1, L2, L3, L4
S100 S200 S300 S400
S10 S20 S30 S40
New Control Plane Plug-n-Play L2 IS-IS Manages Forwarding Topology
5
4
![Page 55: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/55.jpg)
55
• Assigned switch addresses to all TRILL enabled switches automatically (no user configuration required)
• Compute shortest, pair-wise paths • Support equal-cost paths between any TRILL switch pairs
Plug-N-Play L2 IS-IS is used to manage forwarding topology
L1
L2
S1 S2 S3 S4
S11 S12 S42
L2 Fabric
L3
L4
TRILL
Routing Table
Switch IF
S1 L1
S2 L2
S3 L3
S4 L4
S12 L1, L2, L3, L4
… …
S42 L1, L2, L3, L4
![Page 56: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/56.jpg)
56
New Data Plane
5
6
![Page 57: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/57.jpg)
57
S100: CE MAC
Address Table
A
S10 S20 S30 S40
S100 S200 S300
TRILL
B
1/1
Classical Ethernet
S300: CE MAC
Address Table
MAC IF
B 1/2
… …
S200: CE MAC
Address Table
MAC IF
… …
… …
S100 M A B
Lookup B: Miss
Don’t learn
Lookup B: Miss
Flood
Lookup B: Hit
Learn source A
MAC IF
B 1/2
A S100
MAC IF
… …
… …
MAC IF
A 1/1
… …
1/2
![Page 58: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/58.jpg)
58
Classical Ethernet
Conversational Learning
S100: CE MAC
Address Table
A
S10 S20 S30 S40
S100 S200 S300
TRILL Fabric
B
1/1 S300: CE MAC
Address Table
MAC IF
B 1/2
… …
S200: CE MAC
Address Table
MAC IF
… …
… …
MAC IF
B 1/2
A S100
MAC IF
… …
… …
MAC IF
A 1/1
… …
1/2
S300: TRILL
Routing Table
Switch IF
… …
S100 L1, L2, L3, L4
S300 S100 B A
Lookup A: Hit
Send to S100 Lookup A: Hit
Learn source B
MAC IF
A 1/1
B S300
![Page 59: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/59.jpg)
59
![Page 60: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/60.jpg)
60
![Page 61: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/61.jpg)
61
![Page 62: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/62.jpg)
62
![Page 63: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/63.jpg)
63
![Page 64: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/64.jpg)
64
![Page 65: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/65.jpg)
65
![Page 66: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/66.jpg)
66
![Page 67: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/67.jpg)
67
![Page 68: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/68.jpg)
68
![Page 69: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/69.jpg)
69 © 2010 Cisco and/or its affiliates. All rights reserved.
Multicast and Multi-destination Handling in TRILL
![Page 70: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/70.jpg)
70
Packet Flow — Multicast/Broadcast/Unknown Unicast
70
RB1
MR1
RB2
RB4
RB5
MR5
RB6
RB7
RB8
MR8
RB9
MR9
802.1Q
Cloud
802.1Q
Cloud
TRILL
Network
M1
G1
G1
G1
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q
Outer VLAN
Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q
Inner VLAN
Payload ….
MR1
RB9
G1
M1
RB1
M=1
Outer MAC DA
Outer MAC DA
Outer MAC SA
Outer MAC SA
Etype = 802.1Q
Outer VLAN
Etype = TRILL
Egress RB-ID Ingress RB-ID
V/M/R, TTL
Inner MAC DA
Inner MAC DA
Inner MAC SA
Inner MAC SA
Etype = 802.1Q
Inner VLAN
Payload ….
MR5
RB1
Perform MAC lookup on G1 Encapsulate in TRILL
header, set M bit and tree id (egress rbridge id) &
forward to all-RB’s mcast address
Perform lookup on egress rbridge-id to
determine distribution tree
Decapsulate TRILL header Perform MAC lookup on G1 to determine egress ports
M=1
Unchanged From
Ingress to Egress
Changes
Hop-to-Hop (MACs, VLAN, TTL)
All-RB-
MCAST
(or MR9) All-RB-MCAST
RB3
MR3
Note: All-RB=All-Rbridges = 01-80-c2-00-00-40
![Page 71: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/71.jpg)
71
• Control plane:
IGMP snooping operates as usual in TRILL edge switches
TRILL IS-IS learns multicast group membership from IGMP snooping on edge switch
TRILL edge switch announces group interest by using GM-LSPs, creating “pruned trees” for each group on each multidestination tree
• Data plane:
Hardware selects which multidestination tree to use for each flow based on hash function
Once tree selected, traffic constrained to pruned tree for that IP multicast group, based on MAC table lookup
71
![Page 72: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/72.jpg)
72
• IGMP snooping learns of interested receivers on TRILL edge switches
• Membership tracked on CE ports based on receiving IGMP reports/leaves
Only locally connected receivers tracked on a given edge switch
• Group membership advertised in TRILL IS-IS using GM-LSPs
IGMP Reports
IGMP Reports
TRILL
S1
00
S3
00
S2
00
Root of
Tree 1
Root of
Tree 2
Mrouter
IGMP
snooping
IGMP
snooping
IGMP
snooping
GM-
LSPs
GM-
LSPs
Receiver G1
Receiver G2
Receiver G2
Source G1
Source G2 PIM Hellos
IGMP GQs
Ftag 1
Ftag 2
72
![Page 73: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/73.jpg)
73 © 2010 Cisco and/or its affiliates. All rights reserved.
Standardization Status
![Page 74: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/74.jpg)
74
TRILL in 2009/2011
2009: RFC 5556 “TRILL: Problem and Applicability Statement”
2009: TRILL Protocol passed up to IESG for Approval.
2010: TRILL approved IETF Standard (2010-03-15)
Ethertypes, Multicast addresses & NLPID assigned
2010: Successful TRILL control plane interop at UNH IOL
2011: TRILL Protocol base document set:
RFC 6325: “RBridges: TRILL Base Protocol Specification”
RFC 6326: “TRILL Use of IS-IS”
RFC 6327: “RBridges: Adjacency”
RFC 6361: “TRILL over PPP”
RFC 6439: “RBridges: Appointed Forwarders”
2011: TRILL Working Group Re-Chartered to do further
development of the TRILL protocol
![Page 75: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/75.jpg)
75
![Page 76: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/76.jpg)
76
TRILL in 2012/2013
2012: Second Successful TRILL control plane interop at UNH IOL
2013: Additional TRILL documents published:
RFC 6447: FCoE (Fibre Channel over Ethernet) over TRILL
RFC 6850: RBridge MIB
RFC 6905: TRILL OAM Requirements
2013: Third TRILL interop for control and data plane at UNH
IOL week of May 20th
2013: TRILL Working Group Re-Chartered to do further
development of the TRILL protocol
![Page 77: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/77.jpg)
77
Standardization Status • The TRILL protocol RFCs (bold = stds track)
RFC 5556, “TRILL Problem and Applicability”
RFC 6325, “RBridges: TRILL Base Protocol Specification”
RFC 6326, “TRILL Use of IS-IS”
RFC 6327, “RBridges: Adjacency”
RFC 6361, “TRILL over PPP”
RFC 6439, “RBridges: Appointed Forwarders”
RFC 6847, “FCoE over TRILL”
RFC 6850, “Definitions of Managed Objects for RBridges” (MIB)
RFC 6905, “TRILL OAM Requirements”
![Page 78: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/78.jpg)
78
Standardization Status
Document that are fully approved and in the RFC Editor’s Queue. These are expected to issue as standards track RFCs soon:
“TRILL: Fine Grained Labeling:
https://datatracker.ietf.org/doc/draft-ietf-trill-fine-labeling/
“TRILL: BFD Support”
https://datatracker.ietf.org/doc/draft-ietf-trill-rbridge-bfd/
“TRILL: RBridge Channel Support”
https://datatracker.ietf.org/doc/draft-ietf-trill-rbridge-channel/
“TRILL: Edge Directory Assistance Framework”
https://datatracker.ietf.org/doc/draft-ietf-trill-directory-framework/
“TRILL: Clarifications, Corrections, and Updates”
https://datatracker.ietf.org/doc/draft-ietf-trill-clear-correct/
“TRILL: Header Extension”
https://datatracker.ietf.org/doc/draft-ietf-trill-rbridge-extension/
![Page 79: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/79.jpg)
79
Standardization Status
• Non-IETF Assignments:
Ethertypes assigned by IEEE:
TRILL Data: 0x22F3
TRILL IS-IS: 0x22F4
TRILL Fine Grained Labeling: 0x893B
RBridge Channel: 0x8946
Block of multicast addresses assigned to TRILL by IEEE:
01-80-C2-00-00-40 to 01-80-C2-00-00-4F
TRILL NLPID (Network Layer Protocol ID) assigned from ISO/IEC: 0xC0
79
![Page 80: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/80.jpg)
80
More TRILL References
• TRILL Introductory Internet Protocol Journal Article:
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_14-3/143_trill.html
• The first paper: Perlman, Radia. “Rbridges: Transparent Routing”, Proceeding Infocom 2004, March 2004.
http://www.ieee-infocom.org/2004/Papers/26_1.PDF
![Page 81: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/81.jpg)
81
Comparison with MPLS
• TRILL versus MPLS
MPLS is an older, more mature technology with better Quality of Service features, etc.
MPLS is more configuration intensive. TRILL can be auto-configuring.
TRILL provides easier support of multicast
TRILL can scale better because
MPLS requires a label entry at each LSR (Label Switched Router) for each MPLS path through that LSR
TRILL requires a nickname entry at each RBridge for each TRILL switch in the campus
![Page 82: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/82.jpg)
82
Comparison with IP
• TRILL versus IP
IP is an older, more mature technology
TRILL supports VM mobility. Changing subnets changes IP Address, breaking TCP connections
TRILL is better at multicast because
IP requires a complex protocols like PIM to do multicast
TRILL has simple multicast distribution, with pruning for optimization, designed in from the start
![Page 83: MENOG14, Dubai, 2014...10 Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides](https://reader034.fdocuments.us/reader034/viewer/2022042204/5ea52a8178febf6d7c674879/html5/thumbnails/83.jpg)
Thank you.