1 Internet Routing Jennifer Rexford Princeton University jrex/bgp-tutorial.
“Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton...
-
Upload
maryann-johnson -
Category
Documents
-
view
213 -
download
0
Transcript of “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton...
![Page 1: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/1.jpg)
“Hashing Out” the Future of Enterprise and Data-Center Networks
Jennifer RexfordPrinceton University
http://www.cs.princeton.edu/~jrex
Joint with Changhoon Kim, Minlan Yu, Matthew Caesar, and Alex Fabrikant
![Page 2: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/2.jpg)
2
Challenges in Edge Networks
• Large number of hosts–Tens or even hundreds of thousands of hosts
• Dynamic hosts–Host mobility–Virtual machine migration
• Cost conscious–Equipment costs–Network-management costs
Need a scalable and efficient self-configuring network
![Page 3: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/3.jpg)
3
An All-Ethernet Solution?
• Self-configuration–Hosts: MAC addresses–Switches: self-learning
• Simple host mobility–Location-independent flat addresses
• But, poor scalability and performance–Flooding frames to unknown destinations–Large forwarding tables (one entry per address)–Broadcast for discovery (e.g., ARP, DHCP)–Inefficient delivery of frames over spanning tree
![Page 4: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/4.jpg)
4
An All-IP Solution?
• Scalability–Hierarchical prefixes
(smaller tables)–No flooding
• Performance–Forwarding traffic over shortest paths
• But, several disadvantages–Complex joint configuration of routing and DHCP–Clumsy mobility (location-dependent addresses)–Expensive longest-prefix match in data plane
![Page 5: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/5.jpg)
5
Compromise: Hybrid Architecture
Ethernet-based IP subnets interconnected by routers
R
R
R
R
Ethernet Bridging - Flat addressing - Self-learning - Flooding - Forwarding along a tree
IP Routing - Hierarchical addressing - Subnet configuration - Host configuration - Forwarding along shortest paths
R
Sacrifices Ethernet’s simplicity and IP’s efficiency for scalability
![Page 6: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/6.jpg)
6
Can We Do Better?
• Shortest-path routing on flat addresses–Shortest paths: scalability and performance–MAC addresses: self-configuration and mobility
• Scalability without hierarchical addressing–Limit dissemination and storage of host info–Sending packets on slightly longer paths
SH
S
S
S
S
S S
S S
S S S
S
H
H
H
H
HH
H
H
![Page 7: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/7.jpg)
7
Outline
• SEATTLE–Distributed directory service
• BUFFALO–Compact forwarding tables
• SeaBuff–Combining Seattle and Buffalo
• Deployment scenarios–Enterprises vs. data centers
![Page 8: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/8.jpg)
8
SEATTLE
Scalable Ethernet Architecture for Large Enterprises
![Page 9: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/9.jpg)
9
SEATTLE Design Decisions
Objective Approach Solution
1. Avoiding flooding
Never broadcast unicast traffic Network-layer
one-hop DHT2. Restraining
broadcastingBootstrap hosts
via unicast
3. Reducing routing state
Populate host infoonly when and where
it is needed
Traffic-driven resolution with caching
4. Shortest-path forwarding
Allow switches to learn topology
L2 link-state routingmaintaining only
switch-level topology
* Meanwhile, avoid modifying end hosts
![Page 10: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/10.jpg)
10
Network-Layer One-hop DHT
• Maintains <key, value> pairs with function F – Consistent hash mapping a key to a switch–F is defined over the set of live switches
• One-hop DHT– Link-state routing ensures
switches know each other
• Benefits– Fast and efficient
reaction to changes– Reliability and capacity
naturally grow with size of the network
0 12128-1
![Page 11: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/11.jpg)
Location Resolution
11
SwitchesEnd hosts
Control messageData traffic
<key, val> = <MAC addr, location><key, val> = <MAC addr, location>
Host discovery
B
x
HashF(MACx) = B
Store<MACx, A>
Traffic to x
HashF(MACx ) = BTunnel
to A
Notify<MACx, A>
E
Forward directly from D to A
A
Tunnel to B
C
D
yOwner
User
Resolver
Publish<MACx, A>
![Page 12: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/12.jpg)
Address Resolution
12
<key, val> = <IP addr, MAC addr><key, val> = <IP addr, MAC addr>
Traffic following ARP takes a shortest pathwithout separate location resolution
B
DHash
F(IPx) = B
Store<IPx, MACx, A>
BroadcastARP requestfor IPx
HashF(IPx ) = B
Unicast reply<IPx, MACx, A>
E
A
Unicastlook-up to B
C
<IPx ,MACx>
x y
![Page 13: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/13.jpg)
13
Handling Network and Host Dynamics
• Network events–Switch failure/recovery
Change in <key, value> for DHT neighbor Fortunately, switch failures are not common
–Link failure/recovery Link-state routing finds new shortest paths
• Host events–Host location, MAC address, or IP address –Must update stale host-information entries
![Page 14: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/14.jpg)
14
Handling Host Information Changes
Resolver
y
Host talkingwith x
< x, A >
< x, A >
< x, A >
D< x, D >
Oldlocation
New location
< x, D >
< x, D >
< x, D >
Dealing with host mobilityDealing with host mobility
MAC- or IP-address change can be handled similarly
B
xA
C
E
F
![Page 15: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/15.jpg)
15
Packet-Level Simulations
• Large-scale packet-level simulation–Event-driven simulation of control plane–Synthetic traffic based on LBNL traces –Campus, data center, and ISP topologies
• Main results–Much less routing state than Ethernet–Only slightly more stretch than IP routing–Low overhead for handling host mobility
![Page 16: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/16.jpg)
16
Amount of Routing State
SEATTLEw/o caching
SEATTLEw/ caching
Ethernet
SEATTLE reduces the amount of routing state by more than an order of magnitudeSEATTLE reduces the amount of routing
state by more than an order of magnitude
![Page 17: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/17.jpg)
17
Cache Size vs. StretchStretch = actual path length / shortest path length (in latency)
SEATTLE offers near-optimal stretch with very small amount of routing state
SEATTLE offers near-optimal stretch with very small amount of routing state
SEATTLE
ROFL
![Page 18: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/18.jpg)
18
Sensitivity to Mobility
SEATTLE rapidly updates routing statewith very low overhead
SEATTLE rapidly updates routing statewith very low overhead
SEATTLEw/o caching
SEATTLEw/ caching
Ethernet
![Page 19: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/19.jpg)
19
Prototype Implementation
Host-info registrationand notification msgs
User/Kernel Click
XORP
OSPFDaemon
OSPFDaemon
RingManager
RingManager
Host InfoManagerHost InfoManager
SeattleSwitchSeattleSwitch
Link-stateadvertisements
Data Frames
Data Frames
RoutingTable
RoutingTable
NetworkMap
NetworkMap
ClickInterface
ClickInterface
Throughput: 800 Mbps for 512B packets, or 1400 Mbps for 896B packets
![Page 20: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/20.jpg)
20
Conclusions on SEATTLE
• SEATTLE –Self-configuring–Scalable–Efficient
• Enabling design decisions–One-hop DHT with link-state routing–Reactive location resolution and caching–Shortest-path forwarding
![Page 21: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/21.jpg)
21
BUFFALO
Bloom Filter Forwarding Architecture for Large Organizations
![Page 22: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/22.jpg)
Data Plane Scaling Challenge
• Large layer-two networks–Many end-host MAC addresses–Many switches
• Forwarding-table size becomes a problem–Requires a large, fast memory–Expensive and power hungry–Over-provisioning to avoid running out
• Buffalo’s goal–Given a small, fast memory–… make the most of it!
![Page 23: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/23.jpg)
23
Bloom Filters
• Bloom filters in fast memory – A compact data structure for a set of elements– Calculate s hash functions to store element x– Easy to check set membership – Reduce memory at expense of false positives
h1(x) h2(x) hs(x)01000 10100 00010
x
V0Vm-1
h3(x)
![Page 24: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/24.jpg)
24
Bloom Filter Forwarding
• One Bloom filter (BF) per next hop– Store all addresses forwarded to that next hop
Nexthop 1
Nexthop 2
Nexthop T
……packet
query
Bloom Filters
hit
![Page 25: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/25.jpg)
25
BUFFALO Challenges
![Page 26: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/26.jpg)
26
1. Optimize Memory Usage
• Goal: Minimize overall false-positive rate– Probability that one BF has a false positive
• Input:– Fast memory size M– Number of destinations per next hop– The maximum number of hash functions
• Output: the size of each Bloom filter– Larger BF for next hop with more destinations
![Page 27: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/27.jpg)
27
1. Optimize Memory Usage (cont.)
• Constraints– Memory constraint
Sum of all BF sizes <= fast memory size M
– Bound on number of hash functions To bound CPU calculation time Bloom filters share the same hash functions
• Convex optimization problem– An optimal solution exists– Solved by IPOPT optimization tool– Runs in about 50 msec
![Page 28: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/28.jpg)
28
1. Minimize False Positives
• Forwarding table with 200K entries, 10 next hops
• 8 hash functions
![Page 29: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/29.jpg)
29
1. Comparing with Hash Table
• Save 65% memory with 0.1% false positive
65%
![Page 30: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/30.jpg)
30
2. Handle False Positives
• Design goals– Should not modify the packet– Never go to slow memory– Ensure timely packet delivery
• BUFFALO solution – Exclude incoming interface
Avoid loops in one false positive case
– Random selection among the rest Guarantee reachability with multiple FPs
![Page 31: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/31.jpg)
31
2. One False Positive
• Most common case: one false positive–When there are multiple matching next hops–Avoid sending to incoming interface
• We prove that there is at most a 2-hop loop with a stretch <= l(AB)+l(BA)
False positive
AA
Shortest path
BB
dst
![Page 32: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/32.jpg)
32
2. Multiple False Positives
• Handle multiple false positives– Random selection from matching next hops– Random walk on shortest path tree plus a few
false positive links– To eventually find out a way to the destination
dstdst
Shortest path tree for destination
False positive link
![Page 33: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/33.jpg)
33
2. Multiple False Positives (cont.)
• Provable stretch bound–With k false positives–… expected stretch is at most O(k2*3k/3)–Proved by random walk theories
• Stretch bound is actually not that bad–False positives are independent
Different switches use different hash functions
–k false positives for one packet are rare Probability of k false positives drops exponentially in k
• Tighter bounds in special cases: e.g., tree
![Page 34: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/34.jpg)
34
2. Stretch in Campus Network
When fp=0.001%99.9% of the packets
have no stretch
When fp=0.001%99.9% of the packets
have no stretch0.0003% packets have a stretch of
shortest path length
0.0003% packets have a stretch of
shortest path length
When fp=0.5%, 0.0002% packets
have a stretch 6 times of shortest path length
When fp=0.5%, 0.0002% packets
have a stretch 6 times of shortest path length
![Page 35: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/35.jpg)
3. Update on Routing Change
• Use CBF in slow memory– Assist BF to handle forwarding-table updates– Easy to add/delete a forwarding-table entry
CBF in slow memory
BF in fast memory
Delete a route
![Page 36: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/36.jpg)
3. Occasionally Resize BF
• Under significant routing changes–# of addresses in BFs changes significantly–Re-optimize BF sizes
• Use CBF to assist resizing BF–Large CBF and small BF–Easy to expand BF size by contracting CBF
1 0
Hard to expand to size 4
CBF BF
Easy to contract CBF to size 4
![Page 37: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/37.jpg)
BUFFALO Switch Architecture
37
Prototype implemented in kernel-level Click
![Page 38: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/38.jpg)
38
Prototype Evaluation
• Environment– 3.0 GHz 64-bit Intel Xeon– 2 MB L2 data cache (fast-memory size M)– 200K forwarding-table entries and 10 next hops
• Peak forwarding rate– 365 Kpps, 1.9 μs per packet– 10% faster than hash-based EtherSwitch
• Performance with forwarding-table updates– 10.7 μs to update a route– 0.47 s to reconstruct BFs based on CBFs
![Page 39: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/39.jpg)
39
Conclusion on BUFFALO
• BUFFALO–Small, bounded memory requirement–Small stretch–Fast reaction to routing updates
• Enabling design decisions–Bloom filter per next hop–Optimizing of Bloom filter sizes–Preventing forwarding loops–Dynamic updates using counting Bloom filters
![Page 40: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/40.jpg)
40
SeaBuff
Seattle + Buffalo
![Page 41: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/41.jpg)
Seattle + Buffalo
• Seattle–Shortest-path routing and scalable control plane–Fewer host MAC addresses stored at switches
• Buffalo–Less memory for given # of routable addresses–Graceful handling of increase in # of addresses
• Combined data plane
cachedestination
egressswitch
outgoinglink
Seattle
Bloomfilter
Buffalo
![Page 42: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/42.jpg)
Two Small Sources of Stretch
• Seattle: diverting some packets through relay B
• Buffalo: extra stretch from D to B, or B to A
B
E
C
D
A
x y
Traffic to x
Relay for x
![Page 43: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/43.jpg)
Choosing the Right Solution
• Spatial distribution of traffic–Sparse: caching in Seattle is effective–Dense: caching is not as effective
• Temporal distribution of traffic–Stable: shortest path routing is effective–Volatile: forwarding through relay spreads traffic
• Topology–Arbitrary: false positives have more impact–Tree-like: false positives have less impact
![Page 44: “Hashing Out” the Future of Enterprise and Data-Center Networks Jennifer Rexford Princeton University jrex Joint with Changhoon.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649f2c5503460f94c47cb6/html5/thumbnails/44.jpg)
Conclusions
• Growing important of edge networks–Enterprise networks–Data center networks
• Shortest-path routing and flat addressing–Self-configuring and efficient–Scalability in exchange for stretch
• Ongoing work–SeaBuff = Seattle + Buffalo–Theoretical analysis of stretch in Buffalo–Efficient access control in OpenFlow