Scalable High-Performance User Space Networking for Containers
Scalable Flow-Based Networking with DIFANE
description
Transcript of Scalable Flow-Based Networking with DIFANE
![Page 1: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/1.jpg)
Scalable Flow-Based Networking with DIFANE
1
Minlan YuPrinceton University
Joint work with Mike Freedman, Jennifer Rexford and Jia Wang
Do It Fast ANd Easy
![Page 2: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/2.jpg)
2
What’s DIFANE?• Traditional enterprise
– Hard to manage– Limited policies– Distributed
• Flow-based networking– Easy to manage– Support fine-grained policy– Scalability remains a challenge
DIFANE:A scalable way to apply fine-grained
policies in enterprises
![Page 3: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/3.jpg)
HTTP
• Access control – Drop packets from malicious hosts
• Customized routing– Direct Skype calls on a low-latency path
• Measurement– Collect detailed HTTP
traffic statistics
Flexible Policies in Enterprises
3
HTTP
![Page 4: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/4.jpg)
Flow-based Switches
• Install rules in flow-based switches– Store rules in high speed memory (TCAM)
• Perform simple actions based on rules– Rules: Match on bits in the packet header– Actions: Drop, forward, count
4
drop
forward via link 1
Flow space src.
dst.
![Page 5: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/5.jpg)
Challenges of Policy-Based Management
• Policy-based network management– Specify high-level policies in a management system – Enforce low-level rules in the switches
• Challenges– Large number of hosts, switches and policies– Limited TCAM space in switches– Support host mobility– No hardware changes to commodity switches
5
![Page 6: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/6.jpg)
Pre-install Rules in Switches
6
Packets hit the rules Forward
• Problems:– No host mobility support– Switches do not have enough memory
Pre-install rules
Controller
![Page 7: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/7.jpg)
Install Rules on Demand (Ethane, NOX)
7
First packetmisses the rules
Buffer and send packet header to the controller
Install rules
Forward
Controller
• Problems:– Delay of going through the controller– Switch complexity– Misbehaving hosts
![Page 8: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/8.jpg)
DIFANE: Combining Proactive & Reactive
8
FeaturesProactive Reactive(
Ethane) DIFANE
Host mobility
Memory usage
Keep packet in data plane
Install
rules
![Page 9: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/9.jpg)
DIFANE Architecture(two stages)
9
DIstributed Flow Architecture for Networked Enterprises
![Page 10: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/10.jpg)
Stage 1
10
The controller proactively generates the rules and distributes them to
authority switches.
![Page 11: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/11.jpg)
Partition and Distribute the Flow Rules
11
Ingress Switch
Egress Switch
Distribute partition information Authority
Switch A
AuthoritySwitch B
Authority Switch C
reject
acceptFlow space
Controller
Authority Switch A
Authority Switch B
Authority Switch C
![Page 12: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/12.jpg)
Stage 2
12
The authority switches keep packets always in the data plane and
reactively cache rules.
![Page 13: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/13.jpg)
Following packets
Packet Redirection and Rule Caching
13
Ingress Switch
Authority Switch
Egress SwitchFirst packet Redirect
Forward
Feedback:
Cache rules
Hit cached rules and forward
A slightly longer path in the data plane is faster than going through the control plane
![Page 14: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/14.jpg)
Locate Authority Switches• Partition information in ingress switches
– Using a small set of coarse-grained wildcard rules– … to locate the authority switch for each packet
• Distributed directory service but not DHT– Hashing does not work for wildcards– Keys can have wildcards in arbitrary bit positions
14
Authority Switch A
AuthoritySwitch B
Authority Switch C
X:0-1 Y:0-3 AX:2-5 Y: 0-1BX:2-5 Y:2-3 C
![Page 15: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/15.jpg)
Following packets
Packet Redirection and Rule Caching
15
Ingress Switch
Authority Switch
Egress SwitchFirst packet
RedirectForward
Feedback:
Cache rules
Hit cached rules and forward
Cache Rules
Partition Rules
Auth. Rules
![Page 16: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/16.jpg)
Three Sets of Rules in TCAMType Priority Field 1 Field 2 Action Timeout
Cache Rules
210 00** 111* Forward to Switch B 10 sec209 1110 11** Drop 10 sec… … … … …
Authority Rules
110 00** 001* ForwardTrigger cache manager
Infinity
109 0001 0*** Drop, Trigger cache manager
… … … … …
Partition Rules
15 0*** 000* Redirect to auth. switch14 …… … … … …
16
In ingress switchesreactively installed by authority switches
In authority switchesproactively installed by controller
In every switchproactively installed by controller
![Page 17: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/17.jpg)
Cache Rules
DIFANE Switch PrototypeBuilt with OpenFlow switch
17
Data Plane
Control Plane
CacheManager
Send Cache Updates
Recv Cache Updates Only in
Auth. Switches
Authority RulesPartition Rules
Just software modification for authority switches
Notification
Cache rules
![Page 18: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/18.jpg)
Caching Wildcard Rules• Overlapping wildcard rules
– Cannot simply cache matching rules
18
![Page 19: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/19.jpg)
Caching Wildcard Rules• Multiple authority switches
– Contain independent sets of rules– Avoid cache conflicts in ingress switch
19
Authorityswitch 1
Authorityswitch 2
![Page 20: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/20.jpg)
Partition Wildcard Rules• Partition rules
– Minimize the TCAM entries in switches– Decision-tree based rule partition algorithm
20
Cut A
Cut BCut B is better than Cut A
![Page 21: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/21.jpg)
Handling Network Dynamics
21
Network dynamics Cache rules Authority
RulesPartition
RulesPolicy changes
at controller Timeout Change Mostly no change
Topology changes at switches
No change No change Change
Host mobility Timeout No change No change
![Page 22: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/22.jpg)
Prototype Evaluation
• Evaluation setup– Kernel-level Click-based OpenFlow switch– Traffic generators, switches, controller run on
separate 3.0GHz 64-bit Intel Xeon machines
• Compare delay and throughput – NOX: Buffer packets and reactively install rules– DIFANE: Forward packets to authority switches
22
![Page 23: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/23.jpg)
Delay Evaluation
• Average delay (RTT) of the first packet– NOX: 10 ms– DIFANE: 0.4 ms
• Reasons for performance improvement– Always keep packets in the data plane– Packets are delivered without waiting for rule
caching– Easily implemented in hardware to further
improve performance
23
![Page 24: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/24.jpg)
Peak Throughput• One authority switch; Single-packet flow
24
1K 10K 100K 1000K1K
10K
100K
1,000KDIFANENOX
Sending rate (flows/sec)
Thro
ughp
ut (fl
ows/
sec)
2 3 41 ingress switch
ControllerBottleneck (50K)
DIFANE (800K)
Ingress switchBottleneck(20K)
DIFANE further increases the throughput linearly with the number of authority switches.
![Page 25: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/25.jpg)
Scaling with Many Rules• How many authority switches do we need?
– Depends on total number of rules … and the TCAM space in these authority switches
25
Campus IPTV# Rules 30K 5M
# Switches 1.7K 3K
Assumed Authority Switch TCAM size
160 KB 1.6 MB
Required # Authority Switches
5 (0.3%) 100 (3%)
![Page 26: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/26.jpg)
Stepping back …
26
![Page 27: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/27.jpg)
Distributed or Centralized?
27
logically-centralized
in the management
system
Distributed amongst the
network elements
All functions in switches
OpenFlow/NOX
DIFANEController is still in charge
Switches host a distributed directory of the rules
![Page 28: Scalable Flow-Based Networking with DIFANE](https://reader035.fdocuments.us/reader035/viewer/2022081520/56816925550346895de05f42/html5/thumbnails/28.jpg)
Thanks!
28