Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o...
-
Upload
kailyn-applewhite -
Category
Documents
-
view
216 -
download
2
Transcript of Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o...
![Page 1: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/1.jpg)
Openflow App TestingChao SHI,
Stephen Duraski
![Page 2: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/2.jpg)
Motivation
• Network is still a complex stuff !o Distributed mechanismo Complex protocolo Large stateo Asynchronous events, random ordering, race
conditions.
![Page 3: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/3.jpg)
Motivation
• Openflow does not reduce complexity, it just encapsulates it.o The distributed and asynchronous feature of
networking is inherento The correctness of controller not verifiedo Hard to test your apps on Openflow controller
![Page 4: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/4.jpg)
Race Condition
![Page 5: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/5.jpg)
Background
![Page 6: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/6.jpg)
Finite State Machine
• A model of hardware or computer programs
• Stateo Variable value assignment, flags, code block in
execution, etc
• Transition
• Input (Event)
• Output (Action)
![Page 7: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/7.jpg)
State Diagram
![Page 8: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/8.jpg)
Model Checking for Distributed System
• System:o Components & Channels
• State:o Component Stateo System State = Component States + Channel Contents
• Transitiono Enabled Transitions (System level)o System Execution (Sequence of Transitions)
![Page 9: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/9.jpg)
Model Checking
• What to look for:o Deadlocks, incompatible states causing crasho Both hardware and software
• Method:o State Space Enumeration
Depth First Search using stack in the papero No Scalability
• Less suitable for softwareo Large possibility of user inputo "Undecidability cannot be fully algorithmic"
![Page 10: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/10.jpg)
Symbolic Execution
• Tracking the symbolic variable rather than actual value
• Symbolic variable is associated with input
• Real variables are associated with symbolic variables and the expression is updated accordingly
• Fork at every branch.o Still not scalable
![Page 11: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/11.jpg)
![Page 12: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/12.jpg)
Symbolic Execution
• Equivalent classo Same code path for all instances in the same class
• Still problemso Code path coverage does not mean system path coverage
• Combine State Modeling and Symbolic Execution is good !o State modeling to exhaust system stateso Symbolic execution to reduce transition numbers
![Page 13: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/13.jpg)
Design
![Page 14: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/14.jpg)
Modeling in Openflow
• State: Controller + Switch + Host
• Transition
• Input: o Event (Packet_IN, Flow_Removed, Port_Status)o Message Content (OFMessage) ( We don't want it )
• Output:o The actual code path of the event Handler (Atomic)o That problem can be solved using the extension of
Symbolic Execution
![Page 15: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/15.jpg)
![Page 16: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/16.jpg)
![Page 17: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/17.jpg)
OFSwitch and Host Simplification
• Switch is channels + flow table
• Switch has Two simple transitiono process_pkt, process_ofo process_pkt: multiple packet processing - > a state transfer
• Equivalent flow table is the same state
• Host is categorizedo Server-- receive / send_reply o Client -- send / receive
![Page 18: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/18.jpg)
Symbolic Execution
• Find the equivalent class of input packet
• Pick one packet from the class as relevant packet
• Symbolic packet composed of Symbolic variables
• Concrete Controller State
![Page 19: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/19.jpg)
Combination
![Page 20: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/20.jpg)
Initialization: enable discover_packets by every switch
Run discover_packets_transition and enable new transitions
![Page 21: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/21.jpg)
Are you the switch in discover_packets_transition. If so, your next transition is already assigned. If not, continue listening on packets
![Page 22: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/22.jpg)
state of the ctrl is changed
![Page 23: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/23.jpg)
OpenFlow Specific Search Strategies
• PKT-SEQ: Relevant packet sequences
• NO-DELAY: Instantaneous rule updates
• UNUSUAL: Uncommon delays and reordings
• FLOW-IR: Flow independence reduction
• PKT-SEQ is complementary to other strategies in that it only reduces the number of send transitions rather than the possible kind of event orderings
![Page 24: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/24.jpg)
Specifying Application Correctness
• Customizable correctness properties - testing correctness involves verifying that "something bad never happens" (safety) and that "something good eventually happens" (liveness).
• To check these properties, NICE allows correctness properties to (i)access the system state, (ii) register callbacks, and (iii) maintain local state
![Page 25: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/25.jpg)
Specifying Application Correctness
• NICE provides a library of correctness properties applicable to a wide variety of OpenFlow applications.
• Includes: NoForwardingLoops, NoBlackHoles, DirectPaths, StrictDirectPaths, NoForgottenPackets
![Page 26: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/26.jpg)
Implementation
NICE consists of 3 parts:
• A model checker - NICE remembers the sequence of transitions that created the state and restores it by replaying the sequence, state-matching is doing by comparing and storing hashes of the explored states.
• A concolic execution engine - executes the Python code with concrete instead of symbolic inputs
• A collection of models including the simplified switch and several end hosts.
![Page 27: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/27.jpg)
Performance
• The experiments are run on the topology of Figure 1 (reproduced here)
• Host A sends a “layer-2 ping” packet to host B which replies with a packet to A.
• The controller runs the MAC-learning switch program of Figure 3.
• The numbers of transitions and unique states, and the execution time as the number of concurrent pings is increased (a pair of packets) are reported.
![Page 28: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/28.jpg)
Performance
![Page 29: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/29.jpg)
Performance
Comparison to state-of-the-art model checkers SPIN and JPF
• SPIN performs a full search more efficiently, but state-space explosion occurs and SPIN runs out of memory with 7 pings
• SPIN's Partial-order reduction only decreases the growth rate by 18%
• JPF is already slower than nice, and even after extra effort is put into improving the Java model used in JPF, it is still substantially slower.
![Page 30: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/30.jpg)
NICE and Real Applications
• NICE detected 3 bugs in a MAC-learning switch (PySwitch)o Host unreachable after movingo Delayed direct patho Excess flooding
![Page 31: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/31.jpg)
NICE and Real Applications
• NICE found several bugs in a Web Server Load Balancer (dynamically reconfigures data center traffic based on load)o Next TCP packet always dropped after
reconfigurationo Some TCP packets dropped after
reconfigurationo ARP packets forgotten during address
resolutiono Duplicate SYN packets during transitions
![Page 32: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/32.jpg)
NICE and Real Applications
• Energy-Efficient Traffic Engineering (selectively powering down links and redirecting during light loads)o First packet of new flow is droppedo The first few packets of a new flow can be
droppedo Only on-demand routes used under high
loadso Packets can be dropped when the load
reduces
![Page 33: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/33.jpg)
Overhead of running NICE
![Page 34: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/34.jpg)
Coverage vs. Overhead Trade-Offs
There are some current limitations to NICE with the current approach
• Concrete execution on the switch
• Concrete global controller variables
• Infinite execution trees in symbolic execution
![Page 35: Openflow App Testing Chao SHI, Stephen Duraski. Motivation Network is still a complex stuff ! o Distributed mechanism o Complex protocol o Large state.](https://reader036.fdocuments.us/reader036/viewer/2022062515/56649c735503460f94925ae7/html5/thumbnails/35.jpg)
Related work
• NICE uses ideas in model checking and symbolic execution, and is the first to apply those techniques to testing OpenFlow networks.
• Other work such as Frenetic, OFRewind, and FlowChecker can perform limited testing in OpenFlow networks.