Ph.D. Defense

Experimentation Toolsfor Networking

ResearchMathieu Lacage

INRIA, Planète

Nov 15th 2010

Lacage (INRIA) Experimentation Tools & Network Research Nov 2010 1 / 55

Outline

Motivation

Simulated Packets

Direct Code Execution

NEPI

Conclusion

Perspectives


Protocol evaluation


Protocol evaluation

Analyticalanalysis


Protocol evaluation

Analyticalanalysis

Simulator


Protocol evaluation

Analyticalanalysis

Simulator Testbed


Protocol evaluation

Analyticalanalysis

Simulator Testbed

Smallscalefield


Protocol evaluation

Analyticalanalysis

Simulator Testbed

Smallscalefield

Largescalefield


Protocol evaluation

Analyticalanalysis

In thewild

Simulator Testbed

Smallscalefield

Largescalefield


Experimentation contextExperimentation

Realism

Time


Why ?

I One protocol isolated

-> Multiple protocolinteraction

I Steady state -> Transient behavior


Why ?

I One protocol isolated -> Multiple protocolinteraction



Why ?


I Steady state

-> Transient behavior


Why ?




Cheaper hardwareHardware

Cost

Time


ConsequencesMore testbeds

More Field tests

Time


ConsequencesMore testbeds

More Field testsLess

SimulationsAlone

Time


ConsequencesMore Testbed+

Simulation

Time


Downsides

A lot more workI Must master many experimentation

environmentsI Must implement protocols twice


Downsides

A lot more work

I Must master many experimentationenvironments

I Must implement protocols twice


Downsides


environments

I Must implement protocols twice


Downsides


environmentsI Must implement protocols twice


Objectives

I Run real protocol implementation in simulationI Use simulation as realtime emulator


ObjectivesSmooth transition simulations/testbed/field tests


FieldSimulation



I Run real protocol implementation in simulation

I Use simulation as realtime emulator

Field

RealCode

SimulationSimulation




Field

RealCode

Simulation FieldSimulation

RTSim


Problems

Run real protocol implementation in simulation

I Virtualization of execution environment

I Direct Code Execution

I Setup, deployment, of emulation platforms

I NEPI

I Transparency with other simulation models

I Simulated Packets


Problems

Run real protocol implementation in simulationI Virtualization of execution environment


Real Code

Simulation


I NEPI


I Simulated Packets


Problems



Use simulation as realtime emulator


I NEPI


I Simulated Packets


Problems



Use simulation as realtime emulatorI Setup, deployment, of emulation platforms

I NEPI

RTSim

RTSim

FieldField

DeploymentControl


I Simulated Packets


Problems




I NEPI

In both cases


I Simulated Packets


Problems




I NEPI

In both casesI Transparency with other simulation models

I Simulated Packets


Problems


I Direct Code ExecutionUse simulation as realtime emulator


I NEPI


I Simulated Packets


Problems


I Direct Code ExecutionUse simulation as realtime emulator

I Setup, deployment, of emulation platformsI NEPI


I Simulated Packets


ns-3 Contributions

I ns-3: 300 KLOCI Wrote about 80 KLOC


ns-3 Contributions

I ns-3: 300 KLOC

I Wrote about 80 KLOC

Wimax

TCP

Mesh

UDP/Ipv6

MPI

AODV OLSR

Uan Bridge

Spectrum

Csma PointToPoint

Core

Wifi

UDP/IPv4/ARP

Mobility


ns-3 Contributions

I ns-3: 300 KLOCI Wrote about 80 KLOC

Wimax

TCP

Mesh

UDP/Ipv6

MPI

AODV OLSR

Uan Bridge

Spectrum

Csma PointToPoint

Core

Wifi

UDP/IPv4/ARP

Mobility


Scientific Contributions

Simulated packetsI More CPU efficient than other simulatorsI Automatic conversion simulation/network format

Direct Code ExecutionI 10x more CPU efficient than other DCE

frameworksI Large applicability scope

NEPII Unified experiment descriptionI Automated deployment


Scientific ContributionsSimulated packets

I More CPU efficient than other simulatorsI Automatic conversion simulation/network format






I More CPU efficient than other simulators

I Automatic conversion simulation/network formatDirect Code Execution

I 10x more CPU efficient than other DCEframeworks

I Large applicability scopeNEPI

I Unified experiment descriptionI Automated deployment





I 10x more CPU efficient than other DCEframeworks







frameworks








NEPI







NEPII Unified experiment description

I Automated deployment


Outline

Motivation

Simulated Packets


NEPI

Conclusion

Perspectives


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Related Work

Two approaches

I Packet is list of headers: GTNetS, OMNeT++,SSFNet

TCPIP PayloadMAC

I Packet is buffer of bytes: Yans, GloMoSim

TCPIP PayloadMAC


Related Work

Two approachesI Packet is list of headers: GTNetS, OMNeT++,

SSFNet

TCPIP PayloadMAC

I Packet is buffer of bytes: Yans, GloMoSim

TCPIP PayloadMAC


Pros and Cons

List BufferFragmentation, Reassembly XConversion real bytes XSimulation-only data XPretty printing XCPU, memory efficiency X


Our solutionPacket as buffer of bytes

I Fragmentation, ReassemblyI Automatic conversion to/from real bytes

TagsI Simulation-only data

MetadataI Pretty printing

Copy On Write (COW)I CPU, memory efficiency


Performance evaluation

Micro-benchmark scenariosI ReceptionI ForwardingI TransmissionI Re-transmission


Relative speedup

ns-3 OMNet++ GTNetS YansReception 1.00 1.98 2.25 3.07Transmission 1.00 2.13 1.78 2.16Forwarding 1.00 2.83 1.92 2.44Retransmission 1.00 1.92 1.70 4.76


Contributions

More CPU efficient than other simulators

Transparent support for real network bytes


Outline

Motivation

Simulated Packets


NEPI

Conclusion

Perspectives


The manual approachGlobal variables

static int g_var;g_var ++;

static int [100]g_var_array;g_var_array[current_id ()] ++;

Redirect system calls

clock (); dce_clock ();

Re-implement all system calls

clock_t dce_clock (void) {return Simulator::Now ().GetMicroSeconds ();

}


The problem

Manual modifications: does not scaleI Painful to do onceI Impossible to do for software updates


Related work

Network Simulation CradleI Automated source modifications for C codeI Hard to extend to C++

WeavesI Automated textual assembly modificationsI Invalid assumptions about compiler-generated

codeCOOJA

I Automated memory virtualizationI Slow


Related workNetwork Simulation Cradle

I Automated source modifications for C codeI Hard to extend to C++


codeCOOJA






code

COOJAI Automated memory virtualizationI Slow





codeCOOJA



ns-3 DCE

Executable and Linkable Format (ELF) loaderI FastI Automated memory virtualizationI Automated system call redirection

Userspace system callsKernelspace system calls


ns-3 DCE


Userspace system calls

Kernelspace system calls


ns-3 DCE


Userspace system callsKernelspace system calls


Loader Performance

Scenario:

Linux

UDP/IP

PointToPointLink

Linux

UDP/IP

PointToPointLink

kernelspace DCE kernelspace DCE

Linux

UDP/IP

kernelspace DCE

udp-perf

userspace DCE

udp-perf

userspace DCE


Loader Performance

Packets perwall clocksecondvsNumber ofnodes

100

1000

10000

100000

0 10 20 30 40 50 60 70

Coojans-3


Loader Performance

Memory(bytes)vsNumber ofnodes

0

5000

10000

15000

20000

25000

30000

35000

0 10 20 30 40 50 60 70

Coojans-3


System Performance

Packets perwall clocksecondvsNumber ofnodes

1000

10000

100000

1e+06

0 2 4 6 8 10 12 14 16 18 20

dce-nonedce-user

dce-user+kernel


Tested programs

Binary Missing functions/sbin/ip 24/bin/ping 2/bin/traceroute 5/usr/sbin/zebra 42/usr/sbin/ospfd 43/usr/sbin/mip6d 38


Untested programs

Binary Missing functionsccnd 11/usr/sbin/httpd 18


Contributions

10x more efficient than existing alternatives

Larger scope than existing alternativesI Userspace: ping, traceroute, quagga, etc.I Kernelspace: Linux IP, TCP, etc.

Potential usecases:I Debugging platform: single debugger controls

all protocol instancesI Development platformI Test platform


Contributions

10x more efficient than existing alternativesLarger scope than existing alternatives

I Userspace: ping, traceroute, quagga, etc.I Kernelspace: Linux IP, TCP, etc.




Contributions




all protocol instances

I Development platformI Test platform


Contributions




all protocol instancesI Development platform

I Test platform


Contributions






Outline

Motivation

Simulated Packets


NEPI

Conclusion

Perspectives


Objective Scenario

Simulated links/networksI ns-3 simulation models XI Realtime scheduler XI Tap device X

Light weight Virtual MachinesI Linux Network Namespaces X


Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient

Simulated links/networksI ns-3 simulation models XI Realtime scheduler XI Tap device X

Light weight Virtual MachinesI Linux Network Namespaces X


Easy Deployment

ProblemI Tap/VM creation and setupI Coherent IP address assignment across

simulation and VMsI Coherent IP forwarding tables across

simulation and VMsSolution

I Automate everythingBUT

I Need global view of experiment topology


Easy DeploymentProblem

I Tap/VM creation and setup

I Coherent IP address assignment acrosssimulation and VMs

I Coherent IP forwarding tables acrosssimulation and VMs

SolutionI Automate everything

BUTI Need global view of experiment topology



I Tap/VM creation and setupI Coherent IP address assignment across

simulation and VMs

I Coherent IP forwarding tables acrosssimulation and VMs







simulation and VMs








I Automate everything







I Automate everythingBUT

I Need global view of experiment topology


Related Work

I Emulab Tcl: ad hoc, hard to generalizeI OMF Ruby: unclear how to extend it to model

complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


Related Work

I Emulab Tcl: ad hoc, hard to generalize

I OMF Ruby: unclear how to extend it to modelcomplex topologies

I OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


Related Work


complex topologies

I OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


Related Work


complex topologiesI OMNeT++ NED: hard to ensure correctness

I SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


Related Work


complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctness

I Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


Related Work


complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctness

I Geni Omnispec: hard to ensure correctness


Related Work


complex topologiesI OMNeT++ NED: hard to ensure correctnessI SSF DML: hard to ensure correctnessI Geni RSPEC: hard to ensure correctnessI Geni Omnispec: hard to ensure correctness


NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checkingHierarchical


NEPI Object Model

Functional unit / Box

AttributesTrace sourcesConnectorsConnection checkingHierarchical

Example:I IP stackI TCP stackI Ethernet card


NEPI Object Model

Functional unit / BoxAttributes

Trace sourcesConnectorsConnection checkingHierarchical

Example:I IP checksumI IP forwarding


NEPI Object Model

Functional unit / BoxAttributesTrace sources

ConnectorsConnection checkingHierarchical

Example:I Out packetsI In packets


NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectors

Connection checkingHierarchical

Ethernet

IP

dev

node


NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checking

Hierarchical

Ethernet

IP

Ethernet

dev

node node

app


NEPI Object Model

Functional unit / BoxAttributesTrace sourcesConnectorsConnection checkingHierarchical

Ethernet

Switch

Ethernet

node node

Node

dev

cablecable

port0 port1


Objective Scenario

VLCServer

RouterWifiAP

WifiSTA

Router

VLCClient


NEPI Representation

Mobility

WifiNet

Device

Node

ARPICMPIP

TapNodeInterface

Node

FDNetDevice

Mobility

FDNetDevice

YansWifiPhyYansWifiPhy

Delay

YansWifiChannel

Loss

Nist Nist

WifiNet

Device

Manager

ApMac

Node

ARPIP ICMP

Manager

StaMac

TapNodeInterface

Node

ns-3 NetNsNetNs

VlcVlc


Global IP topology

net2net1net0 Wifi Ap Wifi Sta FdFdTap Tap

n1n0 n2 n3


Demo screenshot


Contributions

An integrated experimentation environmentI Unified experiment descriptionI Entire workflow supportI Automated deploymentI Strong coherency checking


Outline

Motivation

Simulated Packets


NEPI

Conclusion

Perspectives


Papers Published

I “Yet another network simulator”, Proceedings ofthe 2006 workshop on ns-2

I “NEPI: using independent simulators,emulators, and testbeds for easyexperimentation”, SIGOPS Operating

Systems Review


Papers in progress

I “The ns-3 network simulator: experiencelearned”, Software Practice & Experience

I “Direct Code Execution”, Networked SystemsDesign and Implementation


Activities

ChairI NSTools 2007

TPCI WNS3 2009, 2010, 2011I Simutools 2008,2009


Simulated Packets

More CPU efficient than other simulators

Transparent support for real network bytes



10x more CPU efficient than other DCE frameworks

A robust implementationI Userspace and kernelspace protocolsI C,C++ protocolsI ABI compatibility for userspace protocols


NEPI

An integrated experimentation environmentI Unified experiment descriptionI Entire workflow supportI Automated deploymentI Strong coherency checking


Impact

Instrumental in creating anI ActiveI GrowingI Open source

Simulation communityI Contributors > 80I Hundreds of usersI > 30 papers by users

2 sigcomm’091 sigcomm’10


ImpactInstrumental in creating an

I ActiveI GrowingI Open source

Simulation community

I Contributors > 80I Hundreds of usersI > 30 papers by users





Simulation communityI Contributors > 80

I Hundreds of usersI > 30 papers by users


0

0.1

0.2

0.3

0.4

0.5

0.6




Simulation communityI Contributors > 80I Hundreds of users

I > 30 papers by users2 sigcomm’091 sigcomm’10

0

100

200

300

400

500

600

700

800




Simulation communityI Contributors > 80I Hundreds of usersI > 30 papers by users



Outline

Motivation

Simulated Packets


NEPI

Conclusion

Perspectives


Tracing with DCE

I Need to recompile for tracing changesI Ideal workflow:

I Trace function foo in bar.ccI Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc

I BUT, not Java: no introspection


Tracing with DCE

I Need to recompile for tracing changes

I Ideal workflow:I Trace function foo in bar.ccI Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc



Tracing with DCE





Tracing with DCE


I Trace function foo in bar.cc

I Trace line 122 in bar.ccI Trace variable foo at line 144 in bar.cc



Tracing with DCE


I Trace function foo in bar.ccI Trace line 122 in bar.cc

I Trace variable foo at line 144 in bar.cc



Tracing with DCE





Dynamic instrumentation

I Locate code in memoryI Parse dynamic loader data structuresXI Parse debugging information

I Insert assembly hooksI Simple case (not thread-safe) XI General case harder



I Locate code in memory

I Parse dynamic loader data structuresXI Parse debugging information




I Locate code in memoryI Parse dynamic loader data structuresX

I Parse debugging informationI Insert assembly hooks

I Simple case (not thread-safe) XI General case harder




I Insert assembly hooks

I Simple case (not thread-safe) XI General case harder




I Insert assembly hooksI Simple case (not thread-safe) X

I General case harder


Multi-threaded Scheduler

Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?

Transparency is key !



Conservative algorithm implemented:

I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?




Conservative algorithm implemented:I Fully transparent for users

I Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?




Conservative algorithm implemented:I Fully transparent for usersI Measurable speedup

I Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?




Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easy

I Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?




Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hard

I Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?




Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?

I Can we use optimistic algorithm transparently ?Transparency is key !



Conservative algorithm implemented:I Fully transparent for usersI Measurable speedupI Thread-safety is easyI Efficient thread-safety is hardI Can we build more efficient thread-safety ?I Can we use optimistic algorithm transparently ?



Thank you !

Questions ?


Ph.D. Defense

Business

Transcript of Ph.D. Defense