1 Per Gunningberg© A Real-World Test-bed for Mobile Ad hoc Networks: Methodology, Experimentations,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Per Gunningberg© A Real-World Test-bed for Mobile Ad hoc Networks: Methodology, Experimentations,...
1Per Gunningberg©
A Real-World Test-bed for Mobile Ad hoc Networks:
Methodology, Experimentations, Simulation and Results.
Per Gunningberg, Erik Nordström, Christian Rohner, Oskar WiblingUppsala University
2Per Gunningberg©
Background and problem
IETF is standardizing MANET (Mobile Adhoc NETwork) routing protocols:
– One proactive protocol - knowledge about all nodes– One reactive protocol - path on the need basis
Based on experiences from three protocols:– AODV - Adhoc On Demand Distance Vector
(reactive)– DSR - Dynamic Source Routing (reactive)– OLSR - OptiMized Link State Routing(proactive)
Problem: But majority of research done through simulations...
3Per Gunningberg©
Part One
A test-bed for evaluating ad hoc routing protocols.
Close to reality
What to measure and how to analyze
Repeatable experiments
Grey Zone Phenomena
Conclusion
4Per Gunningberg©
The Uppsala Ad hoc Protocol Evaluation Testbed (APE)
People carrying laptops with 802.11b
Suitable for indoor experiments that are hard to model in simulation
6Per Gunningberg©
The Ad hoc Protocol Evaluation Testbed (APE)
Execution environment on top of existing OS.– Runs on Win and Linux
Scenarios with movement choreography.
Emphasizes easy management for scaling.
800++ downloads.
7Per Gunningberg©
Laptop instructions (choreography)
node.11.action.0.msg=Test is starting... node.11.action.0.command=start_spyd node.11.action.0.duration=1 node.11.action.1.command=my_iperf c 2 t 330 node.11.action.1.msg=Stay at this location.node.11.action.1.duration=30 node.11.action.2.msg=Start moving! Go to Point A, the end of building. node.11.action.2.duration=75 node.11.action.3.msg=You should have arrived at Point A. Please stay. node.11.action.3.duration=30
8Per Gunningberg©
Measurement procedures
Every node collects SNR from every other node it can hear during the test session
Every event is time stamped
Received Packets/Application results are collected at all nodes
Routing state snapshots are collected
Analysis is done after the test session.
9Per Gunningberg©
Replaying a scenario
• SNR mapped to virtual distance• Each time interval corresponds to a
topological map
T5025 12510
075
150
Point A
Point D
10Per Gunningberg©
APE is a Testbed for…
1. Relative protocol performance comparisons
2. Radio channel effects on ad hoc routing protocols
3. Interactions between hardware, software, protocol, mobility and radio environmentExample: Grey Zone Phenomena
4. Validation of simulation models
5. Generation of traces
12Per Gunningberg©
Challenge
Results should be reproducible and comparable between tests
It follows that experiments must be repeatable...
...and therefore stochastic factors need to be dealt with
So – what can we achieve?
13Per Gunningberg©
Stochastic Factors in Real World Experiments
Node mobility adds frequent changes in the network topology.– We use choreography and “measure
topology differences”
Variations in hardware and software configuration.– We use identical hardware and software.
Time varying radio environment affects link quality and error rates.
14Per Gunningberg©
Topology differences - visual check
RED = Average mobilityGREEN = 25% with lowest mobilityBLUE = 25% with highest mobility
Experiment 1 Experiment 2
15Per Gunningberg©
Part Two
Evaluating MANET protocols with the APE testbed, simulation and emulation.
Scenarios
UDP, Ping and TCP
Side-by-side comparison
Faulty protocol constructs
Conclusion
17Per Gunningberg©
Routing protocols ability to adapt
OLSR - Proactive Link state routing. Monitors neighbors and exchange link state info.
AODV - broadcasts to set up path. HELLO or Link feedback to detect link failure.
DSR - broadcasts with source route. Listens to other traffic to find shorter route. RTT measurements and network ACKs.
React to connectivity changes
18Per Gunningberg©
Emulation
•Same configuration as Real world
•Table-top emulation
•MAC filters force connectivity changes
•Reduces radio and mobility factors
•Interference reduces bandwidth
19Per Gunningberg©
Simulation
•Scenarios recreated in a ns2-simulation using “default” models:
– Transmission range tuned to better match indoors– Mobility with jitter modeled after real world
measurements– Results averaged over 10 runs
•Results provide a baseline
•Can simulations using default (simple) models be used to predict routing protocol performance in complex real world environments?
20Per Gunningberg©
Multidimensional Comparison
Three MANET routing protocol implementations:– OOLSR, AODV-UU, DSR-UU
Three traffic types:– UDP (20 pkts/s CBR)– Ping (20 pkts/s CBR)– TCP (File transfer)
Three mobility scenarios:– End node swap, Relay node swap, Roaming node
Three environments (dimensions):– Simulation, Emulation, Real world
3x3x3x(10 runs) = 270 runs
21Per Gunningberg©
Experimental Test Environment
•Indoors with offices and corridors
•Four nodes (0, 1, 2, 3)
•Four waypoints (A, B, C, D)
•One data stream from node 3 to node 0
23Per Gunningberg©
Scenarios – Relay Node Swap
•End nodes stationary
•Intermediate nodes changes position
•Hop count never smaller than 2
25Per Gunningberg©
Scenarios – End Node Swap
•End nodes change positions
•Intermediary nodes stationary
•Hop count changes from 3 to (2) and 1 and back
27Per Gunningberg©
Scenarios – Roaming Node
•Roaming node is source node
•All other nodes stationary
28Per Gunningberg©
Results – Relay Node Swap
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
Ping (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
TCP (Mbps)
AODV-UU DSR-UU OOLSR
29Per Gunningberg©
Results – End Node Swap
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.5
1
1.5
2
2.5
3
TCP (Mbps)
AODV-UU DSR-UU OOLSR
30Per Gunningberg©
Results – Roaming Node
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
UDP (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.2
0.4
0.6
0.8
1
1.2
Ping (Delivery ratio)
AODV-UU DSR-UU OOLSR
Simulation Emulation Real World
0
0.5
1
1.5
2
2.5
3
3.5
4
TCP (Mbps)
AODV-UU DSR-UU OOLSR
34Per Gunningberg©
Observations
Simulation and Emulation similar in absolute CBR performance but not in relative protocol ranking
Real world CBR performance is significantly lower
Discrepancy grows with traffic complexity and scenario
TCP performance is orders of magnitude lower for real world compared to simulation
periods of no-progress time in real world
35Per Gunningberg©
Observations (continued)
OLSR tries less hard to re-route and therefore achieves more even performance
•Radio factors account for most of the discrepancy between simulation and real world...
•...but secondary effects, such as cross-layer interactions that are protocol specific, dominate, e.g.:
– Lost HELLOs (AODV)– Excessive buffering (DSR)
36Per Gunningberg©
Protocol comparison conclusion
If one protocol performs better than another in simulation, is it possible to assume the same for
the real world?
NO
37Per Gunningberg©
Latency - Ping - Relay Node
Simulation Real World
0
500
1000
1500
2000
2500
3000
3500
4000
Relay node swap (Ping) RTT std. dev.
AODV-UU DSR-UU OOLSR
41Per Gunningberg©
Conclusions
APE aims to address the lack of real world ad hoc experimental research test-beds
Repeatability addressed at a level that allows relative protocol comparisons
The value of cross-environment evaluation
Revealing of sensing problems leading to instabilities and poor performance
Not visible in simulations
42Per Gunningberg©
The End
Paper:
http://www.it.uu.se/research/group/core/publications/GC_technical_report.pdf
APE testbed:http://apetestbed.sourceforge.net/
The Research group:http://www.it.uu.se/research/group/core/