Internet Research Tracings_ Towards Non-Reductionist Methodology
Towards a Methodology for the Development of Routing ...
Transcript of Towards a Methodology for the Development of Routing ...
Towards a Methodology for the Development ofRouting Algorithms in Opportunistic Networks
Diego Freire, Carlos Borrego and Sergi Robles
Dept. of Information and Communications EngineeringUniversitat Autonoma de Barcelona
October 16, 2020
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 1 / 22
Speaker
Diego Mauricio Freire Bastidas is a third year PhD student inthe Dept. of Information and Communications Engineering atAutonomous University of Barcelona. The research interest isfocused in the study of Opportunistic Networks.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 2 / 22
Overview
1 Introduction to Opportunistic Networking (OppNet)
2 State of the art
3 Routing algorithms in OppNet
4 Transform ideas to a real-world application
5 A full step methodology
6 Scenarios in OppNet
7 Why comparing routing algorithms?Traps in the comparison
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 3 / 22
Introduction to Opportunistic Networking (OppNet)
Opportunistic networks highlights:
set of mobile devicesexchange information by exploiting direct communication opportunitiesan end-to-end route never existnodes does not have any knowledge about the network topologyOpportunistic networks allow a flexible and highly dynamicconnection between the nodesTopology network is continuously changing
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 4 / 22
State of the Art
OppNets
cellular network offloading, communication in challenged ar-eas, censorship circumvention and proximity-based applica-tions and Internet of Things (IoT), among others
Evaluation strategies
A standard methodology that allows the evaluation of routingalgorithms in the OppNets field does not exist.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 5 / 22
Routing algorithms in OppNet
Routing protocols in OppNets works on the basis to send amessage from a source to a destination even if an end-to-endroute never exist.
Information is sent ”opportunistically” hop by hop betweensource to destination using the” Store-Carry-and-Forward”approach.
Routing protocols in OppNets can be classified into threebroad families: replication, forwarding, and hybrid family.
Finding the optimum path is a challenge regarding the con-stant change in the network’s topology.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 6 / 22
Methodology for developing routing algorithms
This paper introduces a methodology for the development of routing algorithmsthat takes into consideration opportunistic networking. The proposal focus on therationale behind the methodology, and highlights its most important stages andcomponents. It also discusses the importance of two core elements in the processof protocol designing: the scenario selection, based on essential characteristics,and the choice of standard evaluation metrics. As of now, there has been nocommon methodology for developing new routing algorithms, and this has ledto proposals difficult to compare, to evaluate, and lacking a rigorous objectivityensuring fairness. Thus, there is the urgent need to propose, agree, and use acommon methodology for the development of routing algorithms.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 7 / 22
Methodology stages
1.Conception
2.Model
3.Analysis
4.Simulation
5.Implementation
6.Emulation
7.Application
The methodology has seven well differentiated stages, therationale behind the methodology, and highlights its mostimportant stages and components.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 8 / 22
Methodology stages
Conception: the initial idea behind the algorithm, theconception of the mechanisms.
Model After the initial stage of conception, the idea mustbe reified into a particular mathematical model, which canthen be analyzed formally.
Analysis During this analysis, the mechanisms andprocedures can be checked, some theoretical results can beobtained, and basic limitations can be identified.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 9 / 22
Methodology stages
Simulation In a simulator, the model can be tested in agiven set of scenarios. Eventhough these scenarios involvedatasets that come from the real world (e.g., real traces fromvehicles or people), or even if the simulator simulates veryaccurately all network protocols involved, the model underevaluation is usually executed based on pseudo-code. *Thisdoes not prove that the system being designed can eventuallybe deployed and used for real.
Implementation In this stage, a code is produced so thatthe algorithm can be used on a real scenario. The implemen-tation itself shows the feasibility of the algorithm.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 10 / 22
Methodology stages
Emulation: Emulation allows to run real code in tightlycontrolled (and repeatable) conditions. This stage is the linkbetween a proof-of-concept implementation and the deploy-ment of a software that is useful in the real world and behavesas predicted.
Application The last stage of this methodology is testing therouting algorithm in a real environment, with real devices andusers. This is the ultimate test that shows how the designedalgorithm behaves in the real world and allows to evaluate.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 11 / 22
Methodology stages
The application of the methodology should not be strictly sequen-tial. Some of the work in one stage can help to improve some ofthe previous stages.Applying a methodology like the one described is necessary, butnot enough to produce good quality routing algorithms.
There are two elements that have also to be considered:
Scenarios
Performance metrics
Simulation and Emulation stages need some scenarios
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 12 / 22
Scenarios and metrics in opportunistic networks
Different scenarios and metrics are generally used in articles tomeasure the performance of Routing Algorithms.
As it is difficult to reproduce real-world conditions, the use ofscenarios tries to create a model of them, assuming performancewill be similar.
Many papers use several scenarios to show the algorithm has alarge scope of applicability.
However, articles use to pay little or no attention to the selectionand definition of these scenarios, neglecting the importance theydeserve in the significance of the results.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 13 / 22
Scenario’s components
Scenarios of OppNet consist of a set of nodes and their positionsduring a time frame. When analysing the behaviour of a routingalgorithm on a scenario, more details have to be provided, suchas a set of messages (with recipients and size), and the commu-nication range of the nodes.
S = (Nodes,Messages,NodeContacts) (1)
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 14 / 22
Selection of scenarios
Because it is impossible to test the routing algorithms in all possi-ble scenarios, making a selection of scenarios is required. This pro-cess have to be scientifically justified, to be representative enoughand avoid any possible bias.
Just having different traces that generate different results with-out analyzing the entire spectrum of action of the OppNet com-prehensively would result insufficient.
A fine selection of scenarios will act as a representation of thewhole scope of opportunistic networks.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 15 / 22
All-in-One scenario trap
A valuable scenario aims to be a good representation of reality.This representation must introduce as many elements as the realevent contains, but this could end up into an unrealistic taskdue to the number of characteristics involved.
Therefore, modelling a scenario requires a balance betweensimplification and real-world accuracy has a direct implication ofusability.
Not every characteristic must be taken into account.
Instead of build an ”All-in-One” scenario, the development of aset of different scenarios,
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 16 / 22
Performance metrics
Metrics help to answer the question: how well is a scenarioworking? The evaluation function can be represented as:
M = M1,M2, ...,Mn
Eval(A,S ,M) = (M1(A, S),M2(A,S), ...,Mn(A, S)) (2)
Metrics should always refer to the same measurable properties,thus all proposals have to use exactly the same names for themetrics to avoid confusion.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 17 / 22
Routing comparison
We studied more than 50 opportunistic routing algorithms.Approximately a 62% of the algorithms tests their performance withone or two algorithms when they are presentedLess than 5% of the reviewed algorithms present a comparisonwith 6 algorithm
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 18 / 22
Comparison with Epidemic
Epidemic routing in a simplistic view:
algorithm sends the message to all of their reachableneighbours
Why this is a concern in a comparison?:The algorithm does not have to make any decision whether to
send a message or not.
Epidemic routing is the most compared-to algorithm.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 19 / 22
Fairness in the comparison
Own evaluation metrics and scenariosProposals normally compare their protocol to others
Testing environmentChosen scenariosMetrics and conditions
Bias can be introduced unintentionallyReproducibility??
A way of alleviating these effects and giving better prospects to the (use-ful) development of new routing algorithms in ooportunistic networkingis by using a methodology like the one presented in this paper.
This can decisively help on the difficult task of starting a cultural changeon routing algorithm development, proposal evaluation, and algorithmselection.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 20 / 22
Acknowledgment
This work is partly funded by Secretaria de Educacion Superior,Ciencia, Tecnologıa e Innovacion (SENESCYT, ECUADOR), bythe Catalan AGAUR 2017SGR-463 project, and by the SpanishMinistry of Science and Innovation TIN2017-87211-R project.
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 21 / 22
Thanks
D. Freire, C. Borrego & S. Robles Universitat Autonoma de Barcelona October 16, 2020 22 / 22