cordis.europa.eu€¦ · At A Glance: NetIDE An integrat ed development environment for portable...

At A Glance: NetIDE An integrated development environment for portable network applications

Project Coordinator Elio Salvadori CREATE-NET

Tel: +0461-408400 (1301) Fax: +39-0461-421157

Email: [email protected]

Project website: http://www.netide.eu Partners: CREATE-NET (IT) CZ.NIC (CZ) Fujitsu Technology Solutions (DE) IMDEA Networks (ES) Intel (IR) Telefonica I+D (ES) Thales Communication & Security (FR) University of Paderborn (DE) Duration: 01/2014 – 12/2016 Funding scheme: STREP Total Cost: !4.545m

EC Contribution: !2.844m Contract Number: CNECT-ICT-619543

!

An integrated development environment for portable network applications

NetIDE aims at delivering a single IDE to support the whole development lifecycle of Software-Defined Network controller programs in a vendor-independent fashion. The project includes partners from Italy, Spain, Germany, France, Ireland, Czech Republic.

Main Objectives Nowadays, while most of the programmable network apparatus vendors support OpenFlow, a number of fragmented control plane solutions exist for proprietary Software-Defined Networks (SDN). Thus, network applications developers need to re-code their solutions every time they encounter a network infrastructure based on a different controller. Moreover, different network developers adopt different solutions as high-level control plane programming language (e.g. Frenetic, Procera, Maple,!), which leads to network programs that are neither reusable nor shareable. So despite having OpenFlow as the candidate for a standard interface between the controller and the network infrastructure, interworking between different controllers and network devices is hindered and walled gardens are emerging. NetIDE will deliver a single Integrated Development Environment to support the whole development lifecycle of network controller programs in a vendor-independent fashion. The project is industry-driven thanks to the involvement of key European players and market leaders such as TID, Intel, Thales, Fujitsu and the contribution from CZ.NIC, an high-tech SME. NetIDE will constitute a relevant enabling technology for both equipment vendors and service providers, providing the possibility of much shorter development cycles, supporting flexible deployments on multi-vendor infrastructures and agile testing. Simplifying the Network Apps development process and ensuring their portability are expected to create commercial potential for European telcos and network solution providers. NetIDE major components will be released according to an open-source license with the aim of promoting the establishment of an open SDN model based on an Open Community of developers.

NetIDE will result in one-stop solution for the development

of SDN applications that

covers all the development

lifecycle

N E T I D E FP7 ICT Objective 1.1 Future Networks

Document: CNET-ICT-619543-NetIDE/D 4.3Date: April 21, 2017 Security: PublicStatus: Final Revised Version: 2.0

Document Properties

Document Number: D 4.3

Document Title: NetIDE App Engine �nal Release and Evaluation

Document Responsible: IMDEA

Document Editor: Carmen Guerrero (IMDEA)

Authors:

Carmen Guerrero (IMDEA), Elisa Rojas (TELCA)Roberto Doriguzzi Corin (CREATE-NET), Alec Leckey (Intel)Antonio Marsico (CREATE-NET), Arne Schwabe (UPB)Sergio Tamurejo (IMDEA), Kevin Premious (Thales)

Target Dissemination Level: PU

Status of the Document: Final Revised

Version: 2.0

Production Properties:

Reviewers: Pedro A. Aranda (TID), Elio Salvadori (CREATE-NET)

Document History:

Disclaimer:

This document has been produced in the context of the NetIDE Project. The research leading to these results hasreceived funding from the European Community's Seventh Framework Programme (FP7/2010�2013) under grantagreement n◦ 619543.All information in this document is provided �as is� and no guarantee or warranty is given that the informationis �t for any particular purpose. The user thereof uses the information at its sole risk and liability.For the avoidance of all doubts, the European Commission has no liability in respect of this document, which ismerely representing the authors view.

NetIDEPublic

i



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Abstract:

This document is the last deliverable of the WP4 and reports the work on the NetIDE Engineand Tools during the last year of the project. This deliverable describes the main componentsof the �nal release of both the Engine and Tools. There is also a description and results of theperformance evaluation tests done on some of the components of the NetIDE �nal release.

Keywords:

Engine, Core, Shim, Backend, OpenDaylight, Ryu, ONOS, Floodlight, logger, pro�ler, debugger, memory

management system, Veri�cator, performance evaluation

NetIDEPublic

ii



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Contents

List of Figures iv

List of Acronyms vi

1 Introduction 1

2 The NetIDE Network Engine Final Release 2

2.1 Evolution of the NetIDE Engine Architecture . . . . . . . . . . . . . . . . . . . . . . 32.2 The Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Ryu Backend and Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3.1 Ryu Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3.2 Ryu Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 The ODL Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5 ONOS Backend and Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5.1 ONOS Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.5.2 ONOS Shim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.6 The Floodlight Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 The NetIDE Tools Final Release 15

3.1 Evolution of the NetIDE Tools Architecture . . . . . . . . . . . . . . . . . . . . . . . 153.2 Analysis of SotA on SDN Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3 Integration of external tools to NetIDE architecture . . . . . . . . . . . . . . . . . . 183.4 Logger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.5 Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.5.1 The Debugger tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.5.2 The NetIDE (NetIDE) Wireshark dissector . . . . . . . . . . . . . . . . . . . 22

3.6 Veri�cator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.6.1 Future lines of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.7 Pro�ler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.7.1 Future lines of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.8 Tra�c Emulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.9 Memory Management System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.9.1 Memory Swapping design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.9.2 Swap out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.9.3 Swap in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4 The NetIDE Engine Evaluation 34

4.1 Engine Latency Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.1.1 Test Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.1.2 Serialization/Deserialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.1.3 Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2 Engine Scalability Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.3 Tools Architecture Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.3.1 First Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

NetIDEPublic

iii



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



4.3.2 Second Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.4 Evaluation of the Memory Management System . . . . . . . . . . . . . . . . . . . . . 41

4.4.1 Test methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.4.2 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5 Conclusions 45

A Python NetIP library reference 47

A.1 netIDE_encode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47A.2 netIDE_decode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47A.3 decode_header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47A.4 encode_handshake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48A.5 decode_handshake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48A.6 netIDE_set_module_id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48A.7 netIDE_set_xid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

B Java NetIP library reference 49

B.1 Class ErrorMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50B.2 Class FenceMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51B.3 Class HeartbeatMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52B.4 Class HelloMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53B.5 Class ManagementMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54B.6 Class Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55B.7 Class MessageHeader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57B.8 Class MessageType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61B.9 Class ModuleAcknowledgeMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63B.10 Class ModuleAnnouncementMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . 64B.11 Class NetconfMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65B.12 Class NetIDEProtocolVersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66B.13 Class NetIPConverter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68B.14 Class NetIPUtils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69B.15 Class OpenFlowMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70B.16 Class OpFlexMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72B.17 Class Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73B.18 Class ProtocolVersions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74B.19 Class TopologyUpdateMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

C Bibliography 78

NetIDEPublic

iv



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



List of Figures

2.1 Network Engine architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Network Engine development timeline. . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 ONOS Backend architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Logger trace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.2 Debugger and NetIDE Wireshark dissector at the Tools GitHub repository . . . . . . 213.3 Veri�cator outcomes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.4 Veri�cator outcomes, loop detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.5 Real case for explaining the Veri�cator's operation. . . . . . . . . . . . . . . . . . . . 263.6 Execution times histogram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.7 The Highest values of execution times. . . . . . . . . . . . . . . . . . . . . . . . . . . 283.8 Network Pro�ler displaying �ow aggregate statistics. . . . . . . . . . . . . . . . . . . 29

4.1 Latency Tests Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Blue circles represents the serialization and deserialization operations that are per-

formed in each SDN architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.3 RTT measured with di�erent combinations of client-server controller frameworks. . . 364.4 Cumulative probability of the RTT for composed and merged applications. . . . . . . 374.5 Scenario for the �rst experiment of tools scalability. . . . . . . . . . . . . . . . . . . . 394.6 Number of installed rules without memory swapping at di�erent �ow table sizes. . . 434.7 Number of installed rules with memory swapping at di�erent �ow table sizes. . . . . 434.8 Throughput comparison with and without memory swapping at di�erent �ow table

sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

NetIDEPublic

v



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



List of Acronyms

AMQP Advanced Messaging Queueing Protocol

API Application Programming Interface

CLI command-line interface

EWBI East/Westbound Interface

FOSS Free and Open-Source Software

GUI Graphical User Interface

IDE Integrated Development Environment

MAC Medium Access Control

MMS Memory Management System

NBI Northbound Interface

ODL OpenDaylight

OF OpenFlow

ONOS Open Network Operating System

OSGi Open Services Gateway Initiative

SBI Southbound Interface

SDN Software De�ned Networking

TCAM Ternary Content Addressable Memory

ZMQ ZeroMQ

NetIDEPublic

vi



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



1 Introduction

The objective of this document is to report the development and implementation activities inWP4 during the last year of the project. The evolution of the architecture and the technicaldiscussion on the alternatives and the current solution adopted are discussed in the correspondingWP2 Deliverables: D2.6 NetIDE APIs and Integrated Platform [1] and D2.7 NetIDE Manuals [2].This document is organised as follows: Chapter 2 The NetIDE Engine Final Release describes

the evolution of the NetIDE Engine Architecture with detailed sections for the Core, the RyuBackend and Shim, the ODL Shim, the ONOS Backend and Shim and the Floodlight Backend.Chapter 3 The NetIDE Tools Final Release starts also describing the evolution of the NetIDETools Architecture and continues with speci�c section for each of the Tools: Logger, Debugger,Veri�cator, Pro�ler, Tra�c Emulator and Memory Management System. Chapter 4 The NetIDEEngine Evaluation describes the methodology for evaluating the NetIDE Engine and includes theresults of the latency and scalability tests. Finally, Chapter 5 Conclusions includes a summary ofthe �nal main achievements.

NetIDEPublic

1



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



2 The NetIDE Network Engine Final Release

The NetIDE Network Engine (Figure 2.1) is a Software De�ned Networking (SDN) componentwhich has been designed and implemented to allow multiple SDN applications (Modules in the �g-ure), possibly written for di�erent SDN controllers, to cooperate on controlling the same network.Additionally, it allows network managers to re-use existing SDN applications to extend the capa-bilities of an operational SDN environment independently from the controllers such modules havebeen developed for.The �nal version of the Network Engine's architecture is described in Deliverable D2.7 [2]. Fig-ure 2.1 shows the architecture plus the SDN platforms that are supported by the Network Engine.We highlight the components implemented to compose SDN modules written for di�erent controllersinto a single network application in light blue, while boxes in light green represent the companiontools that we propose for debugging the SDN applications and for improving the performance ofthe SDN network.

Figure 2.1: Network Engine architecture.

We implemented the NetIDE Network Engine based on SDN controller platforms available as Freeand Open-Source Software (FOSS)1. Speci�cally, we adopted OpenDaylight (ODL) [3] and OpenNetwork Operating System (ONOS) [4] as server controllers, Floodlight [5] as client controller andRyu [6] as both. ODL and ONOS are carrier-grade controllers targeted to service providers, enter-prises and mainstream deployments. Being able to scale to a large number of network elements, werecognized both ODL and ONOS as the most suitable SDN platforms for playing the server con-troller's role. On the other hand, Ryu and Floodlight are aimed at facilitating the rapid prototypingof SDN applications. We were able to isolate their Southbound Interface (SBI) drivers to add ourBackend alongside, and to hook into their event dispatching mechanism and drive the executionlogic.ODL and ONOS are based on Apache Karaf [7], and therefore we implemented the Shims for

these platforms as OSGi bundles, leveraging their respective Java Northbound Interface (NBI)sto interact with the network elements. Both implementations support OpenFlow versions 1.0 and1.3 and make use of serialization methods to convert messages from the controller's NBI format

1Closed-source controllers might be considered as well, but are impractical to work within a research project.

NetIDEPublic

2



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



into OpenFlow byte arrays before encapsulating them with the NetIDE header. Vice-versa, de-serialisers are used to convert byte arrays contained in the payload of NetIDE messages (OpenFlowcommands) into NBI Java classes. While for ONOS we could leverage the implementation ofserializer/deserializer methods provided by the OpenFlowJ-Loxi library [8], in ODL (which uses itsown library Open�owJava [9]) we had to implement the serializer from scratch and to extend theexisting deserializer in order to cover all OpenFlow message types.The Shim for Ryu has been implemented as an application based on the Python northbound API.

It includes serialiser and de-serialiser methods based on the Python struct module [10]The Backends for Ryu and Floodlight have been implemented by duplicating and extending the

existing OpenFlow SBIs. Following this strategy, we have been able to (i) easily understand how todistribute the events to (unmodi�ed) SDN modules, i.e. overriding the default logic of the controller,and (ii) intercept the exit point of event handlers executed by the application modules and implementthe fence mechanism [11]. We implemented the Core as a platform-independent OSGi bundle forApache Karaf to simplify the overall deployment procedure of the Network Engine. Recall thatboth ONOS and ODL are based on this framework, therefore Core, server controller and Shim canbe deployed in the same OSGi container. Message serialisation and de-serialisation in the Core arebased on the OpenFlowJ-Loxi library.Some of the tools require interfacing with the Core. This is achieved via the LogPub interface (cf.

Section 3.1) which is in charge of capturing the messages exchanged among the client controllers andthe server controller, and pushing them into a queue. Tools are implemented as external componentswhich are independent from the SDN platforms that compose the Network Engine (cf. Chapter 3).The only exception is the Memory Management System (cf. Section 3.9) which is proposed as anadditional component for SDN controllers (in our speci�c case we use it in the server controllers) toimprove the performance of the network by optimizing the usage of the Ternary Content AddressableMemory (TCAM) memory in the SDN devices in the network infrastructure.All aforementioned components are publicly available at [12] under the Eclipse Public License

v1.0 and Apache License v2.0.

2.1 Evolution of the NetIDE Engine Architecture

In Deliverable D4.1 [13], we presented the �rst version of the Network Engine. The initial proto-type did not comprise the Core to coordinate the Network Engine's controllers, as only one clientcontroller at a time was supported. For the communication between Shim and Backends we ini-tially adopted the protocol (and related API) used by Pyretic [14] between its Backend sub-moduleand the so-called POX client. With this prototype we proved that the Shim/Backend structureconnected by an intermediate protocol was feasible, and from it we started the path towards thedevelopment of an advanced multi-controller platform.

We demonstrated a version of the Network Engine based on such a protocol at NetSoft'15 [15](Figure 2.2), where we shown the whole life-cycle of a network application: from the design withthe Integrated Development Environment (IDE), to the deployment of SDN modules written for aspeci�c controller framework (we used Floodlight or Ryu) on top of a di�erent framework (in thatcase OpenDaylight).In the meantime we introduced ONOS among the platforms supported by the NetIDE , and we

demonstrated a version of the Network Engine using ONOS as server controller at SOSR'15 [16].As a �rst enhancement, we replaced the Pyretic protocol with the NetIDE Intermediate proto-

col [17] to introduce the support multiple client controllers running on the same server controller.This major change in the architecture, forced the re-implementation of the NBI of the Shims andof the SBI of the Backends to support the new communication protocol. As the same time we

NetIDEPublic

3



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



ODL/Ryu ShimsRyu/Floodlight Backends

NetIDE Protocol 1.0 Core (beta)

April 2015NetSoft'15

June 2015SOSR'15

August 2015 September 2015

ONOSShim

NetIDE Protocol 1.2

LogPub

November 2015

NetIDE Protocol 1.3

FENCEmechanism

March 2016

NetIDE Enginewith Core

June 2016NetSoft'16EuCNC'16

RyuShim 2.0

July 2016

NetIDE Protocol 1.4

August 2016

ONOS Shim 2.0 ONOS Backend

October 2016

Figure 2.2: Network Engine development timeline.

started developing inside the Shims the logic to distribute the network events to the di�erent clientcontrollers.We then decided to refactor the Shim and split what is framework-dependent from what is

framework-independent (Deliverable D2.4 [18]). The code that is indeed speci�c to a given Servercontroller framework is the current Shim. The rest of the code was moved into a separate, con-troller framework independent module, the Core of NetIDE . The Core made both Shim and Backendlight-weight and easier to implement for new controllers. Moreover, it provides a convenient placeto connect additional run-time tools using a uniform and documented interface (LogPub).We also updated the NetIDE Intermediate Protocol to speci�cation v1.2, to allow the Core to steerthe execution �ow of client controllers, by means of an announcement/acknowledgment mechanismbetween Core and Backends. With this mechanism, the Backends announce the SDN modules run-ning on top of the client controllers, while the Core acknowledges the announcement with moduleidenti�ers that the Backends insert in the header of NetIDE messages to indicate the producer ofthe message.We introduced the so-called fence mechanism with the NetIDE Intermediate protocol speci�cation

v1.3 to provide (i) a means for the Core to know when the module has �nished processing the inputevent, and (ii) a means in the Core to correlate input events with their output. We demonstrated aversion of the Network Engine based on the fence mechanism and the Core at NetSoft'16 [19] andEuCNC'16 [20].In the �nal steps in the evolution of the Network Engine architecture, we integrated modules

written for the server controller framework into the composed Network Application. Speci�cally,we placed a Backend in the server controller to steer the message �ow for the modules composingNetwork Application that run on the server controller, exactly like for any other module runningon a client controller. We also modi�ed the architecture of the Shim (the Shim 2.0 [1]) by placinghooks inside SBIs to route all messages from the network to the Shim itself, overriding the servercontroller's processing logic. Version 2.0 of the Shim has been implemented for Ryu and ONOSplatforms.In Deliverable 2.7 ([2]) we published the �nal speci�cation of the NetIDE Intermediate protocol,

where we describe how symmetric messages (e.g. statistics, feature requests, con�gurations, etc.,thus the so-called controller-to-switch messages de�ned in the OpenFlow speci�cation)) are handledby the Core. In previous versions of the Engine, this task was accomplished by the Shim.In the remainder of this chapter we provide implementation details of the current version of the

NetIDE Network Engine's components, i.e.: the Core, ONOS, OpenDaylight and Ryu Shims, andRyu and Floodlight Backends. We defer to Deliverable 2.7 [2] for the �nal speci�cation of theNetIDE Intermediate protocol.

NetIDEPublic

4



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



2.2 The Core

In order to not duplicate the content already described in the Deliverable 2.7 [2] we refer to thearchitectural design of the Core to the Core Chapter in that document.The main focus for the Core was to have a central instance that avoid re-implementations in other

parts. The decision that a Core was needed was taken when implementing a library for con�ictresolution that could be reused in the di�erent Shims proved to be to impractical. The di�erentnative programming languages (Python, Java) of the Shims alone made this impractical.Over the time more functionality was added to the Core that was independent of the Shim

and Backends. The functionality includes the interfaces for external tools (LogPub), handling ofOpenFlow routing (core.routing) and recording of installed OpenFlow messages.As already indicated in Deliverable D2.7 [2] one of the main goal for implementing the Core

was modularization and re-usability and the core.api holds all the interfaces that the modules tietogether. We will explain here now on the example of ISHimConnector (see Listing 2.1) how theseprinciples are applied. Providing these explicit interfaces in the core.api bundle allows to explicitlycontrol what interfaces are used between the modules. By only importing the core.api bundleinto other bundles, we ensure that no other interfaces can be used. For example, the core.routingmodule needs to send messages to the Shim. Instead of directly referencing the ShimConnector class,core.routing requests (via Apache Blueprint) a reference to class that implements the interfaceand is thus only able to use the method that are explicitly de�ned in the interface.

Listing 2.1: Shortened version of IShimManager.java interface of the Core

package eu.netide.core.api;

import eu.netide.lib.netip.Message;

public interface IShimManager {

boolean sendMessage(Message message);

long getLastMessageTime ();

IShimConnector getConnector ();

}

2.3 Ryu Backend and Shim

The Ryu Shim is implemented as an application based on the Ryu Python NBI, while the RyuBackend is a SBI implemented starting from the code of controller.py, which is the main com-ponent of the OpenFlow SBI of Ryu. Architectural details and manuals are provided in DeliverableD2.7 of the NetIDE project [2]. In this section we focus on implementation aspects of the functionsprovided by Shim and Backend.

2.3.1 Ryu Backend

The Ryu Backend accomplishes three main functions: (i) interfacing with the NetIDE Core via theNetIDE Intermediate protocol, (ii) recognizing and announcing the SDN modules executed by theRyu controller and (iii) extending the Datapath class implemented in controller.py by addingthe support to the NetIDE protocol and to the fence mechanism.

NetIDEPublic

5



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Interface with the Core. Interfacing with the Core is achieved via the NetIDE API as implementedin the Python NetIP library (see Annex A) and the NetIDE intermediate protocol as documentedin the Annex A of Deliverable D2.7.

Module announcement. As described in the NetIDE protocol speci�cation (Deliverable D2.7),each Backend registers itself and the SDN modules executed by the SDN controller to the NetIDECore. The list of the modules is collected by importing the app_manager class (from ryu.base

import app_manager), which provides a dictionary called SERVICE_BRICKS containing the list of allthe SDN modules.

The BackendDatapath class. It is an extension of the Datapath class implemented in controller.py�le of the Ryu source code. This modi�ed version of the class, implements the fence mechanismby sending a NETIDE_FENCE message after each event handler called in method handle_event.Moreover, the send_msg, which is the interface used by the SDN modules to send messages tothe switches (e.g. FLOW_MOD, PACKET_OUT, etc.), encapsulates the serialized SBI messages with theNetIDE header, like shown in Listing 2.2.

Listing 2.2: Encapsulation of an OpenFlow message with the NetIDE header.

def send_msg(self , msg):

...

msg.serialize ()

msg_to_send = NetIDEOps.netIDE_encode('NETIDE_OPENFLOW ', self.netide_xid ,

module_id , msg.datapath.id, str(msg.buf))

self.channel.socket.send(msg_to_send)

Instead of sending messages directly to the switches as the original send_msg method does, thismodi�ed version relays messages to the NetIDE Core via the ZeroMQ-based channel.

2.3.2 Ryu Shim

The Ryu Shim accomplishes three main functions: (i) interfacing with the NetIDE Core via theNetIDE Intermediate protocol, (ii) implementing a generic handler for all the event messages sentby the network to the server controller and (iii) announcing the control protocol negotiated withthe network to the connected clients.

Interface with the Core. Interfacing with the Core is achieved via the NetIDE API as implementedin the Python NetIP library (see Annex A) and the NetIDE intermediate protocol as documentedin the Annex A of Deliverable D2.7.

Event handler. The Shim must intercept and forward to the upper layer of the Network Engineall the events generated by the network, such as new �ows, port status, �ow removed, etc. Thismethod is di�erent from the usual implementation of event handlers for the Ryu platform, wherethe implementation is preceded by a declaration like the following:

@set_ev_cls(ofp_event.EventOFPPacketIn , MAIN_DISPATCHER)

which indicates the type of event the method is entitled to handle (in this example OpenFlowPACKET_INs) and the context (in this case MAIN_DISPATCHERmeans that the controller has completedthe handshake with the switches).Instead, the Ryu Shim registers as a listener to all the network events as follows:

for event in ofp_event._OFP_MSG_EVENTS:

self.observe_event(ofp_event._OFP_MSG_EVENTS[event ])

NetIDEPublic

6



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Control protocol. During the initial handshake with the Backends, the Shim sends the versionof the control protocol which has been previously negotiated by the server controller with theswitches. This operation avoids incompatibilities between the network applications executed by theclient controllers and the server controller's SBI used to drive the hardware.

Listing 2.3: SBI protocols used to control the devices are stored by the Ryu Shim in an internalstructure (a Python dictionary).

@set_ev_cls(ofp_event.EventOFPSwitchFeatures , CONFIG_DISPATCHER)

def _handle_ConnectionUp(self , ev):

...

msg = ev.msg

if msg.version not in self.supported_protocols[OPENFLOW_PROTO ]:

self.supported_protocols[OPENFLOW_PROTO ]. append(msg.version)

The control protocol negotiated with the switches is recognized when receiving the OpenFlowFEATURE_REPLY from the switches, as shown in Listing 2.3, and stored in a dictionary that is usedto reply to the NETIDE_HELLO message received from the client controllers, as explained in theNetIDE protocol speci�cation.

2.4 The ODL Shim

All modules of OpenDaylight are OSGi bundles that run in Apache Karaf (an OSGi container).The NetIDE module is no di�erent and follows a standardized project structure created by usingthe Maven build tool's Archetype (Maven Archetypes can be thought of as 'templates' for mavenprojects). This creates a suite of Java projects that are required to build your module.

• Features: Each module o�ers a service interface as a Karaf Feature which can be loaded/un-loaded at the command line.

<feature name='odl -netide -rest' version='0.3' description='\acl{odl} ::

Netide :: REST'>

• API: OpenDaylight uses the Yang modeling language to de�ne RPC methods. Java code isauto-generated during the build process.

rpc status {

output {

leaf netip -version {

type int8;

}

leaf -list of -versions {

type int8;

}

leaf connected -switches {

type int32;

}

}

}

• Impl: The functionality of the NetIDE API is implemented in this project

• Con�g: Con�guration of the ODL-shim module component is handled in this project. Defaultattributes are pre-set, but can be overridden if required.

NetIDEPublic

7



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



<module >

<type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:netide:netide -impl">

prefix:netide -impl

</type>

<name>netide -impl</name>

<core -address >127.0.0.1 </core -address >

<core -port>5555</core -port>

<address >127.0.0.1 </address >

<port>6644</port>

<transport -protocol >TCP</transport -protocol >

<switch -idle -timeout >15000 </switch -idle -timeout >

</module >

• NetIPlib: This is the OpenDaylight speci�c serialization library that generates the NetIDEIntermediate Protocol messages. This project di�ers from the ONOS NetIPlib in that it usesthe OpenDaylight project Open�owJava to handle serialization instead of the Loxigen library.

• Shim: This project contains the code for the majority of the functionality of the ODL-shimcomnponent, eg, connection to the Core, switches, handshakes, messageQueue relaying.

The OpenDaylight project has its own infrastructure for building and integrating all participatingmodules (more than 50 projects for the Boron release). Using the Jenkins build tool, all projectsare built nightly and then integrated into a single build that tests for compatibility issues. A reportis generated and forwarded to relevant project leads if issues are found. The OpenDaylight Boron(v5) release can be downloaded from https://www.opendaylight.org/downloads which includesthe ODL-Shim component of the NetIDE Network Engine.

2.5 ONOS Backend and Shim

The ONOS Backend and Shim are ONOS components that can be installed and uninstalled onONOS at runtime. Such components are implemented in form of Java OSGi bundles that areexecuted by Apache Karaf. In this section we provide an overview of the implementation detailsfor both ONOS Backend and Shim. Please note that, the implementation of the ONOS Backendhas been completed after the submission of Deliverable D2.7 [2], therefore in this section we alsoinclude some architectural aspects of such a component.

2.5.1 ONOS Backend

The ONOS Backend is an ONOS Provider that manages the communication between the ONOSCore and the NetIDE Core. In general, ONOS Providers are components responsible for convert-ing speci�c southbound API messages (e.g., OpenFlow, NETCONF, etc.) into ONOS Core datastructures and vice-versa. In the same manner, the ONOS Backend de-serializes the payload ofmessages received via the NetIDE intermediate protocol and converts them into ONOS data struc-tures. Finally, the ONOS Backend noti�es the message to the ONOS Core by means of Java APIs.On the opposite direction, ONOS Core messages are converted into southbound API messages,encapsulated with the NetIDE header and sent to the NetIDE Core.

ONOS Core data structures. The main data structures of ONOS Core are the following: DeviceId,FlowRule and PacketContext. They represent a network device, a forwarding rule and a packetcoming from a network device (e.g., OpenFlow PACKET_INs), respectively.

NetIDEPublic

8

https://www.opendaylight.org/downloads



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



ONOS Core Services. The interaction between the ONOS Providers and the ONOS Core isachieved by means of Java APIs, which are de�ned as ProviderService APIs. As shown in Fig-ure 2.3, this type of APIs represent the southbound of the ONOS Core. ONOS o�ers severalProviderService APIs: (i) the DeviceProviderService, which manages all the life cycle of a net-work device (e.g., noti�cation of connections, disconnections, etc.), (ii) the FlowRuleProviderServicefor the installation, uninstallation and statistics of FlowRules in the network devices and (iii) thePacketProviderService, where PacketContext from the network and from ONOS applications arehandled. The Provider APIs are Java Interfaces that are implemented in the respective Managerclasses (e.g., FlowRuleProviderService is implemented in the FlowRuleManager, etc.). On the topof Manager classes, there are the Service APIs. They provide the interaction between the ONOSCore and the ONOS applications.

Figure 2.3: ONOS Backend architecture.

ONOS Backend architecture. The ONOS Backend architecture is shown in Figure 2.3. TheONOS Backend is composed of three main classes: NetIDEDeviceProvider, NetIDEFlowRuleProviderand NetIDEPacketProvider. Every class is registered and interacts to the respective ProviderServiceAPIs. For example, as shown in Listing 2.4, the NetIDEPacketProvider is registered to thePacketProviderService API. This operation is required in order to use the methods of theseAPIs and to receive messages from the ONOS Core. Every NetIDEProvider class implementsthe methods to perform the conversion of messages coming from the NetIDE Core to the ONOSCore. For example, Listing 2.4 shows how a new connection of a network device is noti�ed to theDeviceProviderServiceAPI: A FEATURE_REPLY is converted into a DeviceId and a DeviceDescriptionstructure and then noti�ed to the ONOS Core via DeviceProviderService.deviceConnected(DeviceId,DeviceDescription) method.

Listing 2.4: Registering the ONOS Backend as PacketProvider.

import org.onosproject.net.packet.PacketProviderRegistry;

import org.onosproject.net.packet.PacketProviderService;

...

@Reference(cardinality = ReferenceCardinality.MANDATORY_UNARY)

protected PacketProviderRegistry packetProviderRegistry;

...

@Activate

public void activate(ComponentContext context) {

...

// Register a new PacketProvider in the ONOS Core

NetIDEPacketProvider packetProvider = new NetIDEPacketProvider ();

PacketProviderService packetService =

packetProviderRegistry.register(packetProvider);

...

}

NetIDEPublic

9



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Listing 2.5: Notify a new Device connection to the ONOS Core.

public class NetIDEDeviceProvider extends AbstractProvider implements

DeviceProvider {

private DeviceProviderService providerService;

...

public void registerNewSwitch(Dpid dpid , OFFeaturesReply msg) {

...

DeviceId did = deviceId(uri(dpid));

...

providerService.deviceConnected(did , description);

providerService.updatePorts(did , buildPortDescriptions(msg.getPorts ()));

...

}

}

On the other side, when ONOS applications want to interact with network devices (e.g., forinstalling a FlowRule in a network device), they use only the ONOS Core northbound via theFlowRuleService API. The ONOS Core is in charge of �nding the right FlowRuleProvider thatimplements the message conversion from the ONOS data structures to a speci�c southbound messageunderstandable by the network device. To this end, ONOS uses Uniform Resource Identi�er (URI )for both Providers and DeviceIds. For example, a network device that supports OpenFlow isregistered by the NetIDEDeviceProvider as of:openflow_device_uid. The of string identi�esthe Provider to which send a request. Indeed, all the NetIDEProviders are instantiated witha ProviderId, as shown in Listing 2.6. After the identi�cation of the right FlowRuleProvider,in our case the NetIDEFlowRuleProvider, the FlowRule is converted into a speci�c southboundmessage, in Listing 2.6 OpenFlow FLOW_MOD, and then sent to the BackendController class, whichimplements the sendOpenFlowMessageToCore (Listing 2.7).

Listing 2.6: Implementation of applyRule method.

public class NetIDEFlowRuleProvider extends AbstractProvider implements

FlowRuleProvider {

...

/**

* Creates a provider with the supplied identifier.

*/

protected NetIDEFlowRuleProvider () {

super(new ProviderId("of", "eu.netide.provider.openflow"));

}

private void applyRule(FlowRule flowRule) {

...

Dpid dpid = Dpid.dpid(flowRule.deviceId ().uri());

...

OFFlowMod flowMod = FlowModBuilder.builder (...);

...

backendController.sendOpenFlowMessageToCore(flowMod ,

(int)flowMod.getXid (), dpid.value (), moduleId);

...

}

}

Listing 2.7: Implementation of sendOpenFlowMessageToCore().

public class NetIDEBackendController implements ICoreListener {

public void sendOpenFlowMessageToCore(OFMessage msg , int xId , long

datapathId , int moduleId) {

ChannelBuffer buf = ChannelBuffers.dynamicBuffer ();

msg.writeTo(buf);

NetIDEPublic

10



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



byte[] payload = buf.array ();

OpenFlowMessage message = new OpenFlowMessage ();

message.getHeader ().setPayloadLength (( short)payload.length);

message.setOfMessage(msg);

message.getHeader ().setDatapathId(datapathId);

message.getHeader ().setModuleId(moduleId);

message.getHeader ().setTransactionId(xId);

coreConnector.SendData(message.toByteRepresentation ());

}

}

Registering ONOS applications as NetIDE modules. ONOS provides the ApplicationServiceAPI that can be used to retrieve information about the running applications. The ONOS Back-end requests the registration of all the active ONOS applications to the NetIDE Core via theMODULE_ANNOUNCEMENT message. Listing 2.8 shows how ONOS application identi�ers are receivedfrom ApplicationService API and then registered to the NetIDE Core via obtainModuleId()

method, which implements the creation of MODULE_ANNOUNCEMENT message and the transmission tothe NetIDE Core.

Listing 2.8: Registration of ONOS applications on NetIDE Core.

import org.onosproject.app.ApplicationService;

...


protected ApplicationService applicationService;

...

Set <Application > runningApplications = applicationService.getApplications ();

for (Application application : runningApplications) {

if (applicationService.getState(application.id()) == ApplicationState.ACTIVE)

{

moduleHandler.obtainModuleId(getXId (), application.id().name(),

moduleHandler.getModuleId(MODULE_NAME));

}

}

...

2.5.2 ONOS Shim

The ONOS Shim manages the NetIDE Intermediate protocol communication with the NetIDE Coreand relays the network messages between the network and the NetIDE Core. For the architecturaldescription of ONOS Shim, please refer to Deliverable D2.7 of the NetIDE project [2].

ONOS OpenFlow protocol. From version 1.7.0, ONOS allows SDN applications to receive allthe OpenFlow message coming from the network by means of the OpenFlowController Java API.This can be achieved by implementing the OpenFlowMessageListener interface and registering thisimplementation in the OpenFlowController API, as shown in the code below.

Listing 2.9: Registration of NetIDEDeviceListener to the OpenFlowController.

import org.onosproject.openflow.controller.OpenFlowController;

...


protected OpenFlowController controller;

...

@Activate


NetIDEPublic

11



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



...

// Register a new OpenFlow message listener in the OpenFlowController

NetIDEDeviceListener ofDeviceListener = new NetIDEDeviceListener ();

controller.addMessageListener(ofDeviceListener);

...

}

Relaying OpenFlow messages. The OpenFlow messages are received from the OpenFlowControllerAPI through the implementation of the method handleIncomingMessage(Dpid dpid, OFMessage

msg), encapsulated in the NetIDE header and then sent to the NetIDE Core, as shown in Listing 2.7.

Listing 2.10: Relaying OpenFlow messages.

public class NetIDEDeviceListener implements OpenFlowMessageListener {

@Override

public void handleIncomingMessage(Dpid dpid , OFMessage msg) {

switch (msg.getType ()) {

case PACKET_IN:

shimController.sendOpenFlowMessageToCore(msg , ShimLayer.getXId (),

dpid.value (), 0);

break;

...

}

}

}

Handling Backend connections. When a new Backend connects to the NetIDE Core, the Shimsends the list of all the switches connected by means of FEATURE_REPLY messages. To this end,the ONOS Shim gets the list of all the connected switches by using the OpenFlowController API,generates the associated FEATURE_REPLY message, encapsulates this message in the NetIDE headerand sends it to the NetIDE Core.

Listing 2.11: Handling asynchronous Backend connections.

@Override

public void onHelloCoreMessage(List <Pair <Protocol , ProtocolVersions >>

requestedProtocols , int moduleId) {

...

for (OpenFlowSwitch sw : controller.getSwitches ()) {

OFFeaturesReply featuresReply = getFeatureReply(sw);

sendOpenFlowMessageToCore(featuresReply ,featuresReply.getXid (),sw.getId (),moduleId);

...

}

}

Blocking the PACKET_IN delivery to other applications running on ONOS. The ONOS Shim im-plements the NetIDE architecture v2.0, as described in Deliverable D2.6 [1] and D2.7 [2]. Therefore,it intercepts the network events before they are delivered to the applications running on ONOS.In general, to block the PACKET_IN delivery chain performed by the OpenFlowController API, theONOS Shim can call the method block() of the OpenFlowPacketContext, an ONOS representa-tion of the OpenFlow PACKET_IN. When an OpenFlowPacketContext is block(), all the ONOSapplications will not process the blocked packet.

Listing 2.12: Blocking the PACKET_IN delivery to other applications running on ONOS.

NetIDEPublic

12



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



public class ShimLayer {


...



...

@Activate


...

// Register a new PACKET_IN listener in the OpenFlowController with the lowest

priority

controller.addPacketListener (0, ofDeviceListener);

...

}

}

public class NetIDEDeviceListener implements PacketListener {

...

@Override

public void handlePacket(OpenFlowPacketContext openFlowPacketContext) {

//Block the packet delivery after this module

openFlowPacketContext.block ();

}

...

}

}

Control protocol. During the initial handshake with the Backends, the Shim sends the version ofthe control protocol which has been previously negotiated by the server controller with the switches,as explained in Annex A of Deliverable D2.7. This operation avoids incompatibilities between thenetworks applications executed by the client controllers and the server controller's SBI used to drivethe hardware.The OpenFlowController API provides the possibility to get all the switches connected to theONOS OpenFlow southbound. The OpenFlowController API saves all the OpenFlow switchesin an internal data structure called OpenFlowSwitch. As shown in Listing 2.13, by using themethod OpenFlowSwitch.factory().getVersion(), the ONOS Shim can get the OpenFlow versionsupported by the switches connected and save it.

Listing 2.13: Deciding the southbound protocol version of the NetIDE protocol.

public class ShimLayer {


...



...

@Activate


...

for (OpenFlowSwitch sw : controller.getSwitches ()) {

Dpid dpid = new Dpid(sw.getId());

OFVersion version = sw.factory ().getVersion ();

Integer ofVersion = version.getWireVersion ();

shimLayer.setSupportedProtocol(ofVersion.byteValue ());

}

}

NetIDEPublic

13



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



2.6 The Floodlight Backend

Developing a module for Floodlight requires you to download the Floodlight source code in advanceand add your module as a new namespace. The advantage of developing the Floodlight Backend(FL-BE) as a module (as opposed to a RESTful SDN application) is having access to the native JavaAPI which o�ers additional functionality not available from the RESTful interface - in particular,the ability to intercept messages going to connected switches from other modules and the ability toreply. The FL-BE code was added as the namespace

net.floodlightcontroller.interceptor

and is loaded when the SDN controller starts up. The architecture of Floodlight requires that allmodules register with the controller on startup and specify where in the order of the other modulesthey start. The FL-BE starts up last as we didn't want to impact on any dependencies betweenexisting modules.

Listing 2.14: FloodlightDefault.properties

floodlight.modules = net.floodlightcontroller.storage.memory.MemoryStorageSource ,\

net.floodlightcontroller.core.internal.FloodlightProvider ,\

net.floodlightcontroller.threadpool.ThreadPool ,\

net.floodlightcontroller.staticflowentry.StaticFlowEntryPusher ,\

net.floodlightcontroller.jython.JythonDebugInterface ,\

net.floodlightcontroller.perfmon.PktInProcessingTime ,\

net.floodlightcontroller.ui.web.StaticWebRoutable ,\

net.floodlightcontroller.learningswitch.LearningSwitch ,\

net.floodlightcontroller.interceptor.NetIdeModule

By changing the port on which Floodlight listens to switch connections allows the FL-BE toconnect a series of dummy switches that send the relevant Open�ow messages to pass the switchconnection handshake with Floodlight. There is a one-to-one mapping between each physical/virtualswitch that the server SDN controller is managing and a dummy switch that connects to Floodlight.By allowing Floodlight to believe it is physically connected to the switches allows for the normaloperations of the Floodlight controller to continue.

net.floodlightcontroller.core.internal.FloodlightProvider.openflowport = 7733

As the FL-BE completes the handshake on behalf of the dummy switch, the Floodlight Core willtrigger the addedSwitch event to other modules that have subscribed to the IOFSwitchListener

interface. This allows SDN applications to push new �ows or react to packet_in events for thisnewly added switch.

net.floodlightcontroller.interceptor.NetIdeModule.coreIp = 127.0.0.1

net.floodlightcontroller.interceptor.NetIdeModule.corePort = 5555

Each dummy switch implemented by the FL-BE intercepts Open�ow messages sent by the con-troller. These are then sent to the NetIDE Core, encapsulated in a NetIDE Intermediate Protocolmessage. These in turn are sent to the Shim which will route them to the correct network element(switch). Similarly, all messages from the switches (e.g., packet_in) are sent by the Shim (throughthe Core) to the FL-BE in an Intermediate Protocol message. The Open�ow message is extractedand sent through the correct Dummy Switch. The connection information for the NetIDE Core isstored in the Floodlight Properties �le.

NetIDEPublic

14



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3 The NetIDE Tools Final Release

3.1 Evolution of the NetIDE Tools Architecture

The NetIDE Tools Architecture evolved from the need to provide NetIDE users with a rich toolecosystem to integrate, debug and run SDN applications. It was one of the original requirementsderived from the Use Cases [21]. In the beginning, only a couple of tools were envisaged, thisis why the �rst idea was to use an API interceptor to get relevant data. This idea was re�nedlater [22] with a central message broker based on RabbitMQ using Advanced Messaging QueueingProtocol (AMQP). This was deemed a heavy and complex solution to answer a small problem.Additionally, the �rst tool developed for NetIDE (cf. the Logger in Section 3.4) was split in two:one part containing all the logic and the other one forwarding the messages directly integrated inthe Shim [23]. This posed even more problems as each tool would have needed its �forwarder� foreach one of the Shims. The update cycles of the di�erent Shims (which depends on the underlyingServer Controller) would mean as much updates for the �forwarders�.The solution to all of these issues was the new tool architecture: one unique forwarding element

within the Core to get all messages, the LogPub, that connects the Tools to the Core and exchangeddata, ∅MQ. Using this new architecture, the Tools use the API provided by the LogPub and canbe updated separately; the LogPub abstract the Tools to the rest of the Core and act as a uniquepoint of entry; the Core/LogPub use a common update cycle independently from from the Shimsand Backends.

3.2 Analysis of SotA on SDN Tools

Any piece of software that facilitates the development, deployment and/or maintenance of SDNarchitecture and, more speci�cally, of network applications can be classi�ed under the SDN toolcategory. They complete the puzzle of an ideal SDN management framework. When a tool isused at development time, we consider it o�ine, whereas when applied at deployment or runtime,we call it online. O�ine tools are mainly veri�ers or model checkers, but this category may alsoinclude simulators; while online tools encompass: loggers, debuggers, pro�lers, memory managersand emulators. Additionally, there are also parts of the SDN architecture that might be consideredas tools, such as the communication protocols and channels, or the intrinsic mechanisms of the SDNcontrollers that allow di�erent SDN applications (or even frameworks) to work together withoutcon�icting with each other. More speci�cally, NetIDE envision the following classes of SDN tools:

• Composition: Composition of SDN applications let the network support upgrades and ex-pansions, including newer functionalities or coordinating several software modules.

• Debugging: Troubleshooting, veri�cation and model checking aim for the same objective:the network as a whole behaves as expected. Their di�erences are mainly related to the partof the network's behavior that need to be analyzed (data, control or application plane) andat which time (development, deployment or runtime).

• Resource management: Resource management guarantees an optimal utilization of net-work resources.

NetIDEPublic

15



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



• Pro�ling: To prove that resource management is e�ective, we need pro�lers or monitors tomeasure the network activity and resource consumption.

• Simulation: To repeat di�erent scenarios without a�ecting the production network, simula-tors and emulators represent an essential tool.

• Graphical user interfaces: Finally, tools can be complemented with a Graphical UserInterface (GUI) that facilitates the con�guration and monitoring of the network and of thetools themselves.

Currently, the Open Networking Foundation (ONF) provided the reference model for most con-troller frameworks. The architecture does not describe the physical components, instead it de�nesthree planes: Data, Controller and Application, plus the Management functions aside. This lastManagement plane has a functional interface with all the three aforementioned planes. Its mini-mum functionality is to allocate resources from a resource pool in the Data plane to a particularclient entity in the Controller plane, and to establish reachability information that permits theData and Controller plane entities to mutually communicate. Management functions are providedby the Operations Support Systems (OSS); which includes infrastructure maintenance (fault analy-sis, diagnostics, alarm correlation and management), logging, con�guration and service persistence,service assurance, tra�c analysis, network and equipment planning, installation, inventory, softwaredistribution and upgrade, policies, and initialization parameters. Some of these functions are in thescope of SDN and are implemented by means of what we call as SDN tools.The Management plane interacts with other planes through the use of interfaces. In this sense,

many speci�cations have been proposed for the Data plane(NETCONF, SNMP or OF-Con�g).However, in case of the Controller plane, standard interfaces have not yet been de�ned to copewith the requirements of maintenance and management operations. As a consequence, the currentapproaches for many of the available SDN tools are either enhancing the controller platform withadditional modules able to expose the required interfaces, or implementing their functions directlyin the core layers of such platforms. NetIDE project envision a clear interface between the threedi�erent planes provided by the LogPub API integrated in the Core. When we convert networkbehavior into a software, the �rst concept that comes into our minds as a tool is a debugger. Wealready foresee two types of SDN debuggers: debuggers for the software or the SDN applications(control plane) and debuggers for the network device or SDN switch (data plane). Another clas-si�cation considers whether a debugger requires the physical network to be deployed or running(online debuggers), or not (o�ine debuggers). We have analyzed the di�erent state-of-the-art SDNdebuggers based on this last parameter, i.e. based on their o�ine and online nature, respectively. InTable 3.1 we summarize and compare the di�erent debugging tools included in the state-of-the-art.The parameters that have taken into account are the following:

• Execution time: Two types of tools: o�ine, which ful�ll their purpose when the networkis not being executed, and online, which accomplish their function while the SDN network isrunning (either at deployment or at runtime).

• Modeling: Whether the tool creates a model for later veri�cation or not.

• Packet history: Tools that keep track of the information from the packets that traversethe network. In some cases, by tracking the tra�c crossing a certain point, in other cases byrecording information about nodes traversed by a packet and the changes on its header �elds.

• Objective plane: Whether analysis and veri�cation focus on the data or control plane.

• Packet injection: Tools that meet this parameter inject a certain number of speci�c packetsin the analyzed network to troubleshoot misbehaviors in the network.

NetIDEPublic

16



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Table 3.1: Comparison table of the di�erent debugging tools

Tools

Properties Execution

timeModeling

Packet

historyPlane

Packet

injectionMain concept

NICE O�ine Yes No Control Plane NoFinding inconsistenciesin SDN applications

VeriCon O�ine Yes No Control Plane NoFinding inconsistenciesin SDN applications

Veri�care O�ine Yes No Control Plane NoFormal veri�cationof SDN applications

Assertion language O�ine Yes No Control Plane NoFormal veri�cationof SDN applications

Abstractions

for modelO�ine Yes No Control Plane No

Proving correctnessof SDN controllers

SOFT O�ine No No Data Plane NoFinding inconsistencies

in OpenFlow (OF) agents

OFLOPS O�ine No No Data Plane NoAnalyzing capabilities

of OF switches

OFTest O�ine No No Data Plane YesCompliance checkof OF switches

OFCBenchmark O�ine No No Control Plane NoAnalyzing capabilitiesof SDN controllers

TASTE O�ine No No Control Plane YesCompliance checkof SDN controllers

OFf O�ine No Yes Both YesDebugging and testing

of SDN controllers and switches(trace replay and breakpoints)

STS O�ine No Yes Both YesDebugging and testing

of SDN controllers and switches(provides right subset to debug)

Pathlet Tracer O�ine No Yes Data Plane YesDebugging and testing

of SDN paths (path tracing)

FlowChecker Online Yes No Data Plane NoDetects miscon�gurations

in OF switches

VeriSDN Online Yes No Data Plane No Formal veri�cation of SDNs

ATPG Online Yes No Data Plane YesConsistency of data plane with

control plane (via packet injection)

Monocle Online Yes No Data Plane YesConsistency of data plane with


VeriDP Online Yes No Data Plane YesConsistency of data plane with


Kinetic Online Yes No Control Plane NoFormal veri�cationof SDN applications

FLOWGUARD Online Yes No Both NoDetects and solves

�rewall policy violations

NetPlumber Online Yes No Both NoDetects and solves invariant

violations (loops, blackholes, etc.)

HSA Online Yes No Both NoDetects and solves invariant


VeriFlow Online Yes No Both NoDetects and solves invariant


FlowTest Online No No Both YesChecks correct implementation

of functions and policies

RuleScope Online No No Data Plane YesInspects SDN forwarding(via packet injection)

SDN Traceroute Online No Yes Data Plane YesInspects SDN forwarding(via packet injection)

SDN-RADAR Online No Yes Data Plane YesInspects SDN performance

(via packet injection)

OFRewind Online No Yes Both YesDebugging and testing of SDNs

(replay of data and control tra�c)

NetSight Online No Yes Both NoDebugging and testing of SDNs

(packet histories)

NetIDEPublic

17



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



We have analyzed what is an SDN tool, why is necessary, how it �ts the SDN architecture,and provided a series of tools classi�ed per type. The �rst conclusion is that we miss a standardarchitecture and interfaces generic enough to be used for the implementation of SDN-speci�c tools.Several SDN tools are currently available, but they are either planned for very speci�c architecturemodels or de�ned for a particular SDN controller framework. Therefore, most of them can be hardlyadopted in a production environment where the maintenance of the network is a critical task, andwhere patching the components of the SDN network to merge the di�erent approaches is usuallynot a viable option.Considering these aspects, we envision the following challenges and future research directions for

the standardization of the Management Plane:

• Create a generalized list of SDN tools: Analyzing and classifying the potential SDNtools should be a �rst step to de�ne the di�erent requirements for management.

• De�ne the management interfaces: Considering the heterogeneity of SDN tools, a secondstep would be de�ning a set of management interfaces that accomplish the minimum commu-nication requirements of these tools. For example, composing applications for SDN platformswould require the standardization of an East/Westbound Interface (EWBI) (which commu-nicates modules at the same layer), while a network pro�ler would require some interface toreach the data plane and manage statistics.

• De�ne messages and data structures: Apart from the interfaces, an SDN tool mightrequire speci�c messages and data structures to work. For instance, a network pro�ler coulduse the SBI to reach the data plane, but would need speci�c messages apart from the onesde�ned in the most popular SBI protocol: OF.

3.3 Integration of external tools to NetIDE architecture

One of the main goals of NetIDE Project is to provide a set of tools for debugging and trou-bleshooting SDN networks. For such purpose, we identify the most �agrant weaknesses of the SDNenvironment:

• Most of the tools depend on the controller platform, so a tool developed for one controllercannot be applied in another.

• There are few tools able to operate at runtime. This provokes that tools do not supportunexpected network events or modi�cations (for instance, some Modelcheckers).

• Tools only address one plane, control plane or data plane, but never both at the same time.

So, after the detection of these gaps in the SDN world, NetIDE designed and implemented aset of tools able to solve such weaknesses. The solution was based on a key architectural elementcalled LogPub able to decouple the functionality of tools from the Network SDN technology. Thiselement acts as an interface between the NetIDE Engine and the tools, forwarding all the messagesexchanged among the Engine modules (Client Controllers, Core and Server Controller) to the tools.At the same time, the tools can send messages to the Client Controllers or to the data plane bymeans of the LogPub. Thereby, thanks to this element, tools are completely independent fromSDN controller platforms and their tasks can be accomplished at runtime (while the SDN networkis operating). However, the fact of having the LogPub does not imply to use it mandatory. Adeveloper can decide whether to take advantage of it or not. For instance, the Application Pro�leris a tool for python-based Controllers which does not use the LogPub. In a nutshell, this interface

NetIDEPublic

18



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



facilitates tools development and does not impose restrictions. Furthermore, the integration ofother tools to NetIDE can be done easily depending on the nature of such tool. Tools that carryout their functionality at runtime and need to read messages exchanged between the datapathelements and the controller or need to inject messages into the data plane are the most suitable forbeing integrated into NetIDE architecture. The process is simple: To read messages, the tool mustconnect to a port where messages are enqueued using ZeroMQ (ZMQ) technology, then the tool willretrieve the messages after removing the NetIDE header using the NetIP library (available in javaand python). To send messages, the tool must follow the inverse process. Firstly, it should createthe message to be sent. After this step, the tool must encapsulate the message (adding NetIDEheader) using, again, the NetIP library. Finally, the tool will send that message to the LogPub bymeans of ZMQ. There are tools which perform their activity over a speci�c controller platform.For instance, Modelcheckers. These are validation tools, which operate before deploying the SDNnetwork, in order to detect network misbehaviors (for example: loops or black holes). Modelcheckerscreate state diagrams based on the SDN applications, the controller platform and the topology. So,this kind of tool can be applied directly in our architecture without modi�cations (we will provide aclient controller, the applications to be executed on top of that controller and a topology). However,the outcomes would be invalid, because the Modelchecker would not have considered the Core, therest of executed client controllers (running di�erent applications) and other tools connected to thearchitecture. Therefore, for integrating properly these speci�c-controller tools, the developer wouldhave to extend them for supporting the rest of architectural elements. Finally, there exist toolsdesigned for a concrete architecture. These need external elements (hardware or software) to carryout their purpose. In this case, the integration will depend on the complexity of adapting the newelements to the NetIDE architecture.

3.4 Logger

The Logger is the simplest tool o�ered by NetIDE project for checking and debugging the behaviourof an SDN network. Its task is to display all the messages which reach the Core. Therefore, theLogger is able to show both messages generated by the data plane, such as those propagated by theNetIDE Engine. Thus, the user is able to visualize what is happening in the network at runtime.It also saves all logs in a plain text �le, results.txt, for o�-line checking. Despite of its simpleoperation, this tool has great relevance since it has served as a conceptual base for the developmentof more complex tools (for instance: the debugger, explained in Section 3.5).The Logger is connected to the NetIDE Engine by means of the LogPub, which is an internal

module of the Core in charge of acting as an interface between the tools and the Engine. TheLogPub receives the messages from the tools and the Engine and pushes them into di�erent queues.In order to accomplish its purpose, the Logger must retrieve the enqueued messages, this task is

performed using ZMQ technology. After, the tool decodes such messages and converts them intohuman readable format. Finally, the Logger displays the messages into a terminal.In previous releases of the tool, there was a color distinction between the messages originated by

the data plane (represented in green) and the messages which came from the Client Controllers (inblue). However, there are messages which are generated by the tools or by other Engine modules(such as the shim) and do not belong to the two groups previously de�ned. Therefore, as it isshown in Figure 3.1, the last version of the logger displays the messages in the same color withoutdistinctions, but it provides more information about the module that generates the message and themodule which receives such message. In this example, we can see how a packet_in originated bya switch, arrives to the shim which forwards the message to the Core. Finally, the Core sends thepacket_in to the Client Controller. In order to ease the message visualization, the message breaksdown in three sections. The �rst one contains the �elds of the NetIDE header, the second section

NetIDEPublic

19



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Figure 3.1: Logger trace.

shows the OpenFlow header, �nally the last part displays the payload of the OpenFlow message.

3.5 Debugger

The main objective of the Debugger is to let an administrator snoop the network tra�c and thetra�c exchanged along the Network Engine. This allows debugging the network, as well as theNetwork Engine behavior. Particularly relevant when a network administrator is composing di�erentapplications from several SDN platforms and wants to assure the Network Engine was programmedaccordingly.For this purpose, the Debugger is comprised of two pieces:

1. A piece of software that produces *.pcap �les

2. A Wireshark [24] dissector to analyse and present those *.pcap �les in a readable format

The network administrator can produce *.pcap �les either using the debugger.py script thatcollects messages from the LogPub module installed in the Core of the NetIDE Network Engine,or can directly generate it by capturing tra�c with Wireshark. The former is o�cially calledthe Debugger tool and it contains a Wireshark dissector that only analyses the Network Enginetra�c, while the latter captures all tra�c and we provide a second NetIDE Wireshark dissectorto distinguish the captured frames. Figure 3.2 shows both pieces of code at the Tools GitHubrepository. For more information on how to install and run each of them, please check the NetIDEManual [2].In the next subsections, we analyse both approaches and explain their code. Although they both

require a Wireshark dissector to analyse the *.pcap �les produced, the design of these dissectordi�er in some parts, because one approach examines all tra�c, while the other �lters it based onthe output produced by the LogPub.

3.5.1 The Debugger tool

Back in D4.2 [22], the approach of the Debugger was to obtain the logs from a LogPub moduleinstalled in the Shim. This caused developing a new LogPub per Shim, which was not scalable.After the evolution of the Network Engine and the inclusion of the Core module (agnostic to theSDN platforms), the decision was to move the LogPub to the Core as a single and independentmodule.The Debugger is composed of the following two components:

1. The debugger.py, executed at runtime and in charge of producing the *.pcap �les

NetIDEPublic

20



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Figure 3.2: Debugger and NetIDE Wireshark dissector at the Tools GitHub repository

2. The corresponding Wireshark dissector to analyse and present those *.pcap �les in a readableformat, which can be installed via the netide.la and netide.so �les. To compile the dis-sector, packet-netide.c contains the NetIDE protocol de�nition and packet-user_encap.c

provides a special link-layer type to present the NetIDE packets.

3.5.1.1 The debugger.py script

The messages published by the LogPub �later collected by the Debugger� do not contain the fullTCP/IP stack, but only the Application layer, i.e. the NetIDE protocol. To avoid producing *.pcap�les with empty �elds for the lower layers (L2, L3 and L4), we decided to use the customizable link-layer header type DLT_USER0 [25], which is one of the types reserved for private use in libpcap [26].The basic logic of the *.pcap �le creation is shown in Listing 3.1. The debugger.py script initiates

a ZMQ connection to the LogPub and waits for logs. For every packet received, a correspondingmessage is added to the *.pcap �le after calling the generatePCAP() functions, which basicallyworks with the bytes received, adding the necessary packet headers. The generated �le is saved asresults.pcap via the bitout stream.

1 bitout = open(args.o+"/results.pcap", 'wb')

2 [...]

3 # Socket to talk to server

4 context = zmq.Context ()

5 socket = context.socket(zmq.SUB)

6 socket.connect("tcp :// localhost :5557")

7 socket.setsockopt(zmq.SUBSCRIBE , "")

8 i = 0

910 print('[*] Waiting for logs. To exit press CTRL+C')

11 while True:

12 dst_id , device_id , msg = socket.recv_multipart ()

13 [...]

14 msg_hexadecimal = binascii.hexlify(msg)

NetIDEPublic

21



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



1516 bytestring = generatePCAP(msg_hexadecimal ,i)

17 i = sum_one(i)

18 bytelist = bytestring.split()

19 bytes = binascii.a2b_hex(''.join(bytelist))

20 bitout.write(bytes)

Listing 3.1: Creation of the *.pcap �le for the Debugger

3.5.1.2 The packet-user_encap.c de�nition for the Wireshark dissector

With the debugger.py script, we will already have a readable *.pcap �le that uses the link-layerheader type DLT_USER0. However, if we open this �le with Wireshark DLT_USER0 will not beassociated to NetIDE yet and the use will have to indicate it manually in the Preferences section.In order to associate the customizable link-layer header type DLT_USER0 with NetIDE in an

automatic way, we de�ned the packet-user_encap.c �le based on the Wireshark documenta-tion [27, 28]. Listing 3.2 shows how we associate the DLT_USER0 type with the name NetIDEand the NetIDE dissector.

1 void proto_reg_handoff_user_encap(void)

2 {

3 [...]

4 user_encap_handle = find_dissector("user_dlt");

5 data_handle = find_dissector("data");

67 user0_encap.encap = WTAP_ENCAP_USER0;

8 user0_encap.payload_proto_name = g_strdup("netide");

9 user0_encap.payload_proto = find_dissector("netide");

10 [...]

11 }

Listing 3.2: Registration of the DLT_USER0 type to associate it with the NetIDE protocol

The packet-user_encap.c together with a small change in the NetIDE Wireshark dissector,described in the next section, produce the netide.la and netide.so �les that will let the userdissect the NetIDE tra�c, as described in the NetIDE Manual [2].

3.5.2 The NetIDE Wireshark dissector

In this case, we directly create the *.pcap �les with Wireshark. Later on, to install the generalNetIDE Wireshark dissector (for network captures not based on LogPub logs), there are also two�les: netide.la and netide.so (di�erent from the Debugger dissector, although they share thename).The NetIDE Wireshark dissector was implemented as described in their documentation [29]. Each

dissector decodes its part of the protocol, and then hands o� decoding to subsequent dissectors foran encapsulated protocol. Every dissection starts with the frame dissector which dissects the packetdetails of the capture �le itself (e.g. timestamps). From there it passes the data on to the lowest-level data dissector, e.g. the Ethernet dissector for the Ethernet header. The payload is then passedon to the next dissector (e.g. IP) and so on. At each stage, details of the packet will be decodedand displayed.The NetIDE dissector has been implemented as a plugin (a shared library or DLL) that registers

itself to handle dissection. To compile the dissector, packet-netide.c contains the NetIDE protocolde�nition.

NetIDEPublic

22



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3.5.2.1 The packet-netide.c de�nition for the Wireshark dissector

The function proto_register_netide() registers the protocol, as shown in Listing 3.3. We canprovide three names that will be used for display in various places. The full and short name areused for example in the Preferences and Enabled protocols dialogs, as well as the generated �eldname list in the documentation. The abbreviation is used as the display �lter name.

1 static const value_string packettypenames [] = {

2 { 1, "NETIDE_HELLO" },

3 { 2, "NETIDE_ERROR" },

4 { 3, "NETIDE_MGMT" },

5 { 4, "MODULE_ANNOUNCEMENT" },

6 { 5, "MODULE_ACKNOWLEDGE" },

7 { 6, "NETIDE_HEARTBEAT" },

8 { 7, "NETIDE_TOPOLOGY" },

9 { 8, "NETIDE_FENCE" },

10 { 17, "NETIDE_OPENFLOW" },

11 { 18, "NETIDE_NETCONF" },

12 { 19, "NETIDE_OPFLEX" }

13 };

1415 static const value_string netidever [] = {

16 { 3, "1.2"},

17 { 4, "1.3"},

18 { 5, "1.4"}

19 };

2021 [...]

2223 void proto_register_netide(void)

24 {

25 static hf_register_info hf[] = {

26 { &hf_netide_ver ,

27 { "NETIDE Version", "netide.ver",

28 FT_UINT8 , BASE_HEX ,

29 VALS(netidever), 0x0,

30 "NETIDE Version", HFILL }

31 },

32 { &hf_netide_type ,

33 { "Type", "netide.type",

34 FT_UINT8 , BASE_DEC ,

35 VALS(packettypenames), 0x0,

36 "Package Type", HFILL }

37 },

38 { &hf_netide_length ,

39 { "Package Length", "netide.length",


41 NULL , 0x0,

42 "Package Length", HFILL }

43 },

44 { &hf_netide_xid ,

45 { "xid", "netide.xid",


47 NULL , 0x0,

48 "Xid", HFILL }

49 },

50 { &hf_netide_module_id ,

51 { "module_id", "netide.module_id",


53 NULL , 0x0,

54 "module_id", HFILL }

NetIDEPublic

23



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



55 },

56 { &hf_netide_datapath_id ,

57 { "datapath_id", "netide.datapath_id",


59 NULL , 0x0,

60 "datapath_id", HFILL }

61 },

62 { &hf_netide_format ,

63 { "Format", "netide.format",


65 NULL , 0x0,

66 "OpFlex/Netconf -Format", HFILL }

67 }

68 };

6970 /* Setup protocol subtree array */

71 static gint *ett[] = {

72 &ett_netide

73 };

7475 proto_netide = proto_register_protocol (

76 "NETIDE Protocol", /* name */

77 "NETIDE", /* short name */

78 "netide" /* abbrev */

79 );

8081 proto_register_field_array(proto_netide , hf, array_length(hf));

82 proto_register_subtree_array(ett , array_length(ett));

83 register_dissector("netide", dissect_netide , proto_netide);

84 }

Listing 3.3: Registration of the NetIDE protocol as a packet type for Wireshark

The initialization of the dissector is done in the proto_reg_handoff_netide() method, as shownin Listing 3.4. At the beginning of the code we created a dissector handler (netide_handle), whichis associated with the NetIDE protocol and with a routine to be called to do the actual dissecting(dissect_netide). Then we associate the handler with a port number (NETIDE_PORT, which isde�ned as 5555 at the beginning of the code) so that the main program will know when to call thedissector.

1 void proto_reg_handoff_netide(void)

2 {

3 static gboolean initialized=FALSE;

4 if (! initialized) {

5 netide_handle = create_dissector_handle(dissect_netide , proto_netide);

6 dissector_add_uint("zmtp.protocol", NETIDE_PORT , netide_handle);

7 data_handle = find_dissector("data");

8 openflow_v1_handle = find_dissector("openflow_v1");



11 }

12 }

Listing 3.4: Initialization of the NetIDE dissector

Finally, the dissector function is de�ned in packet-netide.c as:

static void dissect_netide(tvbuff_t *tvb, packet_info *pinfo, proto_tree *tree)

NetIDEPublic

24



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



This function would be called by the DLT_USER0 type in the case of the Debugger. If weuse the NetIDE Wireshark dissector with a *.pcap �le created with Wireshark, then we will haveinformation for layers 2 (Ethernet), 3 (IP), 4 (TCP), then ZMQ and �nally the NetIDE protocol.

3.6 Veri�cator

One of the required tools for the NetIDE project was a veri�cation tool. This kind of tool is usedfor validating correctness properties in an SDN network. For instance, the absence of loops orblack holes. In this section will be introduced and explained the Veri�cator, the �rst prototype fordetecting loops at runtime under certain conditions and providing general information about theexchanged messages within the NetIDE Engine.The Veri�cator's main objective is to notify the existence or absence of loops in a given topology.

However, this tool provides also information regarding the messages exchanged among the Engine'scomponents. This data helps to debug and detect anomalies in the deployed SDN network.The Veri�cator is connected to the Engine by means of the LogPub. Thanks to this interface,

it is able to retrieve all the messages that pass through the Core. Thereby, the tool analyses eachof the messages, distinguishing between the messages sent by the di�erent Engine modules (ClientControllers or Server Controller). Finally, the Veri�cator extracts relevant information for each ofthe NetIDE Engine modules: amount of sent messages and average, maximum and minimum sizeof messages.

Figure 3.3: Veri�cator outcomes.

Figure 3.3 shows the Veri�cator's menu (in its latest version). Option 1 o�ers the data describedabove, while if the user chooses option 2, this tool will display if there are loops or not (Figure 3.4).

Figure 3.4: Veri�cator outcomes, loop detection.

In order to detect loops, the Veri�cator checks that the Medium Access Control (MAC) sourceand MAC destination are the same in each message. If there is a large amount of messages (100messages) with the same source MAC and destination MAC in a short time lapse (1 second), thenthe tool noti�es the existence of a loop in the underlying topology. To understand better the logicimplemented in the Veri�cator, a real case is shown in Figure 3.5:Host H1 sends an ARP request with source MAC X and destination MAC ff:ff:ff:ff:ff:ff.

This message reaches the Sw1 and the switch forwards it to the Engine, then the simple switchapplication receives the Packet_In event and orders the switch to �ood the message. Therefore,

NetIDEPublic

25



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



H1 H2

Sw1 Sw2

Sw3

Figure 3.5: Real case for explaining the Veri�cator's operation.

the Sw1 sends the packet to Sw2 and Sw3. This process is repeated in Sw2 and Sw3, so the threeswitches send a message with the same destination and source MAC in�nitely, making the networkcollapse.The requirements for loop detection using the Veri�cator tool are the followings:

• The application that handles the network tra�c must be reactive and not proactive. Due tothe messages must reach to the Core and consequently the Veri�cator.

• The applications must not contain any logic responsible for solving topology loops, for instance:a simple learning switch application.

It is worthy to mention that the Veri�cator o�ers two operation modes: 1) The runtime mode,which operates while the network is being executed and is able to adapt to network changes andguarantees the veri�cation of properties when more than one application is running (unlike o�-linemodel checkers). 2) The o�-line mode which checks if there was a loop in the last execution ofthe Engine. So, if a user forgets to activate the Veri�cator (the runtime mode) and the networkcollapses, it is possible to know whether the problem was caused by a loop in the topology or not.This mode reads a �le text which was written by the Debugger tool during the execution of theSDN network.

3.6.1 Future lines of work

The current status of the Verifcator allows to detect inconsistencies in the topology construction,i.e: loops composed by switches. However, it is not possible to detect loops in any topology underany circumstance as it is explained above.Thus, the next step is to develop a tool more complex, able to determine if there is a loop in any

scenario. For this purpose, the Veri�cator would ask the Core for the topology and, along with therules installed by the SDN applications, the tool will create a state diagram. Finally, the tool, usingthe aforementioned information (rules plus topology), will able to identify loops. For instance, if aswitch 'B' receives tra�c from a switch 'A' and there is a rule installed in 'B' that orders to forwardsuch tra�c to 'A', we can determine that a loop exists in the network.The �nal release of the Veri�cator would be very powerful since the user could detect loops at

runtime in any scenario. Furthermore the tool would be able to adapt to topology and applicationschanges.

NetIDEPublic

26



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3.7 Pro�ler

The term Pro�ling is framed within the Software Engineering area. It consists on carrying outa performance analysis in a given program with the aim of detecting failure points or pieces ofcode which should be optimized. In NetIDE , this term is ported and adapted to SDN networks.Thereby, the idea of a pro�ler tool arises for detecting SDN applications that should be optimizedand analyzing the network behavior.The Pro�ler breaks down in two modules: The Application Pro�ler and the Network Pro�ler. The

�rst module applies the term pro�ling as it is de�ned in Software Engineering. Therefore, the goal ofthe Application Pro�ler is to analyze and obtain the execution time of the functions executed in theClient Controllers and in their SDN applications in order to �nd critical points which can damagethe network performance. The current version of Application Pro�ler is for analyzing python-basedcontrollers. To carry out its purpose, the tool is directly connected to Client Controllers, once thecontroller has been executed the tool begins to take measurements by means of cPro�ler (a pythonmodule for extracting the execution time of a program) and write them in a binary �le. when theClient Controller �nish its execution, the binary �le is dumped into a text �le which will be readand processed by a statistical software. Finally, the outcomes obtained will be displayed in twographs. Below, in Figures Figure 3.6 and Figure 3.7 there are two graphics obtained after pro�linga Ryu Client Controller running a simple L2 forwarding application.

0

10

20

30

40

0 100 200 300Execution Time (seconds)

Num

ber

of fu

nctio

ns

0

10

20

30

40

Number of functions

Figure 3.6: Execution times histogram.

Figure 3.6 represents an histogram, where is shown the amount of functions that share the sameexecution time. In this case, there are more than forty functions with a execution time closed tozero seconds and about ten functions which take longer than three hundred seconds.Figure 3.7 shows the functions with the highest execution times. In X axis we �nd the name of

each function, while in Y axis is represented its correspondent value.On the other hand, the Network Pro�ler adapts the term pro�ling to the �eld of networks. This

tool extracts statistical data from the data plane in order to check if the network behavior is theexpected one. The information collected by this module is the following:

NetIDEPublic

27



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



0

100

200

300ap

p_m

anag

er.p

y:24

(<m

odul

e>)

cfg.

py:1

3(<

mod

ule>

)cf

g.py

:17(

<m

odul

e>)

cfg.

py:3

16(<

mod

ule>

)co

llect

ions

.py:

288(

nam

edtu

ple)

cont

rolle

r.py:

138(

_rec

v_lo

op)

cont

rolle

r.py:

218(

serv

e)co

ntro

ller.p

y:23

(<m

odul

e>)

cont

rolle

r.py:

302(

data

path

_con

nect

ion_

fact

ory)

cont

rolle

r.py:

68(_

_cal

l__)

cont

rolle

r.py:

72(s

erve

r_lo

op)

epol

ls.p

y:61

(do_

poll)

gree

nio.

py:1

87(_

tram

polin

e)gr

eeni

o.py

:206

(acc

ept)

gree

nio.

py:3

06(r

ecv)

gree

nthr

ead.

py:2

12(m

ain)

hand

ler.p

y:10

8(re

gist

er_s

ervi

ce)

hub.

py:1

18(s

erve

_for

ever

)hu

b.py

:17(

<m

odul

e>)

hub.

py:3

18(r

un)

hub.

py:4

38(f

ire_t

imer

s)hu

b.py

:84(

join

all)

{__i

mpo

rt__

}__

init_

_.py

:124

(tra

mpo

line)

__in

it__.

py:1

7(<

mod

ule>

)__

init_

_.py

:1(<

mod

ule>

)__

init_

_.py

:6(<

mod

ule>

)__

init_

_.py

:99(

<m

odul

e>)

insp

ect.p

y:10

26(g

etou

terf

ram

es)

insp

ect.p

y:10

53(s

tack

)in

spec

t.py:

25(<

mod

ule>

)lo

g.py

:17(

<m

odul

e>)

man

ager

.py:

19(<

mod

ule>

)m

anag

er.p

y:59

(mai

n){m

ap}

{met

hod

'ena

ble'

of '

_lsp

rof.P

rofil

er' o

bjec

ts}

{met

hod

'pol

l' of

'sel

ect.e

poll'

obj

ects

}{m

etho

d 's

witc

h' o

f 'gr

eenl

et.g

reen

let'

obje

cts}

ofp_

even

t.py:

19(<

mod

ule>

)of

prot

o_pr

otoc

ol.p

y:17

(<m

odul

e>)

poll.

py:7

6(w

ait)

prof

iler.p

y:5(

prof

iled_

func

)re

.py:

188(

com

pile

)re

.py:

226(

_com

pile

)re

ques

t.py:

1(<

mod

ule>

)si

mpl

e_sw

itch.

py:5

8(_p

acke

t_in

_han

dler

)sr

e_co

mpi

le.p

y:17

8(_c

ompi

le_c

hars

et)

sre_

com

pile

.py:

32(_

com

pile

)sr

e_co

mpi

le.p

y:47

8(_c

ode)

sre_

com

pile

.py:

493(

com

pile

)sr

e_pa

rse.

py:3

01(_

pars

e_su

b)sr

e_pa

rse.

py:3

79(_

pars

e)sr

e_pa

rse.

py:6

75(p

arse

)th

read

.py:

36(_

_thr

ead_

body

)tim

er.p

y:53

(__c

all_

_)to

keni

ze.p

y:23

(<m

odul

e>)

type

s.py

:13(

<m

odul

e>)

type

s.py

:20(

<m

odul

e>)

wsg

i.py:

17(<

mod

ule>

)w

sgi.p

y:1(

<m

odul

e>)

Function Name

Exe

cutio

n T

ime

(sec

onds

)

0

100

200

300Seconds

Figure 3.7: The Highest values of execution times.

• Port statistics (For each active port of the switch) contain: received packets, transmittedpackets, received bytes, transmitted bytes, packets dropped by RX, packets dropped by TX,number of receive errors, number of transmit errors, number of frame alignment errors, numberof packets with RX overrun, number of CRC errors and number of collisions.

• Flow statistics provide: table id, match (description of �elds), duration (time �ow has beenalive in seconds), duration (time �ow has been alive in nano seconds), priority, idle timeout,hard timeout, cookie (Opaque controller-issued identi�er), packet count (number of packetsin �ow), byte count (number of Bytes in �ow), actions.

• Table statistics: active entries, packet lookups and packet matches.

• Queue statistics show: received packets, transmitted packets, received bytes, transmittedbytes, dropped packets, received errors, transmitted errors, number of frame alignment errors,number of CRC errors and collisions.

• Flow aggregate statistics o�er: packet count (number of packets in �ows), byte count (numberof Bytes in �ows), �ow count (number of �ows).

With this statistical parameters, it is possible to obtain other relevant information, such as thelink with the most reduced throughput and therefore a possible network bottleneck. The NetworkPro�ler displays a menu where the user can choose the statistics to be visualized.Figure 3.8 shows the Pro�ler Network's menu. The user �rst must create the message, in this

case a �ow aggregate statistics request has been originated introducing the option six. After this

NetIDEPublic

28



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Figure 3.8: Network Pro�ler displaying �ow aggregate statistics.

step, the user must enter the option one to send the message to the data plane, immediately theuser will receive the response. In this example, the response is in blue color, notifying that thereare two active �ows in the network. There is only one response because the deployed topology onlycontained one switch.The Network Pro�ler has experimented an evolution, not in its functionality, but in its integration

within the NetIDE Engine architecture. In previous releases, this tool was composed by two SDNapplications running on top of a Ryu controller. It now consists on a program that is connectedto LogPub. The Network Pro�ler runs two processes concurrently, �rst one is for creating andsending the statistics request messages to the data plane and second one for receiving the statisticsresponse messages and printing them into a screen. This architectural change supposes two majorimprovements: Network Pro�ler is fully independent from the SDN technology used in the networkand it o�ers the capability of collecting statistics when the user prefers, so the information displayedis the most updated without overloading the network with statistics requests sent periodically.

3.7.1 Future lines of work

One of the strengths common in all the tools of NetIDE project is that the results generated bythem are easy to understand. However, in the case of the Application Pro�ler, the outcomes areuseful and easy to understand if the user has slight knowledge about the functions executed inthe SDN applications as well as in the controller. The user must know if the execution time ofa function is anomalous or not. It would be very useful to provide an interval for detecting adisproportionate execution time of a function, however it is not possible since the execution time ofa function depends on the purpose of such function. Nevertheless, this tool was designed to detectvery atypical execution times that can damage the whole performance of the network. For instance,a in�nite loop within a SDN application. This kind of anomalous outcomes can be identi�ed easily.If the problem is located in the applications, the developer has a deep knowledge about the codeand can determine whether the execution time of a function is correct or not. The ApplicationPro�ler could be extended with a module in charge of measuring the memory consumption of theapplications in order to detect memory leaks. This module was developed as a proof of concept butit has not been included in the �nal release due to a problem with the graphical representation ofthe outcomes generated.Regarding the Network Pro�ler, we will include the detection of link disconnections. This tool,

which is perfectly integrated into the IDE, will show, in a graphical way, that a link has beendisconnected. This feature is very interesting if a user has a complex topology with several links ofredundancy and wants to know which links are available to send tra�c.

NetIDEPublic

29



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3.8 Tra�c Emulator

The objective of the Tra�c Emulator tool is to provide a set of applications to emulate tra�c inthe network to allow for quick prototyping and performance prediction. A network may operatee�ciently under standard load, but may fail as tra�c levels increase due to several reasons. Theability to see these indicators will help the developer to include this in their design. As the developerwill require di�erent events to test operation under, the tool come with some samples included. Toincrease the takeup of the tool, the tra�c generation model is extendable to allow developers tocreate their own custom stories that may need to be catered for in their design. The applicationsprovide a list of con�gurable events that can be selected as parameters. This allows the developerto make edits to their code to make the network perform better during these events.The main requirements of this tool were:

• An engine that generates tra�c to run through the network : these are the di�erent applicationsdescribed below

• Rate of tra�c through the engine should be con�gurable to simulate network load or spikes.The di�erent tra�c generation models [30] of Greedy Source model, Poisson Tra�c model,Long-tail tra�c models, Payload data models are examples of what could be selected : theapplications support a wide range of parameters to emulate di�erent types of tra�c

• The engine should be extendable to allow developers to generate custom events: the code forthe applications is easily extendable to allow more behaviours than the defaults

• A simple user interface or reporting screen: the tools use a command-line interface (CLI) butcould be integrated in the IDE

The current applications available are:

• http: a client-server application which simulate HTTP tra�c from a host to a server. Thetra�c patterns are similar to what can be found in this work [31].

• pcap: an application which takes .pcap �les like those who can be found in .pcap reposito-ries [32] [33] or using an external tool like DCT2Gen [34], and create traces playable by a toollike tcpreplay [35].

• tftp: a client-server application which simulate TFTP tra�c.

• iperfp: a application which uses iperf to simulate common �ow patterns (VoIP, video, ...) interms of packet size and bandwidth.

NetIDEPublic

30



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3.9 Memory Management System

The Memory Management System (MMS) for SDN aims at improving the TCAM usage and as aconsequence, the robustness of the network. It transparently optimizes the usage of the availablememory by automatically removing the �ow entries installed by applications that are no longer run-ning (the Garbage Collector introduced in Deliverable D4.2 [22], now called memory de-allocation)and automatically moving the less used wildcard entries to a slower memory when the TCAM isfull (memory swapping).In this section we introduce the memory swapping feature of the MMS and then we evaluate it

in Section 4.4.

3.9.1 Memory Swapping design

The memory swapping mechanism automatically frees the TCAM memory of the switches from theleast used wildcard �ow entries by temporarily moving them to a slower memory. The swappingprocess consists of two di�erent operations. The �rst, called swap out, is performed when the SDNcontroller detects that the �ow tables are full. In this case the mechanism swaps out the least usedwildcard rules to free up TCAM memory space for new entries. Vice-versa, the swap in operationrestores the swapped out rules when they are needed again by the network device to forward thetra�c.

3.9.2 Swap out

By default, the swap out operation is executed when the SDN controller detects that one or moreswitches are operating in table full condition. In case of OpenFlow-enabled switches, that condi-tion is noti�ed to the SDN controller via a TABLE_FULL error message. From OpenFlow 1.4.0 orhigher, the swapping mechanism can be also con�gured to react to TABLE_STATUS events with rea-son VACANCY_DOWN, meaning that the remaining space in the �ow table has decreased to less thana pre-de�ned threshold.Unlike a traditional computer OS virtual memory system, where a page is either allowed to be

swapped out or not, the removal of a �ow rule from the TCAM may change the semantic of thenetwork. This problem may happen because high priority rules may be dependent on low priorityones. As a trivial example, consider a rule set where the default rule drops any packet and anyallowed communication is handled with higher priority rules. When the MMS swaps out an entry ofthis rule set because of resource pressure, any further packet matching this rule will be then caughtby the default rule and dropped, which is an undesirable result. So, whenever we consider a rule asa candidate to swap out, we have to build the (transitive) set of (lower priority) rules which are alsoa�ected. The analysis of the rule-dependency problem, which is outside the scope of this paper, hasalready been tackled by other works such as CacheFlow [36].Authors of CacheFlow propose an algorithm to incrementally analysing and maintaining rule de-pendencies. In such a work, a dependency between a child rule C and a parent rule R is de�ned asfollows: if C is removed from the �ow table, packets that are supposed to hit C will hit rule P.The current design of the MMS leverages on the CacheFlow's algorithms to correctly compute thedependencies between rules and, consequently, to swap out/in the wildcard entries between TCAMand MMS databases. Please note that, the memory swapping mechanism focuses on wildcard en-tries to free TCAM memory space. Thus, it does not take into consideration the rarely used exactmatch rules (which are likely stored in other memories such as DRAM or SRAM), to avoid the riskfreeing the wrong memory. Since exact match rules cannot be parents of wildcard rules, swappingonly wildcard rules does not create any inconsistency in the switch's �ow table.

NetIDEPublic

31



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



The pseudo-code in Listing 3.5 illustrates how the swap out process is executed. When an SDNapplication requests for a �ow rule installation, the MMS intercepts the rule, it checks whether therules contains any wildcard and computes the dependencies with the other wildcard rules alreadyinstalled in the network and saved in the flow_db (lines 2-4 in the Listing). Then, the rule isstored in the flow_db database (line 5) along with the dependency chain. Before installing the �owrule, we retrieve the dependency chain of the new rule from the swap_db that stores the rules thathave been previously swapped out (line 6), in order to avoid network inconsistencies. The actualinstallation of the new rule onto the device is performed at line 7. If the operation returns eithera TABLE_FULL error or a VACANCY_DOWN event, the swapOut function is called to free up some spacein the switch �ow table (line 8) and then the rule is �nally installed.

1 function installFlowRule(flow_rule fr):

2 if check_wildcards(fr) is True:

3 dep_chain = compute_dependencies(fr,flow_db)

4 fr.addDependencies(dep_chain)

5 flow_db.add(fr)

6 swap_chain = get_dependencies(fr,swap_db)

7 while install_rules(fr ,swap_chain) is False:

8 swapOut ()

910 function swapOut ():

11 less_used_fr = get_less_used_fr(flow_db ,quota)

12 for fr in less_used_fr:

13 dep_chain = get_dependencies(fr,flow_db)

14 for dep_fr in dep_chain:

15 if dep_fr.timeout is 0:

16 swap_db.add(dep_fr)

17 remove dep_fr

Listing 3.5: Flow rule installation process.

The swapOut function gets the less used �ow entries from the flow_db (line 11), based on tra�cstatistics collected periodically (as described below in this section). quota is the percentage ofthe installed rules to be swapped out. By default, its value is 20% for all the switches, but thisthreshold can be tuned dynamically based on the performance of the network. Swapped out �owentries are removed from the TCAM of the devices and from the flow_db. Entries with in�nitetimeout are saved in the swap_db (line 16) and automatically restored by the MMS when necessary.Entries with �nite timeout are just dropped, based on the assumption that network applications canautonomously restore them, exactly how they would do if those �ow entries were naturally expireddue to the lack of matching tra�c.A network application may need to uninstall a �ow rule. In this case, the MMS automatically

deletes the corresponding entry from the flow_db and from the swap_db (if necessary) and updatesthe dependency chains of the child entries in those databases.The ranking of the least used rules is determined by periodically collecting tra�c statistics, such

as packet and byte counters, for each wildcard rule. For each rule, the complete history of thecollected samples is kept in the flow_db until the rule is deleted. The classi�cation of the rules iscomputed with the Exponential Weighted Moving Average (EWMA) algorithm [37], which weightsthe statistics in geometrically decreasing order so that the most recent samples are weighted morethan the oldest samples. This approach avoids erroneous classi�cations where, for instance, a �owentry periodically matched by a micro-�ow has more chances to be swapped out than an entrymatched by an elephant �ow far in the past, i.e. no more active.

NetIDEPublic

32



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



3.9.3 Swap in

Swapped out �ow rules are automatically re-installed by the MMS onto the network when needed,transparently to the SDN applications running atop the SDN controller. The mechanism is fairlysimple: when a switch does not �nd a match for an incoming �ow inside its �ow tables, it sendsa new �ow message (e.g. an OpenFlow PACKET_IN) to the SDN controller. The MMS interceptsthe message before it arrives to the applications and checks whether the �ow matches any of theswapped out rules in the swap_db database (line 2 in Listing 3.6). If the MMS �nds a match, itautomatically reinstalls the rule along with its dependency chain into the switch's memory (lines3-5), otherwise the new �ow message is released to the other processes of the SDN controller, whicheventually relays it to all the listening applications.

1 function swapIn(packet pkt):

2 fr = swap_db.get_rule(pkt)

3 if fr is True:

4 swap_chain = get_dependencies(fr,swap_db)

5 install_rules(fr, swap_chain)

Listing 3.6: Swap in process.

NetIDEPublic

33



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



4 The NetIDE Engine Evaluation

This section presents an evaluation of NetIDE Engine performance. NetIDE project o�ers thecapability of reusing SDN applications (written for di�erent controllers) in the same network, thanksto the Engine which is able to manage di�erent controller frameworks and compose the executedSDN applications. Nevertheless, these features imply a cost in terms of latency introduced on thecontrol channel, which is evaluated in the following sections.A preliminary performance evaluation of the Engine has already been provided in a conferencepaper [38]. Compared to that work, the evaluation presented in this document includes (i) testsperformed with two di�erent versions of the OpenFlow protocol (v1.0 and v1.3) and (ii) a scalabilityassessment in terms of number of running application modules and packet rate.Please consider that the results presented in this document have been obtained on a di�erentexperimental setup and by using newer versions of the Engine's components.The second part of this Chapter evaluates the scalability of companion tools and LogPub, the

interface which connect the tools to the NetIDE Engine.

4.1 Engine Latency Tests

This section provides the outcomes of the performance tests accomplished in order to show thelatency introduced by the NetIDE Engine. There are two main sources of latency: the serializationand deserialization processes and the composition mechanism. Therefore, the consequences of thesetwo operations are evaluated by measuring the Round Trip Time (RTT) of ICMP packets.

4.1.1 Test Methodology

Experimental setup. For carrying out the experiments we use a server with the following features:The Operative System is CentOS v7.1, with 92GB RAM and 3.2TB of Hard Disk. In this serverfour virtual machines are installed. The hypervisor used to virtualize is KVM (Kernel-based VirtualMachine) which is a full virtualization solution for Linux on x86 hardware containing virtualizationextensions (Intel VT or AMD-V). The �rst one contains the NetIDE Engine and its con�gurationis: 4 Cores with 16GB RAM and 50GB of storage with an Ubuntu 14.04 LTS. In the second virtualmachine is installed an Open vSwitch (OVS) instance, con�gured with one Core, 2GB RAM and2GB of storage. Finally the other two remaining virtual machines run an Ubuntu 14.04 LTS andare con�gured with a single Core, 2GB RAM and 5GB of storage respectively. Speci�cally, thevirtual machine which contains the Engine, executes the Ryu Client Controller, the Core and theRyu Server Controller. The Engine is connected to the OVS and the other two virtual machinesact as two hosts connected to the OVS machine. So, the resulting scenario is depicted below inFigure 4.1:Metric. As mentioned before, the e�ciency of the Network Engine is evaluated by measuring the

RTT of ICMP packets exchanged between the two hosts. The application running on top of the RyuClient Controller is a learning simple switch application modi�ed to force that all the packets gothrough the Network Engine before being forwarded. Thus, the measured RTT values include twicethe delay caused by the Network Engine's operations, once for each direction of the communication.Please note that tests are performed at 20 ICMP pkt/sec.

NetIDEPublic

34



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Application Modules

Client Controller (Ryu / FL )

Server Controller (Ryu / ONOS / ODL)

Core

VirtualMachine

VirtualMachine

VirtualMachine

VirtualMachine

Figure 4.1: Latency Tests Scenario.

4.1.2 Serialization/Deserialization

Serialization and Deserialization process is de�ned as the operations the Engine performs in orderto convert a message from a chain of bytes into controller's north bound interface format and vice-versa. Therefore, this process is carried out in all the modules of the Engine. Figure 4.2 shows thedi�erence between the amount of serialization/deserialization operations in a standard controllerand in the NetIDE Engine.

Client Controller

Core

Server Controller

Application modules

SDN Controller

Packet In Packet In

Application modules

Figure 4.2: Blue circles represents the serialization and deserialization operations that are performedin each SDN architecture.

As it can be observed in Figure 4.2, NetIDE Engine needs to carry out �ve times more theseoperations than common SDN controllers. The additional delay that the NetIDE Engine introduceswill depend on how the modules implement the serialization and deserialization functions. Therefore,the latency tests are done with di�erent combinations of client and server controller frameworks.Table 4.1 and plot in Figure 4.3 show how the RTT of ICMP packets is a�ected by the serializa-

tion/deserialization operations performed on PACKET_IN and FLOW_MOD OpenFlow messages.

NetIDEPublic

35



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



6.46

11.7

17.94

13.15

0

5

10

15

No Network Engine ONOS OpendayLight RyuServer Controller Platform

Ave

rage

RT

T (

ms)

Client ControllerRyu

(a) OpenFlow Version 1.0

5.22

11.4411.94

13.01

0

5

10

15

No Network Engine ONOS OpendayLight RyuServer Controller Platform

Ave

rage

RT

T (

ms)

Client ControllerRyu

(b) OpenFlow Version 1.3

Figure 4.3: RTT measured with di�erent combinations of client-server controller frameworks.

Table 4.1: Con�dence Intervals with 95% of probability of RTT means.

No Network Engine ONOS Opendaylight RyuOpenFlow v.1.0 6.25 - 6.66 11.61 - 11.79 17.57 - 18.30 13.04 - 13.25OpenFlow v.1.3 5.16 - 5.27 11.36 - 11.85 12.02 - 11.86 12.92 - 13.09

As expected the RTT values are higher when the SDN network is controlled by the NetIDE Engine.Moreover, the results tell us that the delay does not depend on the OpenFlow version, except incase of ODL, where the serializer/deserializer implemented for version 1.3 performs better that thesame functions implemented for version 1.0. In a deeper analysis, we can observe that in the case ofOpenFlow 1.0, The NetIDE Engine using ONOS and Ryu as server controllers, introduces a latencytwo times higher than in a common SDN scenario (without the NetIDE Engine) and approximatelythree times higher if the server controller is OpendayLight. However, as we explained above, theNetIDE Engine must perform �ve serialization/deserialization operations while a common controlleronly accomplishes one. In the case of OpenFlow 1.3 all the server controllers add a similar latency(2.5 times the latency without Engine) due to OpendayLight implements an optimized version ofthe functions in charge of serialization processes.As a conclusion, the outcomes obtained are quite positive since the latency does not increase

linearly with the number of serialization/deserialization operations. Furthermore, in the paper [39]where are performed similar tests, the authors obtain slightly better outcomes in a non-virtualizedenvironment. In fact, we have reproduced such experiments in a non-virtualized environment, usingRyu as the controller platform and the results obtained have been similar to the outcomes of theaforementioned paper.

4.1.3 Composition

We compare the performance of composed SDN applications with what we call �merged SDN appli-cations�. Thus, we run SDN modules on the Network Engine composed in parallel and we measurethe RTT of ICMP packets. Then, we merge the code of such SDN modules into a single SDNapplication that provides the same functions of the composed application, and we run it on theNetwork Engine with the composition disabled. The di�erence in performance measures the cost ofthe composition in terms of additional latency on the control channel.To this end, we compare a �composed application� built with a L2 forwarding module plus a moni-

NetIDEPublic

36



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



toring module, with a �merged application�, which is a monolithic SDN application resulting frommerging the code of the two aforementioned modules.The comparative between both experiments is shown in Figure 4.4.

Merged application avg. RTT: 15.67Composed application avg. RTT: 19.55

0.00

0.25

0.50

0.75

1.00

0 20 40 60Round Trip Time (ms)

Cum

ulat

ive

Pro

babi

lity

Composed ApplicationMerged Application

(a) OpenFlow Version 1.0

Merged application avg. RTT: 13.19Composed application avg. RTT: 15.55

0.00

0.25

0.50

0.75

1.00

0 10 20 30 40Round Trip Time (ms)

Cum

ulat

ive

Pro

babi

lity

Composed ApplicationMerged Application

(b) OpenFlow Version 1.3

Figure 4.4: Cumulative probability of the RTT for composed and merged applications.

Figure 4.4 shows the ECDF of the measured RTT for the applications when are composed andfor the single merged application without composition. The left plot contains the experiments forOpenFlow 1.0 while the right plot provides the experiment outcomes using OpenFlow 1.3. Both�gures o�er similar results, therefore the latency added by the composition mechanism does notdepend on the OpenFlow version. In the worst case, the composition mechanism increases less than4 ms the RTT on average.The conclusion for the experiments is that the impact of the composition mechanism on the

latency is negligible. We expect the impact of composition may increase with the number of appli-cation modules, but it is a performance penalty that can be tolerated when considering the e�ortthat would be needed to re-implement all modules for a di�erent controller framework.

4.2 Engine Scalability Tests

This section aims to evaluate the scalability of the composition mechanism implemented on theNetIDE Engine. For this purpose, we accomplish a set of tests in which gradually is increased thenumber of SDN application modules executed. Speci�cally, we run multiple instances of the sameSDN module. All of them receive the new �ow event at the same time, however only the one withhigher priority actually controls how the tra�c is forwarded across the network.We assess the consequences of running and composing multiple application modules measuring

the RTT of ICMP packets. Test are performed at rates of 10, 20 and 40 new �ows per second (for�ve and ten application modules) and according to the publication [39] these rates represent typicalcontrol tra�c rates in a proactive SDN scenario.The RTT results represented in Table 4.2 do not increase linearly with the number of executed

applications, since the purpose of the Core is to support di�erent controller platforms and carry outthe composition of SDN application modules, however it was not designed for high performance.Therefore, the Core operates worse the more modules are executed causing an increase in thelatency. Nevertheless, the outcomes are quite positive, with less than �ve application modules, theEngine behaves properly for impressive packet rates, such as 200 packets per second. So, we canconclude that the composition mechanism is robust enough to manage scenarios with complex SDNapplications introducing an a�ordable latency.

NetIDEPublic

37



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Table 4.2: Average RTT when varying the packet rate and the number of application modules.

Modules Pkt/sec Average RTT (ms)

80 11.511 200 14.1

400 29.4

10 19.965 20 19.99

40 21.79

10 30.7510 20 31.46

40 61.56

4.3 Tools Architecture Evaluation

As presented in section 3.1, the Tools Architecture evolved quite much during the project. As moreissues were �xed and more tools were added, the question of scalability arose. We saw in our generalevaluation of the Network Engine that the Java process of the Core use most of the workload on asystem. We did not look into the subdivision of this load in each individual element of the Core.Furthermore, we did dot include advanced optimization in our evaluation as we focused on theresearch aspects of the project and preferred arriving at a functioning proof of concept rather thana pre-optimized product. That being said, we saw that the Tools (and thus the whole architectureas well) can handle more than 100 messages per second.In order to prove that both the LogPub (Core module that acts as an interface between the

tools and NetIDE Engine) and the tools are able to handle a large amount of messages withoutscalability problems, two experiments have been accomplished. The �rst test groups the tools whichare connected to the LogPub for receiving messages, for instance: Logger, Debugger and Veri�cator.The second test is focused on the tools that not only receive but send messages to the network, thisis the case of the Network Pro�ler.

4.3.1 First Experiment

The goal of this test is to measure the capacity of the LogPub to receive and send a signi�cantamount of messages in a short time lapse and check if the tools are able to retrieve those messagesand perform their respective functions.Scenario. We use the same server described in section 4.1. The con�guration of the NetIDE

Engine is the following: L2 forwarding application running on top of the Ryu Client Controller,the Core and Ryu Server Controller. Please note that the L2 forwarding application is modi�ed toenforce that all packets sent by the hosts go through the NetIDE Engine and therefore the messagesreach the LogPub. The modi�cation made in the application is for avoiding the installation of rulesin the switch, so the switch sends the incoming packet to the NetIDE Engine. In Figure 4.5, theexperiment setup is represented in detail.Methodology. To avoid interferences, each tool is evaluated individually in two di�erent steps.

In the �rst step, we send 2000 ICMP packets at a rate of 100 messages per second. Those packetsare ICMP requests, therefore when the other host receives them responds with a ICMP reply. So,we have a rate of 200 PACKET_IN in a second, which are handled by the L2 forwading applicationwith 200 FLOW_MOD. All these messages are queued by the LogPub module. Finally, the tool retrievesthe messages from the queue and performs its correspondent task. In the second step, we double

NetIDEPublic

38



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Simple switch

Ryu Client Controller

Ryu Server Controller

Core

Logger

Debugger

VerificatorLogPub

Figure 4.5: Scenario for the �rst experiment of tools scalability.

the packet rate to 200 ICMP requests per second.Results. For the di�erent packet rates, the behavior and performance of the tools changes. In

the �rst test, the outcomes are the followings: the Debugger can receive the messages and writethem into di�erent �les without any inconsistency. The Veri�cator retrieves the messages, processesthem and shows the information without collapsing. Finally, the Logger is able to retrieve themessages, convert them into human readable format and display the information in a terminal,however there is substantial delay between the last ICMP packet transmission and the last messagethat the Logger displays in the terminal.For the second experiment, the Debugger and Veri�cator perform their functionalities without any

problem. Nevertheless, the high consumption of resources of the Logger provokes a general failurein the Engine causing losses of ICMP packets. Therefore, the Logger does not operate properlywith a rate of 200 ICMP requests per second.

Table 4.3: Tool performance at di�erent transmission rates.

Transmission rate LogPub Logger Debugger Veri�cator

100 ICMP per second X X X X200 ICMP per second X x X X

As a conclusion, while the Debugger, the Veri�cator and the LogPub interface are able to operateproperly even at high packet rates, we found that the Logger cannot handle more that 200 packetsper second.

4.3.2 Second Experiment

This second experiment aims to evaluate the proper operation of a tool that sends and receivesmessages at the same time. This kind of tool could be more sensitive to high loads of messagessince it must keep a bidirectional communication with the LogPub apart of accomplishing its ownfunctionality concurrently.

NetIDEPublic

39



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Scenario. The scenario setup is similar to the previous experiment. However, there are somemodi�cations to be highlighted. Instead of connecting the tools depicted in Figure 4.5, it is connecteda modi�ed version of the Network Pro�ler able to send statistic requests to the data plane at di�erentrates. Furthermore, the application running on top of the Ryu Client Controller is a simple L2forwarding application. Finally, the topology changes depending on the test, we begin with oneswitch and we �nalize with four switches.Methodology. By means of the modi�ed Network Pro�ler, we send ten statistics requests to

the data plane at di�erent rates (from 1 request each 5 seconds to 1 request each 500 milliseconds).Such messages are received by the switches, which generate and send statistics responses to theEngine. Finally, our tool receives such responses thanks to the LogPub. Then, the tool processesand displays them into a terminal. This experiment is accomplished at di�erent transmission rates,increasing progressively the number of devices of the data plane. The purpose of the experiment isto asses the performance of the tools that sends and receives messages at the same time, such as:the Network Pro�ler. This second experiment breaks down in four tests explained above.In the �rst test, the topology is composed by one switch and two hosts. The modi�ed Network

Pro�ler sends ten statistics requests to the data plane. The requests are sent every �ve seconds. Weevaluate if the tool is able to receive the correspondent ten statistics responses and display theminto a terminal. The process is repeated with the following transmission rates: one request each2 seconds, one request each second and one request each 500 milliseconds. The second, third andfourth tests are identical but increasing one switch in the topology. Therefore, in the second testwe have two switches which generate two responses when a statistics request is received, while inthe fourth test, the four switches generate four statistics replies to one request.Results. Table 4.4 summarizes the outcomes of the accomplished tests at di�erent transmission

rates.

Table 4.4: Outcomes when varying the transmission rate and the number of switches.

Number of Transmission Requests Responses All the responsesswitches rate sent received were displayed

1/5 s 10 10 X1 1/2 s 10 10 X

1/1 s 10 10 X1/500 ms 10 10 X

1/5 s 10 20 X2 1/2 s 10 20 X

1/1 s 10 20 X1/500 ms 10 20 X

1/5 s 10 30 X3 1/2 s 10 30 X

1/1 s 10 30 X1/500 m 10 30 X

1/5 s 10 40 X4 1/2 s 10 40 X

1/1 s 10 40 X1/500 ms 10 40 X

15 1/1 s 10 150 X

30 1/1 s 10 300 x

NetIDEPublic

40



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



There are monitoring tools that poll the data plane to retrieve network statistics. For a higheraccuracy the tool must send statistics requests very often (one statistics request each second). Withthis experiment we are demonstrating that the Network pro�ler performs quite well since it sends 2statistic requests per second and it is able to receive and process their respective responses withoutcollapsing. Therefore, the Network Pro�ler is able to operate at very aggressive transmission rates.However, the topology of our experiment is composed by four switches (in the last test). Therefore,we have accomplished two tests more emulating a topology (using mininet) with �fteen and thirtyswitches respectively. The tool has sent one statistics requests per second in both cases. In the�rst test, the tool has operated properly without any inconsistency showing a good performance.However, with 30 switches the tool has been able to receive all the messages but it has not beenable to process and display the statistics in a terminal.

4.4 Evaluation of the Memory Management System

We implemented the memory swapping for the ONOS platform as an additional component forthe server controller in the Network Engine's architecture [40]. In this section, we evaluate thememory swapping mechanism by considering two metrics: (i) average end-to-end throughput and(ii) TCAM space available for new �ow entries. The e�ectiveness of the proposed approach ismeasured by comparing the results observed with and without the memory swapping using �owtable of di�erent sizes.

4.4.1 Test methodology

Experimental setup. For the experiments we use both hardware and software OpenFlow-enabledswitches. In the �rst case we use the NEC IP8800 [41], while in the second, Mininet [42] and OpenvSwitch (OVS) [43] switches.We run ONOS and the MMS on a commodity PC equipped with a Intel i7-5600U quad-core CPUrunning at 2.60GHz and 16GB of DDR3 memory working at 1600Mhz. This machine is connectedto the NEC switch via Gigabit Ethernet for the OpenFlow control channel. Two physical hostsare connected to the switch via Gigabit Ethernet to inject network tra�c during the evaluation.For the test with software switches, the commodity PC also hosts Mininet con�gured with a singleOVS-based switch and two virtual hosts attached to it.We use publicly available SMTP and HTTP tra�c traces from [44]. The �rst one produces 23 new�ows per second on average when considering Layer 3 �elds, while the second trace produces 65new �ows per second on average.

Context. The result of the evaluation depends on three main aspects: (i) the �ow table size ofthe switch, (ii) the �ow rate of the tra�c trace, i.e., number of new �ows per second, and (iii) howfast the corresponding �ow entries expire. Thus, we expect our memory swapping mechanism to bemore e�ective with small �ow tables and high rate of �ows controlled using �ow entries with highexpiry timeouts.To stress the memory swapping, we started with the HTTP trace, but unfortunately we experiencedmany disconnections of the OpenFlow channel due to CPU overload of our NEC switch caused bythe relatively high �ow rate produced by this trace. Due to this, we were forced to limit the evalu-ation with the hardware switch to just the SMTP trace.The TCAM of our NEC IP8800 can host up to 1500 �ow entries, but we are interested to un-derstand the e�ectiveness of the memory swapping mechanism when varying the size of the �owtable. For this reason, we con�gure the OVS switch at di�erent �ow table sizes, from 1500 entries(like our NEC), to 2000 entries like a HP 8200/5400 [45]. In this case, we use the HTTP tra�c trace.

NetIDEPublic

41



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Methodology. We compare the memory swapping implementation for ONOS with the defaultONOS memory management system. By default, ONOS holds in pending add state the �ow rulesthat cannot be installed until there is enough memory space in the switches. We demonstrate thatour mechanism is better in terms of (i) memory space available in the switches for the new entries,and (ii) performance of the network measured in terms of end-to-end throughput.All the experiments are executed under the following conditions:

• At time 0, one of the hosts starts injecting the tra�c trace into the switch (either hardwareor software). At this point in time, the �ow table of the switch is empty.

• The switch is controlled by ONOS via the ReactiveForwarding application [46]. This applica-tion reactively installs a �ow entry in the switch for every new incoming �ow.

• The application is con�gured to generate wildcard �ow entries with the only IP source anddestination addresses speci�ed. The application also randomly assigns either in�nite or 10seconds idle timeouts to the �ow entries.

• Dependencies between �ow table entries are synthetically generated based on pairs of randomIP subnet masks and priorities. Thus, we con�gured the forwarding application to randomlyapply di�erent levels of priorities combined with di�erent submasks by following the longestpre�x match principle, where the longer the subnet mask, the higher the priority.

4.4.2 Results and discussion

When the TCAM of a switch is full, new �ow entries cannot be inserted by the controller to handlethe incoming tra�c. This condition produces two main issues which then lead to a drop in the �owthroughput:

• The switch keeps on sending PACKET_IN messages to the controller for the same �ow, resultingin an overload of the controller.

• The switch bu�ers the packets that do not match any �ow entry in the forwarding table.However, bu�ers are usually small and, when the TCAM is full, they get full as well resultingin loss of packets.

With the proposed memory swapping mechanism, the insertion of new �ow rules cannot beprevented by a TCAM full condition, therefore we expect an increase in the end-to-end throughputwith respect to the default ONOS mechanism based on keeping the �ow rules in pending state.Table 4.5 summarizes the results obtained with the NEC hardware switch using the SMTP tra�ctrace (23 new �ows per second), and with the OVS software switch using the HTTP trace (65 new�ows per second).Speci�cally, when using the memory swapping we measure a throughput increase from 15.81

Kbps to 19.85 Kbps (21% on average) with the NEC switch and the SMTP trace. When using thesoftware switch and the HTTP trace, we observe di�erent performance depending on the size of the�ow table. We measure throughput increases of 26%, 15% and 6% with �ow table sizes of 1500,1750 and 2000 �ow entries respectively.Results obtained with the OVS-based software switch are also reported in Figures 4.6, 4.7 and

4.8. Vertical blue lines in Figures 4.6 and 4.7 represent the TABLE_FULL error messages sent by theswitch operating in full table condition to ONOS, when ONOS tries to install the pending rules.By default, this installation process is automatically performed every 5 seconds if the pending addqueue is not empty. Please note that, for the sake of readability, we represent only one error message

NetIDEPublic

42



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Table 4.5: Average throughput.

Flow Table SizeTra�c

trace

Throughput MMS

Active [Kbps]

Throughput MMS

Inactive [Kbps]

1500 (NEC) SMTP 19.85 15.81

1500 (OVS) HTTP 80.17 58.97

1750 (OVS) HTTP 85.63 72.38

2000 (OVS) HTTP 86.54 81.13

0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]

MMS inactiveTABLE_FULL event

(a) Flow Table size: 1500 rules.

0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]


(b) Flow Table size: 1750 rules.

0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]


(c) Flow Table size: 2000 rules.

Figure 4.6: Number of installed rules without memory swapping at di�erent �ow table sizes.

0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]

MMS activeTABLE_FULL event


0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]



0

500

1000

1500

2000

2500

0 10 20 30 40 50 60 70 80 90

# flo

ws

Time [s]



Figure 4.7: Number of installed rules with memory swapping at di�erent �ow table sizes.

0

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60 70 80 90

Thro

ughp

ut [K

bps]

Time [s]

MMS activeMMS inactive


0

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60 70 80 90

Thro

ughp

ut [K

bps]

Time [s]

MMS activeMMS inactive


0

20

40

60

80

100

120

140

160

0 10 20 30 40 50 60 70 80 90

Thro

ughp

ut [K

bps]

Time [s]

MMS activeMMS Inactive


Figure 4.8: Throughput comparison with and without memory swapping at di�erent �ow table sizes.

NetIDEPublic

43



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



for each attempt.Without the memory swapping, the �ow table is constantly full and every time ONOS tries toempty the pending add queue, it gets a TABLE_FULL error (Figures 4.6a and 4.6b). The problemis partially mitigated when the software switch is con�gured with a larger �ow table (Figure 4.6c).In this case, the number of �ow rules which are evicted for expiring timeout is often su�cient tomake space for the rules waiting in the pending add queue. Conversely, the memory swappingprocess frees the 20% of the TCAM capacity in reaction to TABLE_FULL errors (Figure 4.7). Moreprecisely, the least matched entries are moved to the Swap Database and the free space in theTCAM is used by ONOS to install the rules in pending add state and, possibly, the new rulesthe ReactiveForwarding application generates to control new �ows. The bene�ts of the memoryswapping are demonstrated by the reduced number of TABLE_FULL errors, as shown in Figure 4.7,and by the increased performance in terms of end-to-end throughput at di�erent �ow table sizes,as shown in Figure 4.8 and also summarized in Table 4.5.

NetIDEPublic

44



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



5 Conclusions

The objective of this document is to show the progress on the implementation of the NetIDEarchitecture: the Engine and the Tools during the third year of the project.The �nal release of theNetIDE Engine includes three Backends (Ryu, ONOS and Floodlight), three Shims (Ryu, ODL andONOS) and one Core that includes the module for integration of the tools (LogPub).We implemented the NetIDE Network Engine based on SDN controller platforms available as

open-source, we adopted OpenDaylight and ONOS as server controllers, Floodlight as client con-troller and Ryu as both. ODL and ONOS are carrier-grade controllers targeted to service providers,enterprises and mainstream deployments. Being able to scale to a large number of network ele-ments, we recognized both ODL and ONOS as the most suitable SDN platforms for playing theserver controller's role. On the other hand, Ryu and Floodlight are aimed at facilitating the rapidprototyping of SDN applications. We were able to isolate their Southbound Interface (SBI) driversto add our backend alongside, and to hook into their event dispatching mechanism and drive theexecution logic.The �nal release of the Tools includes the Logger, Pro�ler, Debugger, Veri�cator, Tra�c Emulator

and Memory Management System. Some of the tools require interfacing with the Core. This isachieved via the LogPub interface which is in charge of capturing the messages exchanged among theclient controllers and the server controller, and pushing them into a queue. Tools are implementedas external components which are independent from the SDN platforms that compose the NetworkEngine. The LogPub abstract the Tools to the rest of the Core and act as an unique point of entry.The core/LogPub use a common update cycle independently from the shim and backends. The onlyexception is the Memory Management System which is proposed as an additional component forSDN controllers (in our case we use it in the server controllers) to improve the performance of thenetwork by optimizing the usage of the Ternary Content Addressable Memory (TCAM) memory inthe SDN devices in the network infrastructure.We have analyzed the state of the art of SDN tools: what is an SDN tool, why is necessary, how

it �ts the SDN architecture, and provided a series of tools classi�ed per type. The �rst conclusion isthat we miss a standard architecture and interfaces generic enough to be used for the implementationof SDN-speci�c tools. Several SDN tools are currently available, but they are either planned forvery speci�c architecture models or de�ned for a particular SDN controller framework. Therefore,most of them can be hardly adopted in a production environment where the maintenance of thenetwork is a critical task, and where patching the components of the SDN network to merge thedi�erent approaches is usually not a viable option.A set of test on scalability and latency try to analyze the behavior and limits of some of the

main components of the NetIDE architecture. Regarding the latency introduced by the NetIDEEngine, there are two main sources of latency: the serialization and deserialization processes andthe composition mechanism. Therefore, the consequences of these two operations are evaluated bymeasuring the Round Trip Time (RTT) of ICMP packets. As expected, the RTT values are higherwhen the SDN network is controlled by the NetIDE Engine because it needs to carry out �ve timesmore these operations than common SDN controllers.Regarding the evaluation of the composition mechanisms, we compare the performance of paral-

lel composed SDN applications with respect a single application with the same functionality. Thedi�erence in performance measures the cost of the composition in terms of additional latency on thecontrol channel. The conclusion for the experiments is that the impact of the composition mech-

NetIDEPublic

45



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



anism on the latency is negligible. We also measure the scalability of the composition mechanismimplemented on the NetIDE Engine in terms of the number of SDN applications.In the case of the tools, we evaluate how both the tools and LogPub are able to handle a large

amount of messages. Two experiments have been accomplished. The �rst test groups the toolswhich are connected to the LogPub for receiving messages, for instance: Logger, Debugger andVeri�cator. The second test is focused on the tools that not only receive but send messages to thenetwork, this is the case of the Network Pro�ler. We also provide a evaluation for the particularcase of the Memory Management System.

NetIDEPublic

46



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



A Python NetIP library reference

Reference of the Python implementation of the NetIP library [47] compatible with the latest versionof the NetIDE Intermediate protocol (v1.4.1). For further details on the NetIDE protocol andmessages exchanged between the Network Engine layers, please refer to the Deliverable D2.7 of theNetIDE project [2].

A.1 netIDE_encode

This method encodes a SBI message (e.g. OpenFlow) into a NetIDE message.

def netIDE_encode(type , xid , module_id , datapath_id , msg):

msg is the serialized SBI message, while type, xid, module_id and datapath_id are the �elds ofthe NetIDE header as described in the protocol speci�cation in [2]. The returned value is the bytearray containing all the provided values.The following example shows how to encode an OpenFlow message:

msg.serialize ()

msg_to_send = NetIDEOps.netIDE_encode('NETIDE_OPENFLOW ', self.netide_xid ,

module_id , msg.datapath.id, str(msg.buf))

self.channel.socket.send(msg_to_send)

A.2 netIDE_decode

Method that decodes NetIDE messages from binary format into a tuple containing the NetIDEheader �elds and the SBI message carried by the NetIDE message.

def netIDE_decode(raw_data):

raw_data is the NetIDE message as received through the communication (ZeroMQ) channel. Themethod returns a tuple containing the following items:

['VERSION ', 'TYPE', 'LENGTH ', 'XID', 'MOD_ID ', 'DPID', 'MESSAGE ']

A.3 decode_header

This method is used to decode NetIDE header of a NetIDE protocol message (�rst 16 bytes).

def netIDE_decode_header(raw_data):

raw_data is the NetIDE message as received through the communication channel. The methodreturns a tuple containing the following items:

['VERSION ', 'TYPE', 'LENGTH ', 'XID', 'MOD_ID ', 'DPID']

Fields of the tuple can be obtained like shown in the following example:

NetIDEPublic

47



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



decoded_header = NetIDEOps.netIDE_decode_header(msg)

if decoded_header is False:

continue

message_type = decoded_header[NetIDEOps.NetIDE_header['TYPE']]

A.4 encode_handshake

Encodes the control protocols supported by the SDN platform that is sending the message tocomplete the initial handshake between the client and server controllers.

def netIDE_encode_handshake(protocols):

where protocol is a dictionary which contains the list of the supported protocols. For instance:

self.supported_protocols[OPENFLOW_PROTO] = [OPENFLOW_10 ,OPENFLOW_12 ,OPENFLOW_13]

self.supported_protocols[NETCONF_PROTO] = []

This method returns a binary array to be sent during the handshake process via a NETIDE_HELLO

message.

A.5 decode_handshake

It decodes a NETIDE_HELLO message received during the handshake process.

def netIDE_decode_handshake(raw_data , length):

raw_data is the payload of the NetIDE message as received through the communication channeland length is the length of such a payload in bytes. The methods returns a tuple which containsthe supported protocols.

A.6 netIDE_set_module_id

It takes a NetIDE binary message and returns a new binary message with a new modules_id value.

def netIDE_set_module_id(raw_data , new_mod_id):

raw_data is the NetIDE message including NetIDE header and payload, and new_mod_id is the newmodules_id value.

A.7 netIDE_set_xid

It takes a NetIDE binary message and returns a new binary message with a new xid value.

def netIDE_set_xid(raw_data , new_xid):

raw_data is the NetIDE message including NetIDE header and payload, and new_xid is the newxid value.

NetIDEPublic

48



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



B Java NetIP library reference

Reference of the Java implementation of the NetIP library [47] compatible with the latest versionof the NetIDE Intermediate protocol (v1.4.1). For further details on the NetIDE protocol andmessages exchanged between the Network Engine layers, please refer to the Deliverable D2.7 of theNetIDE project [2].The documentation here is automatically generated from the self documenting Java classes via

Javadoc and Texdoclet.Classes

ErrorMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Class representing a message of type ERROR.

FenceMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Class representing a message of type FENCE.

HeartbeatMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Message representing a hearbeat message

HelloMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Class representing a message of type HELLO.

ManagementMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Class representing a message of type MANAGEMENT.

Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Class representing a simple NetIP message.

MessageHeader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Class representing a NetIP header.

MessageType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Enumeration of MessageTypes according to the NetIP speci�cation.

ModuleAcknowledgeMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Represents a ModuleAcknowledge message.

ModuleAnnouncementMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Represents a ModuleAnnouncement message..

NetconfMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Class representing a message of type NETCONF.

NetIDEProtocolVersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Enumeration of the NetIDE Protocol version bytes and convicence methodsto parse the version numbers

NetIPConverter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Class to parse raw messages and convert them to NetIDE Message

NetIPUtils . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Utility methods for handling NetIP messages.

OpenFlowMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70Class representing a message of type OPENFLOW.

OpFlexMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Class representing a message of type OPFLEX.

Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Enumeration of known protocols, mapped to their respective representationvalue, as speci�ed in the NetIDE intermediate protocol speci�cation.

NetIDEPublic

49



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



ProtocolVersions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Enumeration of known protocol versions.

TopologyUpdateMessage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Message in NetIDE that signal an update of the Topology.

B.1 Class ErrorMessage

Class representing a message of type ERROR. Note that this only serves as a convenience class - ifthe MessageType is manipulated, the class will not recognize that.

Declaration

public class ErrorMessage

extends eu.netide.lib.netip.Message

Constructor summary

ErrorMessage() Instantiates a new Error message.

Method summary

getPayload()getSupportedProtocols() Gets supported protocols.setSupportedProtocols(List) Sets supported protocols.

Constructors

• ErrorMessage

public ErrorMessage ()

� Description

Instantiates a new Error message.

Methods

• getPayload

public byte[] getPayload ()

� Description copied from Message (in B.6, page 55)

Gets the current payload as bytes.

� Returns � the payload This method has to ensure that the returned payload re�ectsthe current state of any convenience �elds!

• getSupportedProtocols

public java.util.List getSupportedProtocols ()

NetIDEPublic

50



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Description

Gets supported protocols.

� Returns � the supported protocols

• setSupportedProtocols

public void setSupportedProtocols(java.util.List supportedProtocols)

� Description

Sets supported protocols.

� Parameters

∗ supportedProtocols � the supported protocols

Members inherited from class Message

eu.netide.lib.netip.Message (in B.6, page 55)

getHeader, getPayload, header, payload, setHeader, toByteRepresentation, toString

B.2 Class FenceMessage

Class representing a message of type FENCE. Note that this only serves as a convenience class - ifthe MessageType is manipulated, the class will not recognize that.

Declaration

public class FenceMessage


Constructor summary

FenceMessage()

Method summary

getPayload()setPayload(byte[]) Sets payload.

Constructors

• FenceMessage

public FenceMessage ()

NetIDEPublic

51



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Methods

• getPayload





• setPayload

public void setPayload(byte[] payload)

� Description

Sets payload.

� Parameters

∗ payload � the payload




B.3 Class HeartbeatMessage

Message representing a hearbeat message

Declaration

public class HeartbeatMessage


Constructor summary

HeartbeatMessage()

Constructors

• HeartbeatMessage

public HeartbeatMessage ()




NetIDEPublic

52



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



B.4 Class HelloMessage

Class representing a message of type HELLO. Note that this only serves as a convenience class - ifthe MessageType is manipulated, the class will not recognize that.

Declaration

public class HelloMessage


Constructor summary

HelloMessage() Creates a new instance of the HelloMessage class.

Method summary

getPayload()getSupportedProtocols() Returns the list of mappings of supported protocols.setSupportedProtocols(List) Sets the list of mappings of supported protocols.

Constructors

• HelloMessage

public HelloMessage ()

� Description

Creates a new instance of the HelloMessage class.

Methods

• getPayload





• getSupportedProtocols

public java.util.List getSupportedProtocols ()

� Description

Returns the list of mappings of supported protocols.

� Returns � The list of mappings of supported protocols.

• setSupportedProtocols

NetIDEPublic

53



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



public void setSupportedProtocols(java.util.List supportedProtocols)

� Description

Sets the list of mappings of supported protocols.

� Parameters

∗ supportedProtocols � The new list of mappings of supported protocols.




B.5 Class ManagementMessage

Class representing a message of type MANAGEMENT. Note that this only serves as a convenienceclass - if the MessageType is manipulated, the class will not recognize that.

Declaration

public class ManagementMessage


Constructor summary

ManagementMessage()

Method summary

getPayload()getPayloadString() Returns the payload stringsetPayloadString(String) Sets the payload string

Constructors

• ManagementMessage

public ManagementMessage ()

Methods

• getPayload





NetIDEPublic

54



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



• getPayloadString

public java.lang.String getPayloadString ()

� Description

Returns the payload string

� Returns � The payload string of the message

• setPayloadString

public void setPayloadString(java.lang.String payloadString)

� Description

Sets the payload string

� Parameters

∗ payloadString � the new payload string.




B.6 Class Message

Class representing a simple NetIP message. This also is the base class for concrete message classes.

Declaration

public class Message

extends java.lang.Object

All known subclasses

TopologyUpdateMessage (in B.19, page 76), OpFlexMessage (in B.16, page 72), OpenFlowMessage (in

B.15, page 70), NetconfMessage (in B.11, page 65), ModuleAnnouncementMessage (in B.10, page 64), Mod-

uleAcknowledgeMessage (in B.9, page 63), ManagementMessage (in B.5, page 54), HelloMessage (in B.4,

page 53), HeartbeatMessage (in B.3, page 52), FenceMessage (in B.2, page 51), ErrorMessage (in B.1, page

50)

Field summary

header The header.payload The payload.

Constructor summary

Message(MessageHeader, byte[]) Creates a new instance of the Message class.

NetIDEPublic

55



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Method summary

getHeader() Gets the header.getPayload() Gets the current payload as bytes.setHeader(MessageHeader) Sets the header.toByteRepresentation() Returns the message's byte representation, including the

header.toString()

Fields

• protected MessageHeader header

� The header.

• protected byte[] payload

� The payload.

Constructors

• Message

public Message(MessageHeader header ,byte[] payload)

� Description

Creates a new instance of the Message class.

� Parameters

∗ header � The header to use.

∗ payload � The payload.

Methods

• getHeader

public MessageHeader getHeader ()

� Description

Gets the header.

� Returns � the header.

• getPayload


� Description



NetIDEPublic

56



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



• setHeader

public void setHeader(MessageHeader header)

� Description

Sets the header.

� Parameters

∗ header � The new header.

• toByteRepresentation

public byte[] toByteRepresentation ()

� Description

Returns the message's byte representation, including the header.

� Returns � The byte representation.

• toString

public java.lang.String toString ()

B.7 Class MessageHeader

Class representing a NetIP header.

Declaration

public class MessageHeader

extends java.lang.Object implements java.lang.Cloneable

Field summary

HEADER_BYTES The length of a header in bytes.

Constructor summary

MessageHeader()

Method summary

clone()getDatapathId() Gets datapath id.getMessageType() Gets message type.getModuleId() Gets module id.getNetIDEProtocolVersion() Gets net iDE protocol version.getPayloadLength() Gets payload length.getTransactionId() Gets transaction id.setDatapathId(long) Sets datapath id.setMessageType(MessageType) Sets message type.

NetIDEPublic

57



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



setModuleId(int) Sets module id.setNetIDEProtocolVersion(NetIDEProtocolVersion) Sets net iDE protocol ver-

sion.setPayloadLength(short) Sets payload length.setTransactionId(int) Sets transaction id.toByteRepresentation() Returns the header's byte representation.toString()

Fields

• public static final int HEADER_BYTES

� The length of a header in bytes.

Constructors

• MessageHeader

public MessageHeader ()

Methods

• clone

protected native java.lang.Object clone() throws

java.lang.CloneNotSupportedException

• getDatapathId

public long getDatapathId ()

� Description

Gets datapath id.

� Returns � the datapath id

• getMessageType

public MessageType getMessageType ()

� Description

Gets message type.

� Returns � the message type

• getModuleId

public int getModuleId ()

� Description

Gets module id.

NetIDEPublic

58



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Returns � the module id

• getNetIDEProtocolVersion

public NetIDEProtocolVersion getNetIDEProtocolVersion ()

� Description

Gets net iDE protocol version.

� Returns � the net iDE protocol version

• getPayloadLength

public short getPayloadLength ()

� Description

Gets payload length.

� Returns � the payload length

• getTransactionId

public int getTransactionId ()

� Description

Gets transaction id.

� Returns � the transaction id

• setDatapathId

public void setDatapathId(long datapathId)

� Description

Sets datapath id.

� Parameters

∗ datapathId � the datapath id

• setMessageType

public void setMessageType(MessageType messageType)

� Description

Sets message type.

� Parameters

∗ messageType � the message type

• setModuleId

public void setModuleId(int moduleId)

NetIDEPublic

59



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Description

Sets module id.

� Parameters

∗ moduleId � the module id

• setNetIDEProtocolVersion

public void setNetIDEProtocolVersion(NetIDEProtocolVersion

netIDEProtocolVersion)

� Description

Sets net iDE protocol version.

� Parameters

∗ netIDEProtocolVersion � the net iDE protocol version

• setPayloadLength

public void setPayloadLength(short payloadLength)

� Description

Sets payload length.

� Parameters

∗ payloadLength � the payload length

• setTransactionId

public void setTransactionId(int transactionId)

� Description

Sets transaction id.

� Parameters

∗ transactionId � the transaction id

• toByteRepresentation

public byte[] toByteRepresentation ()

� Description

Returns the header's byte representation.

� Returns � The byte representation.

• toString


NetIDEPublic

60



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



B.8 Class MessageType

Enumeration of MessageTypes according to the NetIP speci�cation.

Declaration

public final class MessageType

extends java.lang.Enum

Field summary

ERROR The ERROR type.FENCE FENCE messageHEARTBEAT The HEARTBEAT TypeHELLO The HELLO type.MANAGEMENT The MANAGEMENT type.MODULE_ACKNOWLEDGE The MODULE_ACKNOWLEDGE.MODULE_ANNOUNCEMENT The MODULE_ANNOUNCEMENT.NETCONF The NETCONF type.OPENFLOW The OPENFLOW type.OPFLEX The OPFLEX type.TOPOLOGY_UPDATE The TOPOLOGY_UPDATE.UNSUPPORTED The UNSUPPORTED type.

Method summary

getValue() Gets the byte value.parse(byte) Parse message type.valueOf(String)values()

Fields

• public static final MessageType HELLO

� The HELLO type.

• public static final MessageType ERROR

� The ERROR type.

• public static final MessageType MANAGEMENT

� The MANAGEMENT type.

• public static final MessageType MODULE_ANNOUNCEMENT

� The MODULE_ANNOUNCEMENT.

• public static final MessageType MODULE_ACKNOWLEDGE

� The MODULE_ACKNOWLEDGE.

• public static final MessageType HEARTBEAT

� The HEARTBEAT Type

• public static final MessageType TOPOLOGY_UPDATE

� The TOPOLOGY_UPDATE.

NetIDEPublic

61



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



• public static final MessageType OPENFLOW

� The OPENFLOW type.

• public static final MessageType NETCONF

� The NETCONF type.

• public static final MessageType OPFLEX

� The OPFLEX type.

• public static final MessageType FENCE

� FENCE message

• public static final MessageType UNSUPPORTED

� The UNSUPPORTED type.

Methods

• getValue

public byte getValue ()

� Description

Gets the byte value.

� Returns � the value

• parse

public static MessageType parse(byte value)

� Description

Parse message type.

� Parameters

∗ value � the value

� Returns � the message type

• valueOf

public static MessageType valueOf(java.lang.String name)

• values

public static MessageType [] values ()

Members inherited from class Enum

java.lang.Enum

clone, compareTo, equals, �nalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

NetIDEPublic

62



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



B.9 Class ModuleAcknowledgeMessage

Represents a ModuleAcknowledge message.

Declaration

public class ModuleAcknowledgeMessage


Constructor summary

ModuleAcknowledgeMessage() Creates a new instance of the ModuleAcknowledgeMes-sage class.

Method summary

getModuleName() Gets module name.getPayload()setModuleName(String) Sets module name.toString()

Constructors

• ModuleAcknowledgeMessage

public ModuleAcknowledgeMessage ()

� Description

Creates a new instance of the ModuleAcknowledgeMessage class.

Methods

• getModuleName

public java.lang.String getModuleName ()

� Description

Gets module name.

� Returns � the module name

• getPayload





NetIDEPublic

63



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



• setModuleName

public void setModuleName(java.lang.String moduleName)

� Description

Sets module name.

� Parameters

∗ moduleName � the module name

• toString





B.10 Class ModuleAnnouncementMessage

Represents a ModuleAnnouncement message..

Declaration

public class ModuleAnnouncementMessage


Constructor summary

ModuleAnnouncementMessage() Creates a new instance of the ModuleAnnounce-mentMessage class.

Method summary

getModuleName() Gets module name.getPayload()setModuleName(String) Sets module name.toString()

Constructors

• ModuleAnnouncementMessage

public ModuleAnnouncementMessage ()

� Description

Creates a new instance of the ModuleAnnouncementMessage class.

NetIDEPublic

64



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Methods

• getModuleName

public java.lang.String getModuleName ()

� Description

Gets module name.

� Returns � the module name

• getPayload





• setModuleName

public void setModuleName(java.lang.String moduleName)

� Description

Sets module name.

� Parameters

∗ moduleName � the module name

• toString





B.11 Class NetconfMessage

Class representing a message of type NETCONF. Note that this only serves as a convenience class- if the MessageType is manipulated, the class will not recognize that.

Declaration

public class NetconfMessage


Constructor summary

NetconfMessage() Instantiates a new Netconf message.

NetIDEPublic

65



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Method summary


Constructors

• NetconfMessage

public NetconfMessage ()

� Description

Instantiates a new Netconf message.

Methods

• getPayload





• setPayload


� Description

Sets payload.

� Parameters





B.12 Class NetIDEProtocolVersion

Enumeration of the NetIDE Protocol version bytes and convicence methods to parse the versionnumbers

NetIDEPublic

66



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Declaration

public final class NetIDEProtocolVersion


Field summary

VERSION_1_0 The Version 1.0VERSION_1_1 The Version 1.1VERSION_1_2 The Version 1.2VERSION_1_3 The Version 1.3VERSION_1_4 The Version 1.4

Method summary

getValue() Gets value.parse(byte) Parse net iDE protocol version.valueOf(String)values()

Fields

• public static final NetIDEProtocolVersion VERSION_1_0

� The Version 1.0


� The Version 1.1


� The Version 1.2


� The Version 1.3


� The Version 1.4

Methods

• getValue


� Description

Gets value.


• parse

public static NetIDEProtocolVersion parse(byte value)

� Description

Parse net iDE protocol version.

NetIDEPublic

67



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Parameters


� Returns � the net iDE protocol version

• valueOf

public static NetIDEProtocolVersion valueOf(java.lang.String name)

• values

public static NetIDEProtocolVersion [] values ()


java.lang.Enum


B.13 Class NetIPConverter

Class to parse raw messages and convert them to NetIDE Message

Declaration

public abstract class NetIPConverter


Constructor summary

NetIPConverter()

Method summary

parseConcreteMessage(byte[]) Parse concrete message.parseHeader(byte[]) Parse header.parseRawMessage(byte[]) Parse raw message.

Constructors

• NetIPConverter

public NetIPConverter ()

Methods

• parseConcreteMessage

public static Message parseConcreteMessage(byte[] data)

NetIDEPublic

68



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Description

Parse concrete message.

� Parameters

∗ data � the data

� Returns � the message

• parseHeader

public static MessageHeader parseHeader(byte[] data)

� Description

Parse header.

� Parameters


� Returns � the message header

• parseRawMessage

public static Message parseRawMessage(byte[] data)

� Description

Parse raw message.

� Parameters


� Returns � the message

B.14 Class NetIPUtils

Utility methods for handling NetIP messages.

Declaration

public abstract class NetIPUtils


Constructor summary

NetIPUtils()

Method summary

ConcretizeMessage(Message) Concretizes the given message to the correspondingconvenience class.

StubHeaderFromPayload(byte[]) Gets a stub header from the given payload withthe length correctly set and protocol version set to 1.1.

NetIDEPublic

69



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Constructors

• NetIPUtils

public NetIPUtils ()

Methods

• ConcretizeMessage

public static Message ConcretizeMessage(Message message)

� Description

Concretizes the given message to the corresponding convenience class.

� Parameters

∗ message � the message

� Returns � the concretized message (e.g. an instance of OpenFlowMessage)

• StubHeaderFromPayload

public static MessageHeader StubHeaderFromPayload(byte[] payload)

� Description

Gets a stub header from the given payload with the length correctly set and protocolversion set to 1.1.

� Parameters


� Returns � the message header

B.15 Class OpenFlowMessage

Class representing a message of type OPENFLOW. Note that this only serves as a convenience class- if the MessageType is manipulated, the class will not recognize that.

Declaration

public class OpenFlowMessage


Constructor summary

OpenFlowMessage() Instantiates a new Open �ow message.

Method summary

getOfMessage() Gets of message.getPayload()setOfMessage(OFMessage) Sets of message.toString()

NetIDEPublic

70



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Constructors

• OpenFlowMessage

public OpenFlowMessage ()

� Description

Instantiates a new Open �ow message.

Methods

• getOfMessage

public OFMessage getOfMessage ()

� Description

Gets of message.

� Returns � the of message

• getPayload





• setOfMessage

public void setOfMessage(OFMessage ofMessage)

� Description

Sets of message.

� Parameters

∗ ofMessage � the of message

• toString





NetIDEPublic

71



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



B.16 Class OpFlexMessage

Class representing a message of type OPFLEX. Note that this only serves as a convenience class -if the MessageType is manipulated, the class will not recognize that.

Declaration

public class OpFlexMessage


Constructor summary

OpFlexMessage() Instantiates a new Op �ex message.

Method summary


Constructors

• OpFlexMessage

public OpFlexMessage ()

� Description

Instantiates a new Op �ex message.

Methods

• getPayload





• setPayload


� Description

Sets payload.

� Parameters


NetIDEPublic

72



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle






B.17 Class Protocol

Enumeration of known protocols, mapped to their respective representation value, as speci�ed inthe NetIDE intermediate protocol speci�cation.

Declaration

public final class Protocol


Field summary

NETCONF The NETCONF protocol.OPENFLOW The OPENFLOW protocol.OPFLEX The OPFLEX protocol.

Method summary

getValue() Gets the value of this protocol in byte representations.parse(byte) Parse protocol.valueOf(String)values()

Fields

• public static final Protocol OPENFLOW

� The OPENFLOW protocol.

• public static final Protocol NETCONF

� The NETCONF protocol.

• public static final Protocol OPFLEX

� The OPFLEX protocol.

Methods

• getValue


� Description

Gets the value of this protocol in byte representations.


• parse

NetIDEPublic

73



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



public static Protocol parse(byte value)

� Description

Parse protocol.

� Parameters


� Returns � the protocol

• valueOf

public static Protocol valueOf(java.lang.String name)

• values

public static Protocol [] values ()


java.lang.Enum


B.18 Class ProtocolVersions

Enumeration of known protocol versions. Useful for Protocol to supportedVersion mappings.

Declaration

public final class ProtocolVersions


Field summary

NETCONF_1_0 The NETCONF_1_0.OPENFLOW_1_0 The OPENFLOW_1_0.OPENFLOW_1_1 The OPENFLOW_1_1.OPENFLOW_1_2 The OPENFLOW_1_2.OPENFLOW_1_3 The OPENFLOW_1_3.OPENFLOW_1_4 The OPENFLOW_1_4.OPFLEX_0_0 The OPFLEX_0_0.

Method summary

getProtocol() Gets the protocol.getValue() Gets value.parse(Protocol, byte) Parse protocol versions.valueOf(String)values()

NetIDEPublic

74



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



Fields

• public static final ProtocolVersions OPENFLOW_1_0

� The OPENFLOW_1_0.









• public static final ProtocolVersions NETCONF_1_0

� The NETCONF_1_0.

• public static final ProtocolVersions OPFLEX_0_0

� The OPFLEX_0_0.

Methods

• getProtocol

public Protocol getProtocol ()

� Description

Gets the protocol.

� Returns � the protocol

• getValue


� Description

Gets value.


• parse

public static ProtocolVersions parse(Protocol protocol ,byte value)

� Description

Parse protocol versions.

� Parameters


∗ protocol � Protocol to test against

NetIDEPublic

75



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



� Returns � the protocol versions

• valueOf

public static ProtocolVersions valueOf(java.lang.String name)

• values

public static ProtocolVersions [] values ()


java.lang.Enum


B.19 Class TopologyUpdateMessage

Message in NetIDE that signal an update of the Topology.

Declaration

public class TopologyUpdateMessage


Constructor summary

TopologyUpdateMessage() Instantiates a new Topology update message.

Method summary

getPayload()getTopology() Gets topology.setTopology(String) Sets topology.

Constructors

• TopologyUpdateMessage

public TopologyUpdateMessage ()

� Description

Instantiates a new Topology update message.

Methods

• getPayload


NetIDEPublic

76



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle






• getTopology

public java.lang.String getTopology ()

� Description

Gets topology.

� Returns � the topology

• setTopology

public void setTopology(java.lang.String topology)

� Description

Sets topology.

� Parameters

∗ topology � the topology




NetIDEPublic

77



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



C Bibliography

[1] The NetIDE Consortium. NetIDE deliverable 2.6 - NetIDE APIs and Integrated Platform v2,June 2016.

[2] The NetIDE Consortium. NetIDE deliverable 2.7 - NetIDE Manual, September 2016.

[3] OpenDaylight - A Linux Foundation Collaborative Project. http://www.opendaylight.org.

[4] ONOS - Open Network Operating System. http://onosproject.org/.

[5] Project Floodlight. http://www.projectfloodlight.org/floodlight/.

[6] Ryu SDN Framework. https://osrg.github.io/ryu/.

[7] Apache Karaf. http://karaf.apache.org/.

[8] OpenFlowJ Loxi. https://github.com/floodlight/loxigen/wiki/\OpenFlowJ-Loxi.

[9] OpenDaylight Open�owJava. https://github.com/opendaylight/\openflowjava.

[10] Python struct. https://docs.python.org/2/library/struct.html.

[11] The NetIDE Consortium. NetIDE deliverable 2.5 - NetIDE Runtime Architecture Consolida-tion, March 2016.

[12] NetIDE GitHub Repository. https://github.com/fp7-netide.

[13] The NetIDE Consortium. D4.1 - NetIDE Network Engine v1. Technical report, The EuropeanCommission, 2014.

[14] Christopher Monsanto, Joshua Reich, Nate Foster, Jennifer Rexford, and David Walker. Com-posing Software-de�ned Networks. In Proceedings of the 10th USENIX Conference on Net-worked Systems Design and Implementation, nsdi'13, pages 1�14, 2013.

[15] R. Doriguzzi-Corin, E. Salvadori, P. A. Aranda Gutiérrez, C. Stritzke, A. Leckey, K. Phemius,E. Rojas, and C. Guerrero. NetIDE: Removing vendor lock-in in SDN. In Network Softwariza-tion (NetSoft), 2015 1st IEEE Conference on, 2015.

[16] R. Doriguzzi-Corin, E. Salvadori, M. Gerola, and M. Santuari. An Approach to Exposing andSharing Network Services in Software-De�ned Networking. In ACM Sigcomm Symposium onSDN Research (SOSR), 2015.

[17] The NetIDE Consortium. NetIDE deliverable 2.3 - NetIDE IRF APIs Integrated Platform v1,Apr 2015.

[18] The NetIDE Consortium. NetIDE deliverable 2.4 - NetIDE Core concepts and Architecturev2, Sep 2015.

NetIDEPublic

78

http://www.opendaylight.org

http://onosproject.org/

http://www.projectfloodlight.org/floodlight/

https://osrg.github.io/ryu/

http://karaf.apache.org/

https://github.com/floodlight/loxigen/wiki/\OpenFlowJ-Loxi

https://github.com/opendaylight/\openflowjava

https://docs.python.org/2/library/struct.html

https://github.com/fp7-netide



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



[19] P. A. A. Gutiérrez, E. Rojas, A. Schwabe, C. Stritzke, R. Doriguzzi-Corin, A. Leckey, G. Pe-tralia, A. Marsico, K. Phemius, and S. Tamurejo. Netide: All-in-one framework for next gener-ation, composed sdn applications. In 2016 IEEE NetSoft Conference and Workshops (NetSoft),2016.

[20] P. A. A. Gutiérrez, T. Rasheed, C. Stritzke, R. Doriguzzi-Corin, and S. Tamurejo. NetIDE- Controller Independence supporting composed SDN Applications. http://www.eucnc.eu/

2016/www.eucnc.eu/indexe637.html?q=node/134, 2016.

[21] The NetIDE consortium. D5.1 - Use case requirements. Technical report, The EuropeanCommission, 2014.

[22] The NetIDE Consortium. D4.2 - Debugging Toolset 1st Release. Technical report, The Euro-pean Commission, 2015.

[23] The NetIDE consortium. D2.1 - NetIDE Architecture. Technical report, The European Com-mission, 2014.

[24] Wireshark network protocol analyzer. https://www.wireshark.org/.

[25] LIBPCAP - Link-Layer Header Types. http://www.tcpdump.org/linktypes.html.

[26] TCPDUMP/LIBPCAP public repository. http://www.tcpdump.org/.

[27] Wireshark - How to Dissect Anything. https://wiki.wireshark.org/

HowToDissectAnything.

[28] Wireshark - User DLTs protocol table. https://www.wireshark.org/docs/wsug_html_

chunked/ChUserDLTsSection.html.

[29] Wireshark - Adding a basic dissector. https://www.wireshark.org/docs/wsdg_html_

chunked/ChDissectAdd.html.

[30] Tra�c Generation Model. http://en.wikipedia.org/wiki/Traffic_generation_model.

[31] On Tra�c Patterns of HTTP Applications. https://hal.archives-ouvertes.fr/

hal-00685658/document.

[32] Publicly available PCAP �les. http://www.netresec.com/?page=PcapFiles.

[33] UMassTraceRepository. http://traces.cs.umass.edu/index.php/Network/Network.

[34] DCT2Gen. https://www-old.cs.uni-paderborn.de/en/research-group/

research-group-computer-networks/people/dr-philip-wette/dct2gen.html.

[35] Tcpreplay - Pcap editing and replaying utilities. http://tcpreplay.appneta.com/.

[36] Naga Katta, Omid Alipourfard, Jennifer Rexford, and David Walker. CacheFlow: Dependency-Aware Rule-Caching for Software-De�ned Networks. In Symposium on SDN Research, SantaClara, CA, USA, 14-15 March 2016.

[37] James M. Lucas, Michael S. Saccucci, Robert V. Baxley, Jr., William H. Woodall, Hazem D.Maragh, Fedrick W. Faltin, Gerald J. Hahn, William T. Tucker, J. Stuart Hunter, John F.MacGregor, and Thomas J. Harris. Exponentially weighted moving average control schemes:Properties and enhancements. Technometrics, 32(1):1�29, January 1990.

NetIDEPublic

79

http://www.eucnc.eu/2016/www.eucnc.eu/indexe637.html?q=node/134

http://www.eucnc.eu/2016/www.eucnc.eu/indexe637.html?q=node/134

https://www.wireshark.org/

http://www.tcpdump.org/linktypes.html

http://www.tcpdump.org/

https://wiki.wireshark.org/HowToDissectAnything

https://wiki.wireshark.org/HowToDissectAnything

https://www.wireshark.org/docs/wsug_html_chunked/ChUserDLTsSection.html

https://www.wireshark.org/docs/wsug_html_chunked/ChUserDLTsSection.html

https://www.wireshark.org/docs/wsdg_html_chunked/ChDissectAdd.html

https://www.wireshark.org/docs/wsdg_html_chunked/ChDissectAdd.html

http://en.wikipedia.org/wiki/Traffic_generation_model

https://hal.archives-ouvertes.fr/hal-00685658/document

https://hal.archives-ouvertes.fr/hal-00685658/document

http://www.netresec.com/?page=PcapFiles

http://traces.cs.umass.edu/index.php/Network/Network

https://www-old.cs.uni-paderborn.de/en/research-group/research-group-computer-networks/people/dr-philip-wette/dct2gen.html

https://www-old.cs.uni-paderborn.de/en/research-group/research-group-computer-networks/people/dr-philip-wette/dct2gen.html

http://tcpreplay.appneta.com/



Tel: +0461-408400 (1301) Fax: +39-0461-421157




!







lifecycle



[38] R. Doriguzzi-Corin, P. A. Aranda Gutiérrez, E. Rojas, H. Karl, and E. Salvadori. Reusabilityof Software-De�ned Networking Applications: A Runtime, Multi-Controller Approach. In 12thInternational Conference on Network and Service Management (CNSM), 2016.

[39] Roberto Doriguzzi-Corin, Pedro A Aranda Gutierrez, Elisa Rojas, Holger Karl, and Elio Sal-vadori. Reusability of software-de�ned networking applications: A runtime, multi-controllerapproach. In Network and Service Management (CNSM), 2016 12th International Conferenceon, pages 209�215. IEEE, 2016.

[40] Memory Management System source code. https://github.com/fp7-netide/Tools/tree/

master/memory-management-system/onos.

[41] NEC IP8800 OpenFlow Networking.

[42] Mininet Network Emulator. http://mininet.org/.

[43] The Open vSwith Project. http://openvswitch.org.

[44] Alberto Dainotti, Antonio Pescapé, Pierluigi Salvo Rossi, Francesco Palmieri, and GiorgioVentre. Internet tra�c modeling by means of Hidden Markov Models. Computer Networks,52(14):2645 � 2662, 2008.

[45] HP OpenFlow 1.3 Administrator Guide, June 2015. Rev. 2.

[46] ONOS ReactiveForwarding.

[47] Python NetIP library. https://github.com/fp7-netide/Engine/tree/master/libraries/

netip/python.

NetIDEPublic

80

https://github.com/fp7-netide/Tools/tree/master/memory-management-system/onos

https://github.com/fp7-netide/Tools/tree/master/memory-management-system/onos

http://mininet.org/

http://openvswitch.org

https://github.com/fp7-netide/Engine/tree/master/libraries/netip/python

https://github.com/fp7-netide/Engine/tree/master/libraries/netip/python

cordis.europa.eu€¦ · At A Glance: NetIDE An integrat ed development environment for portable...

Documents

Transcript of cordis.europa.eu€¦ · At A Glance: NetIDE An integrat ed development environment for portable...