1
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Project deliverable Project Number: Project Acronym: Project Title:
643275 FESTIVAL FEderated interoperable SmarT ICT services
deVelopment And testing pLatforms
Instrument: Thematic Priority
Research and Innovation action ICT
Title
D1.4 Requirements validated and final
architecture
architecture
Contractual Delivery Date: Actual Delivery Date:
April 1st 2016 April 23rd 2016
Start date of project: Duration:
October, 1st 2014 36 months
Organization name of lead contractor for this deliverable: Document version:
CEA V2.20
Dissemination level ( Project co-funded by the European Commission within the Seventh Framework Programme)
PU Public X
PP Restricted to other programme participants (including the Commission
RE Restricted to a group defined by the consortium (including the Commission)
CO Confidential, only for members of the consortium (including the Commission)
2
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Authors (organizations) :
Jander Botelho do Nascimento (CEA), Levent Gurgen (CEA); Martino Maggio, Giuseppe Ciulla (ENG); Juan
Ramon Santana (UC); Mengxuan Zhao (EGM); Toyokazu Akiyama (KSU); Sophie Vallet Chevillard (Inno);
Shuuichirou Murata (ACTUS)
Reviewers (organizations) :
Giuseppe CIULLA (ENG)
Abstract:
This deliverable presents the final architecture of the FESTIVAL federated testbeds platform. The architecture
has been designed by taking into account the unique requirements of such federated testbed environment
involving heterogeneous resources of various types (open data, IoT data, IT resources and living labs. The
deliverable provide guidelines to the technical development that will be performed in the WP2 in terms of
integration of the heterogeneous testbeds available within the project. The deliverable details the components
of the four layers of the architecture, from the resources to the portal layer, interactions among the
components, as well as detailed description of API to reserve, monitor and control resources within the
federated testbed.
Keywords : Federation architecture, IoT, gateway, interoperability, cloud, scalability, open data, experiment federation
Disclaimer
THIS DOCUMENT IS PROVIDED "AS IS" WITH NO WARRANTIES WHATSOEVER, INCLUDING ANY WARRANTY
OF MERCHANTABILITY, NONINFRINGEMENT, FITNESS FOR ANY PARTICULAR PURPOSE, OR ANY WARRANTY
OTHERWISE ARISING OUT OF ANY PROPOSAL, SPECIFICATION OR SAMPLE. Any liability, including liability for
infringement of any proprietary rights, relating to use of information in this document is disclaimed. No
license, express or implied, by estoppels or otherwise, to any intellectual property rights are granted herein.
The members of the project FESTIVAL do not accept any liability for actions or omissions of FESTIVAL
members or third parties and disclaims any obligation to enforce the use of this document. This document is
subject to change without notice.
3
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Acronyms and Definitions
Acronym Defined as
ACID Atomicity Coherence, Isolation and Durability
API Application Programming Interface
CA Certificate Authority
CNIL Commission Nationale de l'Informatique et des Libertés
(French National Commission of Informatics and Freedom)
DBMS Database Management Systems
EaaS Experimentation as a Service
FODP Federated Open Data Platform
GUI Graphical User Interface
IT Information Technology
IoT Internet of Things
JSON JavaScript Object Notation
KPI Key Performance Indicator
SFA Slice based Federation Architecture
URL Uniform Resource Locator
VM Virtual Machine
4
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Revision History
The following table describes the main changes done in the document since it was created.
Revision Date
Description Author (Organisation)
V0.1 25/02/2016 Add table of content and description of
the section
Jander Nascimento (CEA)
V0.2 15/03/2016 Driver API content Giuseppe Ciulla (ENG)
V0.3 17/03/2016 Sensinact Data format Jander Nascimento (CEA)
V0.4 18/03/2016 Updated Use cases and requirements Toyokazu Akiyama (KSU);
Shuuichirou Murata (ACTUS)
V0.5 18/03/2016 Federation quality monitoring Mengxuan Zhao (EGM)
V0.6 18/03/2016 Experimentation as a Service layer,
Experimentation workflow
Juan Ramon Santana (UC)
V0.7 25/03/2016 Typos, SFA sections, LL section, seq.
diagram in Generic Driver invoker, admin.
API controller
Giuseppe Ciulla (ENG)
V0.8 25/03/2016 Add multi-domain use case diagram,
update generic enabler definition
Toyokazu Akiyama (KSU);
Shuuichirou Murata (ACTUS)
V0.9 25/03/2016 Resource availability and service
availability corrections, image update,
adding QoE indicators
Mengxuah Zhao (EGM); Sophie
Vallet Chevillard (Inno)
V0.10 25/03/2016 Update gateway API mapping definition,
include storage service, indexes,
references, executive summary, abstract
Jander Nascimento (CEA);
Levent Gürgen (CEA)
V0.11 31/03/2016 Experiment workflow section Juan Ramon Santana (UC)
V0.12 31/03/2016 Conclusion section, experiment control
section
Jander Nascimento (CEA);
V0.13 31/03/2016 Review from ENG Giuseppe Ciulla (ENG)
V1.00 08/04/2016 Release candidate version Jander Nascimento (CEA);
V1.50 10/04/2016 Suggestion for improvements Giuseppe Ciulla (ENG);
V2.10 18/04/2016 Final version Jander Nascimento (CEA);
V2.20 23/04/2016 Final revision and submission Levent Gurgen (CEA)
5
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
The FESTIVAL Project
The development of the Internet of Things is set to have a strong impact on many aspects of society. Testbeds
and experimental facilities, both of small scale and up to city scale, will be an essential enabler to facilitate
the development of this vision. Facilitating the access to these test-beds to a large community of
experimenters approach is a key asset to the development of a large and active community of application
developers, necessary to address the many challenges faced by European and Japanese societies. FESTIVAL
project’s vision is to provide IoT experimentation platforms providing interaction facility with physical
environments and end-users, where experimenters can validate their Smart ICT service developments in
various domains such as smart city, smart building, smart public services, smart shopping, participatory
sensing, etc. FESTIVAL testbeds will connect cyber world to the physical world, from large scale deployments
at a city scale, to small platforms in lab environments and dedicated physical spaces simulating real-life
settings. Those platforms will be connected and federated via homogeneous access APIs with an
“Experimentation as a Service” (EaaS) model for experimenters to test their added value services. There have
been long years of research work in Europe and Japan on federation of testbeds and more recently on IoT
testbeds. FESTIVAL will as much as possible make reuse of existing software and hardware available in Europe
and in Japan for building such testbeds. Mutually applying European enablers for Japanese testbeds and vice
versa in real-life trials that the project will organize, FESTIVAL will bring have a strong impact in bridging the
gap among the aforementioned component technologies. FESTIVAL, as a common testbed infrastructure for
efficient communication and collaboration among stakeholders, will push European and Japanese IoT
testbeds and their practices one step beyond the current state of the art.
Participant
no. Participant organisation name
Participant
short name Country
1 (CO) Commissariat à l'énergie atomique et aux énergies alternatives –
Laboratoire d’électronique et des technologies de l’information CEA France
2 Universidad de Cantabria UC Spain
3 Engineering Ingegneria Informatica SpA ENG Italy
4 Easy Global Market EGM France
5 Inno TSD Inno France
6 Ayuntamiento de Santander SAN Spain
7 Sopra SOP France
8 (CO) Osaka University OSK Japan
9 Japan Research Institute for Social Systems JRISS Japan
10 Kyoto Sangyo University KSU Japan
11 Knowledge Capital Management Office KMO Japan
12 JR West Japan Communications JRC Japan
13 Ritsumeikan University Rits Japan
14 ACUTUS ACUTUS Japan
6
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Table of content
1. EXECUTIVE SUMMARY 14
2. FINAL FESTIVAL ARCHITECTURE 15
2.1. Architecture overview .......................................................................................................... 15
2.2. Uniform Access Layer ............................................................................................................ 17
2.2.1. Driver APIs ........................................................................................................................................... 17
2.2.1.1. Federated Open Data Platform Mapping 19
2.2.1.2. IoT Gateway Mapping 20
2.2.1.3. IT Resources Mapping 21
2.2.1.4. Living Lab Manager Mapping 23
2.2.2. Data model .......................................................................................................................................... 27
2.2.2.1. Actions of resources 33
2.3. Experimentation as a Service Layer ....................................................................................... 42
2.3.1. Resource Access Manager ................................................................................................................... 42
2.3.1.1. Resource Discovery 42
2.3.1.2. Resource Data Broker 43
2.3.1.3. Resource Directory 45
2.3.2. Experiment Monitoring ....................................................................................................................... 45
2.3.3. Experiment Control .............................................................................................................................. 46
2.3.3.1. Execution control 46
2.3.3.2. Failure control 47
2.3.3.3. Scheduling 48
2.3.4. Platform Administration ...................................................................................................................... 48
2.3.5. Storage Service .................................................................................................................................... 49
2.3.6. Analysis and Software Tools Repository .............................................................................................. 50
7
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.3.7. EaaS API Controller .............................................................................................................................. 50
2.3.7.1. EaaS APIs 51
2.3.7.2. Modules involved in execution of EaaS APIs 55
2.3.7.3. Administration API Controller 78
2.4. FESTIVAL Experimentation Portal .......................................................................................... 81
2.4.1. Experimenter home page and experiment definition ......................................................................... 81
2.4.2. Documentation .................................................................................................................................... 86
2.4.3. Living Lab Administration Portal .......................................................................................................... 88
2.4.4. Administration Portal .......................................................................................................................... 91
2.5. Security ................................................................................................................................ 93
2.5.1. General overview of the security architecture .................................................................................... 93
2.5.2. Authentication process ........................................................................................................................ 94
2.5.2.1. Single Sign-on 95
2.5.3. Authorization Process .......................................................................................................................... 96
2.5.4. Intra-layer communication .................................................................................................................. 97
2.5.5. End-to-end encryption support for experimenters ............................................................................. 99
3. EXPERIMENT WORKFLOW 100
3.1. Generic use case description ............................................................................................... 100
3.2. Boundary conditions ........................................................................................................... 100
3.3. Account creation and documentation .................................................................................. 101
3.4. Experiment definition ......................................................................................................... 102
3.5. Execution of the experiment ............................................................................................... 104
3.6. End of the experiment ........................................................................................................ 106
3.7. Template based experiment use case .................................................................................. 108
An implementation example ........................................................................................................ 109
4. FEDERATION QUALITY MONITORING 111
8
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
4.1. FESTIVAL Key Performance Indicators ................................................................................. 111
4.2. Collection of KPI data .......................................................................................................... 113
4.2.1. Automated KPI data collection .......................................................................................................... 113
4.2.1.1. Resource availability 114
4.2.1.2. Service availability and performance 114
4.2.1.3. Application/Service throughput 114
4.2.1.4. Application/Service reliability 115
4.2.2. KPI data collection from the portal.................................................................................................... 115
4.2.2.1. KPI Survey 115
4.2.2.1. Workflow for user feedback collection 116
5. CONCLUSION 119
6. REFERENCES 120
9
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
List of figures
Figure 1 The overall view of the FESTIVAL architecture .................................................................................. 14
Figure 2 FESTIVAL Architecture overview ....................................................................................................... 16
Figure 3 IT Resources Driver based on SFA ..................................................................................................... 22
Figure 4 Living Lab Manager Architecture ....................................................................................................... 24
Figure 5 FESTIVAL data model ......................................................................................................................... 27
Figure 6 Experiment control class diagram .................................................................................................... 47
Figure 7 Experiment resource state diagram ................................................................................................. 48
Figure 8 Data storage farm .............................................................................................................................. 49
Figure 9 API get testbeds architecture sub schema ........................................................................................ 56
Figure 10 API get testbeds sequence diagram ................................................................................................ 57
Figure 11 API get resources architecture sub schema .................................................................................... 58
Figure 12 API get resources sequence diagram .............................................................................................. 58
Figure 13 API get resource status sequence diagram ..................................................................................... 59
Figure 14 API get availability of resource architecture sub schema ............................................................... 60
Figure 15 API get availability of resource sequence diagram.......................................................................... 60
Figure 16 API lock resources sequence diagram ............................................................................................. 61
Figure 17 API unlock resources sequence diagram ......................................................................................... 62
Figure 18 API create experiment architecture sub schema ............................................................................ 63
Figure 19 API create experiment sequence diagram ...................................................................................... 64
Figure 20 API update experiment sequence diagram ..................................................................................... 66
Figure 21 API delete experiment sequence diagram ...................................................................................... 68
Figure 22 API start experiment architecture sub schema ............................................................................... 70
Figure 23 API start experiment sequence diagram ......................................................................................... 71
Figure 24 API stop experiment sequence diagram .......................................................................................... 73
10
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 25 API add resources to experiment architecture sub schema ........................................................... 74
Figure 26 API add resources to experiment sequence diagram ...................................................................... 75
Figure 27 API remove resources from experiment sequence diagram ........................................................... 77
Figure 28 Home Page, new experiment definition .......................................................................................... 82
Figure 29 Home page experiment details ....................................................................................................... 83
Figure 30 IoT resource selection view ............................................................................................................. 84
Figure 31 IT resources selection view .............................................................................................................. 85
Figure 32 Open Data resource selection ......................................................................................................... 86
Figure 33 IoT detailed information including RAML documentation of APIs .................................................. 87
Figure 34 IoT information page ....................................................................................................................... 88
Figure 35 Living lab general information and contact point ........................................................................... 89
Figure 36 Service and expertise information .................................................................................................. 90
Figure 37 Pending requests to the living lab manager .................................................................................... 91
Figure 38 Administration Portal, experiment view ......................................................................................... 92
Figure 39 Security sequence diagram ............................................................................................................. 94
Figure 40 Authentication Process .................................................................................................................... 95
Figure 41 Authorization Process ...................................................................................................................... 97
Figure 42 Architecture components within the VPN....................................................................................... 98
Figure 43 Example of the Encrypted HTTPS communication with FESTIVAL EaaS .......................................... 99
Figure 44 Account creation and access to documentation workflow ........................................................... 101
Figure 45 Experiment definition workflow .................................................................................................... 103
Figure 46 Execution of the experiment workflow ......................................................................................... 105
Figure 47 End of the experiment workflow ................................................................................................... 107
Figure 53 IT resource management use case on JOSE platform ................................................................... 110
Figure 48 Quality probe ................................................................................................................................. 113
11
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 49 Rapid feedback from experimenters ............................................................................................. 115
Figure 50 Account creation workflow .......................................................................................................... 116
Figure 51 Experiment definition and QoE workflow .................................................................................... 117
Figure 52 Workflow of the experiment with the feedback requests ............................................................ 118
12
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
List of tables
Table 1 Driver APIs ........................................................................................................................................... 19
Table 2 Mapping between Driver APIs and Federated Open Data Platform APIs .......................................... 20
Table 3 IoT Gateway API .................................................................................................................................. 21
Table 4 Mapping between Driver APIs and SFA APIs ...................................................................................... 23
Table 5 Mapping between Driver APIs and Living Lab Manager APIs ............................................................. 26
Table 6 Template for description of the entities of the FESTIVAL data model ............................................... 28
Table 7 Description of Experimenter entity .................................................................................................... 28
Table 8 Description of Experiment entity ........................................................................................................ 29
Table 9 Description of Resource entity ........................................................................................................... 30
Table 10 Description of Location entity .......................................................................................................... 31
Table 11 Description of Lock entity ................................................................................................................. 31
Table 12 Description of Testbed entity ........................................................................................................... 32
Table 13 Description of Contact Information entity ....................................................................................... 32
Table 14 Description of Federation entity ....................................................................................................... 33
Table 15 Model for representing actions on resources .................................................................................. 35
Table 16 Resource actions example - Open Data ............................................................................................ 36
Table 17 Resource actions example - IoT device ............................................................................................. 39
Table 18 Resource actions example - IT resource ........................................................................................... 41
Table 19 Resource Discovery API .................................................................................................................... 43
Table 20 Resource Data broker API ................................................................................................................. 44
Table 21 EaaS APIs ........................................................................................................................................... 55
Table 22 Administration APIs .......................................................................................................................... 80
Table 23 Access token request ........................................................................................................................ 96
Table 24 Token response ................................................................................................................................. 96
13
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Table 27 Multi-domain use case on JOSE ...................................................................................................... 109
Table 25 QoS indicators ................................................................................................................................. 111
Table 26 List of QoE indicators ...................................................................................................................... 112
14
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
1. Executive Summary
FESTIVAL project aims at providing a complete set of resources that experimenters can use to test and
validate their developments in the IoT and smart services domain. IoT is related to the physical world, thus
real-life conditions are essential to validate the IoT applications. The involvement of end-users are also of
tremendous importance to validate the quality of user experience. Considering these requirements, FESTIVAL
project brings together European and Japanese testbeds that allow access to various resources such as open
data (e.g., transportation data in the city), IoT (real-time air quality data in a city), IT (available processing
resources in a virtual machine) and living lab resources (surveys with real end-users). The objective is to allow
experimenters taking benefit of this rich set of European and Japanese resources via a unique tool accessing
this heterogeneous set of resources in a uniform and federated way. The experimenters will be able to use,
compose and extend those resources for validating their research in close to real-life conditions in a multi-
device, multi-protocol and multi-cultural environment.
Figure 1 The overall view of the FESTIVAL architecture
This document consolidates the initial FESTIVAL architecture defined in the Deliverable 1.3 [1]. The overall
view of the architecture is illustrated at the Figure 1. The architecture described in this document aims at
providing the blueprint to be used to build the federated FESTIVAL testbed. It specifies a common resource
data model and a set of uniform APIs that will be used by the experimenters to build and deploy rapidly and
efficiently their experiments. Thanks to the FESTIVAL's uniformed approach, the experiments will be portable
across several testbeds and replicable with minimum effort of adaptation.
15
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2. Final FESTIVAL Architecture
2.1. Architecture overview
An initial version of the architecture has been built during the first year of the project and presented
in "D1.3 First Architecture" [2]. In comparison to the initial version, the final architecture described in
this deliverable includes:
identification of new components;
more precise schematization of the EaaS layer components;
identification of relations between the components of the EaaS Layer;
Detail specifications of the Uniform Access Layer components;
more precise shaping of the Security components.
Figure 2 illustrates the FESTIVAL architecture, which is composed of 4 main layers:
- Uniform Access Layer aims at individually federating the same type of resources under a resource
type specific “driver APIs”. This layer is detailed in the Section 2.2.
- Experimentation as a Service Layer aims at federating the different resource types under a common
EaaS API. This layer is detailed in the Section 2.3. .
- Experimentation Portal Layer provides a unique point of access to the federated set of FESTIVAL
resources. This layer is detailed in the Section 2.4.
- Security Layer is transversal across all layers to provide an end-to-end secure access and control of
the FESTIVAL resources. This layer is detailed in the Section 2.5.
In addition to the layers of the architecture, this deliverable also describes an experimentation example
to illustrate the involvement of the architectural components within a typical experimentation workflow
(Section 3). A separate section on quality monitoring describes how the architecture foresees monitoring
of quality of service and quality of experience related parameters (Section 4).
16
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 2 FESTIVAL Architecture overview
17
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.2. Uniform Access Layer
Uniform Access Layer of the FESTIVAL architecture aims at unifying the access to resources (Open Data, IoT
devices, IT resources and living lab resources) hosted by the different testbeds federated in FESTIVAL.
Uniform access is provided through a set of APIs implemented by four “drivers” specific to each type of
testbed, namely Open Data Driver, IoT Gateway Driver, IT Resources Driver and Living Lab Driver. In addition,
a uniform FESTIVAL data model has been defined to provide the basic structure for shaping and representing
the fundamental resource information (resource description, capabilities, properties, etc.). Drivers APIs and
the FESTIVAL data model are presented in the sections 2.2.1 and 2.2.2 respectively.
2.2.1. Driver APIs
The four drivers of the Uniform Access Layer of the FESTIVAL architecture implements a common set
of APIs, which enable components of EaaS Layer to access resources using the same API independently
of the type of a specific resource. The common set of APIs is reported in Table 1.
API Description Input
Parameters
Output
Parameters
get testbeds Return information about available testbeds;
the method will accept in input the references
to requested testbeds (e.g.: identification
codes). If no references are provided, the
method will return information about all
available testbeds. The method will provide in
output the information about the requested
testbeds.
References to
testbeds.
Information about
requested
testbeds.
get resources Return the list of resources provided by
specific testbeds (one or more). The method
will return information about resources
obtained from testbeds specified in the
request. The method will accept in input two
parameters: the references to the testbeds
and the search options containing additional
values for filtering and ordering the list of
resources (e.g.: pagination, ascending or
descending order, keywords, etc...)
References to
testbeds.
Search options.
Information about
resources.
18
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
reserve
resources
The method performs the reservation and/or
instantiation of a set of resources (one or
more) in a specific range of dates (start and
end dates); the method will accept in input
five parameters: the reference to the
experiment that uses the resources, the
references to resources, the start date, the
end date of the reservation and optionally
additional information. The method will
return a message that in case of success
confirms the reservation of the resources and
in case of failure provides information about
the cause of the failure itself.
Reference to
the
experiment.
References to
resources.
Start date of
the
reservation.
End date of the
reservation.
Additional
information.
Message
release
resources
The method performs the release of a
reserved/instantiated resource; the method
will accept in input two parameters: the
reference to the experiment that uses the
resources and the references to resources to
be released. The method will return a
message that in case of success confirms the
release of the resources and in case of failure
provides information about the cause of the
failure itself.
Reference to
the
experiment.
References to
resources.
Message
get resource
status
The method verifies the status (e.g.:
AVAILABLE or NOT AVAILABLE) of a set of
resources (one or more); the method will
accept in input the references to the
resources. The method will return
information about the status of the requested
resources.
References to
resources.
Status of
resources.
subscribe The method executes the subscription to a
specific resource (e.g., for getting "live" data
from an IoT device, to get notified when a
resource is released); the method will accept
in input two parameters: the reference to the
resource and a call-back URL for getting the
data. The method will return a message that
in case of success confirms the subscription
References to
the resource.
Call-back URL.
Message
19
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
and in case of failure provides information
about the cause of the failure itself.
stop
subscription
The method terminates the subscription to a
specific resource; the method will accept in
input the reference to the resource and it will
return a message that in case of success
confirms the end of the subscription and in
case of failure provides information about the
cause of the failure itself.
References to
the resource.
Message
Table 1 Driver APIs
It is important to underline that even if each driver implements these set of APIs, this does not imply
that the implementation of every single API acts on resources; a concrete example of this particular
circumstance is given by “reserve resources” API that is not applicable on an Open Data, because an
Open Data is “open” and publicly available by definition [3] and cannot be reserved.
In sections 2.2.1.1, 2.2.1.2, 2.2.1.3 and 2.2.1.4 the mapping between the driver APIs and the
corresponding (or the possible corresponding) ones of each aggregator is reported.
2.2.1.1. Federated Open Data Platform Mapping
In this section, the possible mapping between driver APIs and APIs provided by Federated Open Data
Platform (FODP) is described (see Table 2).
FODP provides RESTful APIs; these are described in details in "D2.3 Open data guidelines and federated
testbed policy" [4].
Driver API FODP API Mapping Description
get testbeds REST method “/nodes” of
Administration APIs of FODP (get
federated Open Data Portals).
This API returns all federated Open Data
Portals.
get resources REST method “/search” of Client
Application APIs of FODP
(perform a search specifying as
search parameter a particular
federated open data portal).
This API executes a search of Open Data in
the federation of Open Data Portal
managed by FODP. It is possible to specify
different search parameters, including the
references to specific Open Data Portals
(testbeds). In that way it is possible to
retrieve all resources provided by a given
set of testbeds.
20
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
reserve resources Not Applicable This API is not applicable to FODP because
it is not possible to reserve/release an
Open Data.
release resources Not Applicable This API is not applicable to FODP because
it is not possible to reserve/release an
Open Data.
get resource status REST method “/search” of Client
Application APIs of FODP
(perform a search specifying as
search parameter a specific
Open Data).
This API checks the existence (the status)
of a specific Open Data. It executes a
search of Open Data in the federation of
Open Data Portal managed by FODP. It is
possible to specify in search parameters a
specific Open Data. In that way it is
possible to retrieve the specific Open Data
and to provide the confirmation of its
existence.
subscribe Not Applicable This API is not applicable to FODP because
it is not possible create a subscription to
an Open Data.
stop subscription Not Applicable This API is not applicable to FODP because
it is not possible create a subscription to
an Open Data.
Table 2 Mapping between Driver APIs and Federated Open Data Platform APIs
2.2.1.2. IoT Gateway Mapping
In the Table 3, method and the URL mapping of the IoT Gateway (sensinact platform) resources with
the respect to the RESTful API at Driver layer are explained. In the Driver API, mapping some of them
are mapped on the same RESTful service of sensiNact; an example is the mapping of “get resource
status” and “get resource”, those are two different call definitions in the API, but they point at the
same RESTful service in order to obtain the required information.
Driver API IoT Gateway API Mapping Description
get testbed getProviders Fetches the list source gateways that provided the data.
Since one given gateway (target gateway) can provide data
21
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
that is manage by remote gateways, it is important to
obtain which are the sources of the presented data.
get resources getResources Returns a list of resources (sensor, actioners) description
that are available in the target gateway
reserve resources reserveResources These methods has not yet implemented in the current
gateway implementation. However, sensiNact architecture
already takes into account authorization and access rights,
which will be extended to include such resource
reservation.
release resources releaseResources
get resource status getResources This is executed in order to retrieve all the information that
concerns a given resource including its status.
subscribe subscribe Returns a subscription ID, which allows the gateway to
notify the requester in case of update monitored resource.
This notification is done via a HTTP call from the gateway
to the subscription client
stop subscription unsubscribe Signal to the target gateway that a given client no longer
need to receive update notification. Be aware that the
client is unsubscribed once the server is not able to reach
the client to deliver a message.
Table 3 IoT Gateway API
2.2.1.3. IT Resources Mapping
IT Resource Driver, is the component in charge of managing the IT testbeds federated within FESTIVAL.
It is based on the Slice based Federation Architecture (SFA) [5], which provides specifications for
federating testbeds based on different technologies; its main concepts are:
Sliver: the IT resource that a user could create in a testbed;
Slice: a set of sliver that belongs to a particular testbed.
From a general point of view its implementations are based on three modules:
Registry: maintains a list of all slices belonging to the federated testbeds;
Aggregate Manager: it is in charge of managing the resources belonging to a single testbed.
This component should be implemented by each testbed; its main purpose is to expose SFA
API (e.g.: "list resources", "allocate", "provision", "status" and "delete"). Description of
resources is based on an xml-like format named RSpec (Resource Specification) [6];
22
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Slice Manager: It is in charge of communicating with the different Aggregate Managers and
managing the different slices and slivers of a testbed, in order to guarantee the federation of
heterogeneous testbeds.
A testbed that wants to federate within FESTIVAL has to implement an Aggregate Manager in order to
allow IT Resources Driver to request resources through SFA Wrapper and the Aggregate Manager's
APIs. For what concerns Aggregate Managers, a generic implementation is SFAWrap [7] that offers
wrappers for specific technologies on which a testbed is based, for example OpenStack [8].
The goal of the SFA Wrapper is :
to maintain the list of all the slices belonging to the federated testbeds.
to allow experimenters to know which resources are available in those testbeds .
to allow experimenters to create and use resources.
IT resources Driver enables FESTIVAL platform to interact with SFA Wrapper and with the underlying
testbeds. SFA Driver architecture is depicted in Figure 3.
Figure 3 IT Resources Driver based on SFA
23
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Driver API SFA API Mapping Description
get testbed get testbeds This API retrieves the list of the federated SFA
testbeds.
get resources list resources This API returns a list of the possible virtual machine
that a user can allocate in a federated testbed.
reserve resources allocate and provision This API allocates a virtual machine on behalf of the
authenticated user who issued the request. The user
has to choose between the possible virtual machine
retrieved by using “list resources” API. The allocated
resource has to be provisioned for being fully
accessible by the user.
release resources delete This API deletes the virtual machine previously
allocated by an authenticated user.
get resource status get status This API retrieves the status of a virtual machine. The
user, who is issuing the request, must have the
proper permission rights on the virtual machine.
subscribe Not applicable This API is not applicable because Aggregate
Managers does not expose a resource subscription
API.
stop subscription Not applicable It is not applicable because subscribe API is not
allowed.
Table 4 Mapping between Driver APIs and SFA APIs
2.2.1.4. Living Lab Manager Mapping
The Living Lab Manager is the component in charge of managing the living labs federated in FESTIVAL. Its
main functionalities are:
Aggregation of living labs involved in FESTIVAL project.
Managing the resources of these living labs for experiments.
Figure 4 shows the architecture of this aggregator:
24
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 4 Living Lab Manager Architecture
The main modules of the Living Lab Manager are:
Living Lab Administration
Living Lab Experiment Management
Communication Management
Living Lab Manager Repository
The Living Lab Administration module provides a set of APIs in order to allow the registration of Living
Labs in the federation. By means the Living Lab Administration Portal, the administrator of a Living
Lab can compile a registration form by inserting information that will be saved in the Living Lab
Manager Repository, such as:
General Details of Living Lab: name, description, location, etc.…
25
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Assets of Living Lab: services, expertise, methodologies, Living Lab communities and
stakeholder groups.
During the definition phase of an experiment, the experimenter can explore the Living Labs federated
in FESTIVAL and their assets, retrieving these data from the repository.
The Living Lab Experiment Management module is the component in charge of managing the request
from experimenter. The main tasks executed by means this module are:
Request from experimenter: the administrator of the Living Lab receives a request containing
all the data about the general information of the experiment (name, description, period, etc…)
and the specific activities that should be performed by the living lab, in terms of services,
methodologies and target of people to involve in the experiment.
Confirm participation: the administrator of the Living Lab can confirm the participation of the
Living Lab to the experiment by using the Living Lab Administration Portal. In this way, he
completes the planning process of the Living Lab resources in the experiment.
It is important to underline that, unlike the other aggregators (Federated Open Data Portal, sensiNact
and SFA), the interactions between the experimenter and the Living Labs have to be managed
differently because a Living Lab involves human resources, so the process cannot be completely
automated but it will be performed in an asynchronous way. Therefore, the Living Lab Experiment
Management takes into account the request for a new experiment, but all the interactions between
the administrator of the Living Lab and the experimenter, from the submission of the request and final
confirmation of the participation of the Living Lab to the experiment, are manually processed .
The way to execute this process can be completely managed by experimenter and by the administrator
of the Living Lab in a custom way (e.g. by mail, by phone, etc.). The Living Lab Manager supports both
the experimenter and the administrator of the Living Lab in each phase of the interaction process, so
for the communication process, the aggregator provides a tool for exchanging messages. The
Communication Management module provides the APIs to managing this functionality.
Table 5 shows the mapping between the LL Driver API and the LL Manager API:
Driver API Living Lab Manager
API
Mapping Description
get testbeds get livinglabs This API retrieves the list of living labs registered in
the Living Lab Manager Repository.
get resources get services
get expertise
get methodologies
In this case the list of resources is obtained by the
union of results provided by four APIs that retrieve
services, methodologies, expertise, and
communities/stakeholders of a Living Lab.
26
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
get people
reserve resources make request This API create the experimentation request for the
administrator of the Living Lab, including logistic
and technical information.
release resources stop collaboration This API notifies to the Living Lab Administrator that
its involvement in the experiment is terminated.
get resource status Not applicable In this case it is not possible automatically provides
status of the human resources and services of a
Living Lab.
subscribe Not applicable It is not possible to create a subscription to human
resources and services of a Living Lab.
subscribe stop
subscription
Not applicable It is not applicable because subscribe API is not
allowed.
Table 5 Mapping between Driver APIs and Living Lab Manager APIs
An example of workflow describing the interaction between an experimenter and an administrator of
Living Lab is the following:
1. Experimenter browses the Living Lab section of Festival portal and obtains an overview of all
living labs federated and their details. In this case, the Festival platform calls the getLivinglabs
API for retrieving the living labs and getServices, getExpertise, getMethodologies, getPeople
APIs for retrieving the main assets of each living lab.
2. Experimenter chooses the living lab and the assets to use in the experiment. Experimenter
sends a request to the administrator of the Living Lab including relevant information about the
experiment; a call to makeRequest API send a notification to Living Lab Manager component.
3. The administrator of the Living Lab views the request and starts a communication with the
experimenter in order to define various aspect of the involvement of the Living Lab. The actors
perform this process by choosing the most appropriate communication system, the one
offered by Festival Platform or others.
4. At the end of the communication process, the experimenter can start the experiment at the
scheduled date; during the execution of the experiment, interactions with the administrator
of the Living Lab.
5. At the end of the experiment, the interaction between experimenter and the administrator of
the Living Lab will finish with the final step in which the administrator communicates the
results of the performed activities (media, survey filled by end-users, report, etc.).
27
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.2.2. Data model
Starting from the first draft version of the data model of FESTIVAL entities presented in D1.3 [2], a new
updated version is presented in this section. Data model of FESTIVAL federation provides the basic structure
for shaping and representing the fundamental information and data that play a key role in FESTIVAL itself.
The data model consists of eight entities:
Experimenter
Experiment
Resource
Location
Lock
Testbed
Contact Information
Federation
These entities, their attributes and relations between them are depicted in Figure 5.
Figure 5 FESTIVAL data model
28
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
In the following, a more detailed description of each entities of the data model is reported; description of
entities is based on the structure reported in Table 6, which reports their basic information: name,
description, attributes, and relations with other entities of the data model.
Entity Name The name of the entity represented in the data model
Description General description of the entity represented in the data model
Attributes List of attributes of the entity and relative description
Relations Relations of the entity of the data model with other entities of the data model
Table 6 Template for description of the entities of the FESTIVAL data model
Entity Name Experimenter
Description This entity represents the final user of the FESTIVAL platform: the experimenter.
Experimenter uses functionalities offered by FESTIVAL for building (discovering and
collecting resources) and running (execution, monitoring and analysis) their experiment.
Attributes id The unique identifier of the experimenter
name The name of the experimenter
email The email address of the experimenter
role The role assigned to the experimenter: the
role identifies his/her privileges in relation
to functionalities offered by FESTIVAL
platform (e.g.: the maximum number of
experiment he/she can run in a month). The
role of an experimenter can assume one of
the following two values:
Basic
Expert
Relations Collaborates with An experimenter can collaborate with zero
or more other experimenters for managing
experiments.
Manages An experimenter can manage zero or more
experiments.
Table 7 Description of Experimenter entity
29
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Entity Name Experiment
Description This entity represents the main concept of FESTIVAL: experiments designed and conducted
by experimenters through the use of functionalities provided by FESTIVAL platform.
Attributes id The unique identifier of the experiment.
title The title of the experiment.
creation date The date in which the experiment is created.
start date The date in which the experiment is planned
to start.
end date The date in which the experiment is planned
to end.
description The textual description of the experiment
provided by the experiments that defines
the experiment.
status The status of the experiment: the status of
the experiment can assume one of the
following four values:
Created
Running
Stopped
Error
Relations involves An experiment can involve zero or more
resources in its execution.
has An experiment has zero or more Lock
information for locking resources needed
for its execution.
Table 8 Description of Experiment entity
30
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Entity Name Resource
Description This entity represents the resources provided by testbeds federated in FESTIVAL, which can
be used by experimenters for building their experiments.
Attributes id The unique identifier of the resource.
title The title of the resource
description The textual description of the resource as
provided by the testbed that owns it.
type The type of the resource: the type can
assume one of the following four values:
Open Data
IoT
IT
Living Lab
resource URL The URL at which it is possible to retrieve
the resource or it is possible to interact with
it.
availability
The availability status of the resource: the
availability can assume one of the following
values:
Available
Not available
Not working
actions
The list of actions that an experimenter can
perform on the resource (for more details
refer to section 2.2.2.1).
Relations has A resource can be associated from zero to
one location that specifies its geographic
position.
Table 9 Description of Resource entity
31
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Entity Name Location
Description This entity represents a geographic location of a testbed or of a specific resource.
Attributes latitude The north–south position of the point on
the Earth’s surface locating a testbed or a
resource.
longitude The east–west position of the point on the
Earth’s surface locating a testbed or a
resource.
Relations
Table 10 Description of Location entity
Entity Name Lock
Description This entity indicates a period in which a resource is locked for an experiment. This
information is managed only at federation level; it is not related to the testbeds.
Attributes start date The start date of the lock period of a
resource.
end date The end date of the lock period of a
resource.
Relations defines the lock period of A lock is related to only one resource; on the
other side, a resource can be associated to
different locks. Each lock defines a different
lock period.
Table 11 Description of Lock entity
Entity Name Testbed
Description This entity represents the testbed federated in FESTIVAL: testbeds provide their resources
enabling experimenters to design, build and run their experiment.
Attributes id The unique identifier of the testbed.
title The title of the testbed.
32
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
description The textual description of the testbed.
type The type of the testbed: the type can
assume one of the following four values:
Open Data
IoT
IT
Living Lab
availability The availability status of the testbed: the
availability can assume one of the following
values:
Available
Not available
Not working
URL The URL at which it is possible to retrieve
more information about the testbed (e.g.: a
web page).
Relations has A testbed is associated exactly to one
contact information.
provides A testbed can provide zero or more
resources.
Table 12 Description of Testbed entity
Entity Name Contact Information
Description This entity represents contact information (email, phone number, etc…) for contacting a
testbed.
Attributes email The email address for contacting a testbed.
phone number The phone number for contacting the
testbed.
contact person The name of the person in charge of
representing the testbed.
Relations
Table 13 Description of Contact Information entity
33
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Entity Name Federation
Description This entity represents the FESTIVAL as a whole.
Attributes title The title of the federation (FESTIVAL).
description The textual description of the federation.
Relations has A federation is associated to zero or more
testbeds.
Table 14 Description of Federation entity
2.2.2.1. Actions of resources
In this section the model for representing the actions those can be performed on resources is introduced and
described.
Possible actions that can be performed on a resource (“actions” attributes of the Resources entity described
in Table 9) should be described using a specific format in order to provide a unique method to represent
them and facilitate experimenters to execute those actions.
From a general point of view the model (reported in Table 15) for representing possible actions is based on
JSON. The model is derived from the JSON representation for REST services provided by JSDL (JSON Service
Description Language) [9] and by OpenAPI Specification [10].
34
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
{
"info": { //this field provides general information about the document
"title": “title of the specific model, e.g.: Actions on sensor #10”,
"description": "description of the specific model, e.g.: this document describe the possible actions, and relative URL, that can be performed on sensor #10",
"termsOfService": "URL to the web page that provides information about the term s of service about the described actions",
"contact": { //this field provides information about the person or the company that provide the resource and its relative actions
"name": " name of the contact person or of the company that manages the resource",
"email": "email address of the contact person or of the company that manages the resource ",
"url": "URL to the web page of the contact person or of the company that manages the resource"
},
"license": { //this field provides information about the l icense under the which the documents published
"name": "name of the license under the which the documents published",
"url": " URL to the web page reporting the complete text of the license"
}
},
"host": " host name for executing the actions reported in the document, e.g.: mytestbed.org",
"basePath": "base path for executing the actions reported in the document, e.g.: /mysensors",
"schemes": [ //protocol that can be used for executing the actions, e.g.: http ],
"consumes": [ //data format accepted in input by the actions, e.g.: application/json],
"produces": [ //format of the data produced by actions, e.g.: application/json],
"paths": { //this field provides information about the specific actions that can be performed on the resource; each reported path represents an action and its relative path for its execution
"/path/to/the/action/": { //a specific action and its path
"method of the action": {
"description": "Returns volume",
35
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
"operationId": "the identification code the action, e.g.: action#1_of_sensor#10",
"responses": { //this field describes the possible responses of the action, e.g.: correct execution, error, etc…
"response": { //this field describes a specific response
"description": "description of the specific response",
"schema": { //this field describes the schema of the data provided by the action in the response; alternatively, this filed can contain a reference to a schema contained in the same document},
"examples": { //this field contains an example of the possible data provided by the action in the response
"data format": "this filed indicated the data format of the response data, e.g.: application/json"
}
}
}
},
"definitions": { //this is an optional field containing the schema of the data that can be provided in input to the actions and of the data that the actions can provide in output in the respons es
}
}
Table 15 Model for representing actions on resources
On the basis of the generic model reported in Table 9 three different examples are provided for describing
actions on Open Data, IoT devices and IT resources; in this context it is not possible to describe actions on
resources those concern Living Labs because Living Labs envisage that interactions between them (and their
resources) and experiments should be conducted manually and through a contact person of the Living Lab.
Open Data
In the case of an Open Data, the only actions those can be performed are the retrieving (download) of files
associated to the Open Data itself.
An example of document describing this kind of action is reported in Table 16.
In this specific case, the schema of input and output data could be not necessary, because the “download”
actions can provide a generic file.
36
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
{
"info": {
"title": "Dataset ",
"description": "A sample API that retrieve a dataset",
"termsOfService": "http://od-platform.eng.it/terms/",
"contact": {
"name": "FOD Platform Team",
"email": "[email protected]",
"url": "http://od-platform.eng.it/"},
"license": {
"name": "MIT",
"url": "http://github.com/gruntjs/grunt/blob/master/LICENSE-MIT"}},
"host": "od-platform.eng.it",
"basePath": "/",
"schemes": ["http" ],
"produces": ["text/csv"],
"paths": {
"/download": {
"get": {
"description": "Download a CSV file",
"operationId": "getCSV",
"responses": {
"200": {
"description": "csv file",
"schema": {"type": "file" }}}}}}}
Table 16 Resource actions example - Open Data
37
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
IoT device
In the case of an IoT device, such as a sensor or an actuator, the example presented in Table 17 reports the
type of data accept in input and provided in output by the actions (in the specific case of the example these
are “application/json”) and their relative schemas reported in the “definitions” filed.
{
"info": {
"title": "Volume Resource",
"description": "A sample API that manage the volume of media player",
"termsOfService": "http://sensinact/terms/",
"contact": {
"name": "sensiNact Team",
"email": "[email protected]",
"url": "http://sensinact.com"},
"license": {
"name": "MIT",
"url": "http://github.com/gruntjs/grunt/blob/master/LICENSE-MIT"}},
"host": "sensinact.com",
"basePath": "/sensinact",
"schemes": ["http"],
"consumes": ["application/json"],
"produces": ["application/json"],
"paths": {
"/providers/sensor_10/services/sensor/resources/media_player/GET": {
"get": {
"description": "Returns volume",
"operationId": "getVolume",
"responses": {
38
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
"200": {
"description": "volume value",
"schema": {"$ref": "#/definitions/Response"},
"examples": {"application/json": "...."}}}},
"/providers/sensor_10/services/sensor/resources/media_player/SET": {
"post": {
"description": "Set the volume of the media player",
"operationId": "setVolume",
"parameters": [{
"name": "volume",
"in": "body",
"description": "value of the volume",
"required": true,
"schema": {"$ref": "#/definitions/Payload"},
"examples": {"application/json": "...."}}],
"responses": {
"200": {
"description": "volume response",
"schema": {"$ref": "#/definitions/Response"},
"examples": {"application/json": "...."}}}}}},
"definitions": {
"Payload": {
"type": "object",
"properties": {
"parameters": {
"type": "array",
"items": {"$ref": "#/definitions/PayloadData"}}}},
39
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
"Response": {
"type": "object",
"required": [ "name", "timestamp", "type", "value" ],
"properties": {
"name": {"type": "string"},
"type": {"type": "string"},
"value": {"type": "number", "format": "double"},
"timestamp": {"type": "string", "format": "date"}}},
"PayloadData": {
"type": "object",
"required": [ "name", "type", "value" ],
"properties": {
"name": {"type": "string"},
"type": {"type": "string"},
"value": {"type": "number", "format": "double"}}}}}}
Table 17 Resource actions example - IoT device
IT resource
The example for IT resources reported in Table 18 is quite similar to the example for IoT devices reported in
Table 17.
40
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
{
"info": {
"title": "VM Management",
"description": "A sample API that manage start, restart and stop of a VM",
"termsOfService": "http://itmanager/terms/",
"contact": {
"name": "IT Manager Team",
"email": "[email protected]",
"url": "http://itmanager.com"},
"license": {
"name": "MIT",
"url": "http://github.com/gruntjs/grunt/blob/master/LICENSE-MIT" }},
"host": "itmanager.com",
"basePath": "/itmanager",
"schemes": ["http"],
"consumes": ["application/xml"],
"produces": ["application/xml"],
"paths": {
"/action": {
"post": {
"description": "Perform an action a VM (start/stop/restart)",
"operationId": "actionVM",
"parameters": [{
"name": "Payload",
"in": "body",
"description": "action object",
"required": true,
41
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
"schema":{"$ref": "#/definitions/Payload"},
"examples": {"application/json": "...."}} } ],
"responses": {
"200": {
"description": "volume response",
"schema": {"$ref": "#/definitions/Response"},
"examples": {"application/json": "...."} }}}}},
"definitions": {
"Payload": {
"type": "object",
"required": ["URN", "action", "credential"],
"properties": {
"URN": {"type": "string"},
"action": {"type": "string"},
"credential": {"$ref": "#/definitions/Credential"}}},
"Credential": {
"type": "object",
"required": ["type", "value", "version"],
"properties": {
"type": {"type": "string"},
"version": {"type": "string"},
"value": {"type": "string"}}}}}
Table 18 Resource actions example - IT resource
42
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.3. Experimentation as a Service Layer
This section contains the description of the components of the EaaS layer, which will provide the
functionalities of the FESTIVAL platform to perform new experiments. In Deliverable D1.3 [1], the basic
functionalities of these modules from a logical point of view are explained. In this section the modules
with the corresponding changes due to the modification of the architecture are described.
Additionally, every module provides a set of functionalities that are exposed to other modules
deployed into EaaS platform.
Every subsection includes the description of a module within the architecture, as well as the
considered functionalities and the relations with the other modules. The functionalities are described
from a logical point of view, and their implementation will correspond to the Work Package 2 and will
be reported in the forthcoming Work Package 2 deliverables.
2.3.1. Resource Access Manager
This module is basically in charge of providing the access to the resources. The module provides the
logic to make the corresponding requests to the different drivers in order to process them. This
component can be divided in three different modules that provide the corresponding functionalities.
2.3.1.1. Resource Discovery
The Resource discovery module is in charge of requesting available resources to the testbeds. The response
of the Resource Discovery will be the existing FESTIVAL resources in the aggregators using the driver API. The
experimenter that wants to access to the resources will ask to this module which are the available resources
in FESTIVAL through the EaaS API controller, providing the security check and the logic for managing the
driver API output.
This module is linked with the Resource Directory module as it will serve as cache for existing resources.
FESTIVAL does not manage the resources directly (this task belongs to the aggregators and to the testbeds).
As commented in D1.3 [1], the Resource Directory will return the description of the resources using the
FESTIVAL model (section 2.2.2). Additionally, the Resource Discovery returns the available methods to access
the resources depending on the experimenter rights. It also provides search capabilities to discover resources
depending on different parameters (e.g. sensor type, machine power, etc).
The Resource Discovery component provides all the logic to discover existing resources to the experimenter.
Using this component, the experimenter will be able to search through the offered resources in the FESTIVAL
federation so as to plan his experiment. In that sense, the Resource Discovery will expose the description of
the different resources following the FESTIVAL model, thus providing a homogenous format for resource
description.
Additionally, this component supports different searching possibilities, in order to look for the resources
depending on their capabilities (e.g.: processing power in VMs or sensor types). Within this description, the
43
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
services exposed by the resources are also included, so the experimenter will be able to know the methods
to access such resources.
Functionality Description Relations
getAvailableResources()
Get the available
resources from a
specific testbed.
ResourceDirectory
Returns the “cached”
FESTIVAL resources
from the drivers.
Security
Returns the access
rights of the
experimenter.
getAvailableResourcesBy()
Get the available
resources by one of
the available types
(Open Data, Iot, IT
and Living Lab) with a
condition (e.g. sensor
type, processing
capabilities, etc)
ResourceDirectory
Returns the “cached”
FESTIVAL resources
from the drivers
Security
Returns the access
rights of the
experimenter
getServicesFrom()
Return the services
to access the values
of the resources
ResourceDirectory
Returns the “cached”
FESTIVAL resources
from the drivers
ResourceDataBroker
Returns the available
services for the
specified sensors
Security
Returns the access
rights of the
experimenter
Table 19 Resource Discovery API
2.3.1.2. Resource Data Broker
This module provides the appropriate interfaces to access the data from the aggregators. It will provide the
functionality to subscribe to the appropriate resource from the drivers. Additionally, other calls depending
on the sensors will be also addressed by the Resource Data Broker. The rationale behind using this module
instead of accessing directly to the drivers is to provide the security check against the security module in
FESTIVAL.
44
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
From an experimenter point of view, he/she will be able to subscribe to a specific sensor to get near real-
time updates through this module. Additionally, he could access to the last value or a set of historical values
from a specific sensor.
Functionality Description Relations
subscribeTo() Subscribe to the
specified resource
Driver API
Perform a
subscription query to
the driver API
Security
Returns the access
rights of the
experimenter
unsubscribeFrom()
Unsubscribe (in order
to perform this
query, the id of the
subscription must be
given)
Driver API
Perform an
unsubscription query
to the driver API
Security
Returns the access
rights of the
experimenter
getSubscriptionDetails()
Get the details of the
subscription (given an
id, returns the
sensor)
Driver API Returns the details of
the subscription
Security
Returns the access
rights of the
experimenter
getSensorHistoricalData()
Get the historical
data from an specific
sensor in a given
period of time
Driver API Returns the details of
the subscription
Security
Returns the access
rights of the
experimenter
getSensorLastValueData()
Return the last value
from an specific
sensor
Driver API Returns the details of
the subscription
Security
Returns the access
rights of the
experimenter
Table 20 Resource Data broker API
45
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.3.1.3. Resource Directory
Finally, the third main module of the Resource Manager is the Resource Directory. This module provides the
set of resources existing in FESTIVAL that are available to be used. The Resource Directory serves as a
database with the cache of the FESTIVAL resources.
The description of the resources is explained in section 2.2.2. The format used in the Resource Directory is
the generic FESTIVAL resource. This module will be used to keep the logic between the locked resources and
the reserved resources, in order to not overpass the other modules. It will work as follows: when a new
resource is going to be locked, the Resource Directory, using the FESTIVAL Storage Service module, will check
if the resource is already locked or not. If it is not the case, the resource will be locked for exclusive use of
the experimenter. Additionally, the module will serve the data of existing resources to the Resource
Discovery, in order to avoid as much as possible redundant requests to the drivers and the testbeds. This is
because the existing resources in the testbed are permanent in most of the cases, and the queries to the
testbeds are not needed continuously.
2.3.2. Experiment Monitoring
The Experiment Monitoring module will provide to the experimenter the possibility of monitor the
experiments being performed in the platform. This includes not only the monitoring of the experiment
itself but also the resources that are being in use.
The monitoring of the experiment is divided in two different functionalities:
Monitoring the status of the reserved resources.
So as to monitor the reserved resources, the platform will access to the last value received
from the sensors in the experiment. Additionally, a process in background will measure certain
parameters such as the mean time between measured sensor values or if there was any
problem with the network (e.g.: if the instantiated machines were not accessible for a certain
period of time).
In order to provide this functionality, the module will access to the resource manager upon
request, when the experimenter requests the status of the resources and testbeds.
Additionally, for each of the experiments, there will be a service running that will be in charge
of requesting the status of the resources associated to each of the experiments. If any of the
resources is not accessible for any reason, the module will keep track of the resources and will
store the time that the resource was down. It is also possible to send an alert to the
experimenter if any of the resources was down for a big period of time.
Monitoring an experiment by providing a set of services to be implemented by the
experimenter.
46
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
This is related to the monitoring of deployed services and applications by the experimenter.
The experimenter will have the opportunity to access a set of services and libraries to perform
frequent updates of the status of an experiment. These updates will be sent as a log to a
centralized service in FESTIVAL, where the experimenter will be able to access with his/her
credentials.
The implementation of this functionality will depend on the implementation of certain libraries
or functionalities by the experimenter. The basic workflow will be as follows:
1. The deployed service implements a library to send measurements or logs to an
endpoint provided by FESTIVAL. The endpoint will depend on the technology used that
is not yet defined. There are several technologies available, such as OML [11] or a
simple REST interface with a predefined format in JSON.
2. FESTIVAL will gather all the data coming from this interface and will store it in the
storage service.
The experimenter will be able to access his/her data later, through the corresponding interfaces. Data
will be possibly processed with existing tools such as Kibana [12].
2.3.3. Experiment Control
Experiment Control is responsible for taking care of all activities linked to the experiment execution.
Some of its responsibilities are execution control, resource allocation, access and failure control. This
module is located into EaaS layer as a core service for the experimenter as a backend of the experiment
control user interface.
This section will explain mainly two subparts of this component, which are execution control and
failure control. Execution handles resource allocation and access control as well, thus most part of this
component will be described in the section designated as such.
2.3.3.1. Execution control
Due to the core role played by the experiment control this component is subject to evolution and
improvements throughout its own execution. Thus, internal quality measurements will be taken into
this component in order to allow runtime improvement on the actions taken by such component.
Since the actions taken across this component are fundamental, all of them are logged into a human
readable format containing essentially, but not exclusively: the user responsible for the action, the
timestamp for the execution, action requested and the IP from which the execution was requested.
47
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 6 Experiment control class diagram
As central part of the architecture, the experiment control requires to invoke some existing components in
order to obtain some information that is required to compose the experiment.
2.3.3.2. Failure control
It is necessary to address the possible failure during the entire experiment life-cycle. Before establishing a
failure control mechanism on the experiment, a state diagram is defined for the resources allocated in the
experiment. This state diagram is depicted in the Figure 7.
Every transition in the depicted state diagram generates an ActionLog record. ActionLog object is a persistent
object that records the actions that produced state change in the resource. Since this record is persisted, the
Storage Service will be used in order to do so.
48
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 7 Experiment resource state diagram
2.3.3.3. Scheduling
Some experiment can specify certain resource to be in exclusive access, meaning that those resources will
not be available to other experiment while the experiment is in execution, or in a state equivalent to it.
The initial scheduling algorithm adopted is First-Come-First-Served; this algorithm must be decoupled
enough to be replaced by any other algorithm in the future.
Thus, every resource has its own timeline, and there are not overlap areas in their timeline. Resources that
can be shared, except for the experiments that explicitly its exclusive access.
2.3.4. Platform Administration
The platform administration module is in charge of providing the specific functionalities to manage
the platform. This module will have access to all the experiments being carried out in FESTIVAL, as
well as the actions performed within the framework of such experiments. The access to this module
will be exclusive for the administrators and the users will not have any rights or APIs to do so.
Some functionalities provided by this module are:
49
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
o Management of given roles to the platform experimenters.
o Management of the existing experiments: delete the experiments or modify its duration.
o Etc..
2.3.5. Storage Service
The purpose of Storage Service is to normalize the persistent data access. Thus, all components that have
access to such service can persist information in the platform. This data can be stored and retrieved in
different fashions: either structured or non-structured.
As structured data storage the back-end side will be handled by a database compliant with the specification
SQL-2011 [13]. This model follows the conventional structured SQL specification and is submitted to the
normalization form well known in the database field, with a well-known syntax to retrieve the stored data.
The structured-data farm follows the ACID [14] properties with respect to its data.
Non-structured storage is, usually but not exclusively, key-value information stored in a NoSQL backend. In
which the normalization rules are not applicable; the implementation engine running in the back-end side
will be defined later on.
The goals of this service are:
Simplify the access to persistent service.
Simplifying auditing on stored information (the central data storage ease the task of auditing data in
case of a formal verification by the concerned organism, e.g. CNIL)
Ensure the backend database available.
Ease maintainability.
This storage mechanism is an assistance service that allows EaaS components to store and manage their own
data without caring about deploy a Database Management System (DBMS).
Figure 8 Data storage farm
50
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
In the Figure 8 it is possible to see that the storage service handles structured and non-structured data in
order to allow used the storage service to support various kind of purpose.
Furthermore, a third part of the Data storage is represented by System data. The System data farm contains
the information that may be used (shared) by different components in the platform. This non-isolated data
will accessible by components located in any of the layers in the EaaS architecture. KPI information is an
example of shared data, it will be stored in this System data farm, and by consequence it can be used by
other components.
The three different segments will be handled separately, with no data linkage at any level.
2.3.6. Analysis and Software Tools Repository
The Analysis and Software Tools Repository is a component to provide additional features to the
experimenter using the EaaS platform.
This module provides a repository of several software tools that can be useful to make some
experiments, combining different resources from FESTIVAL. This repository will be accessible through
the FESTIVAL Platform and will contain existing open source libraries and applications, and the
software developed within FESTIVAL that might be useful to analyze data gathered from sensors.
Furthermore, the applications that are provided by the repository are also provided, depending on the
testbed, as predefined templates. This means to instantiate the reserved VMs with the applications
already installed and ready to be used. As an example of this, we can consider an experimenter that
want to test and try a Generic Enabler from FIWARE. In FESTIVAL, the experimenter will be able either
to directly download the GE from the repository or reserve a VM with the GE preinstalled.
Moreover this module provides analysis tools to the experimenter. Experimenter will have a service
storage capability to store logs from the deployed experiments (section 2.3.5). In order to process
these logs and gather important information from them, this module will provide the tools to analyse
the logs and present them in a user-friendly environment to check the status of the experiment.
2.3.7. EaaS API Controller
EaaS API Controller provides functionalities for enabling FESTIVAL Experimentation Portal (described in
section 2.4. ) to interact with the underlying modules of FESTIVAL platform. EaaS APIs are described in section
2.3.7.1, while section 2.3.7.2 provides details about modules of FESTIVAL architecture involved in the
execution of EaaS API. EaaS API Controller includes a sub module devoted to the administration of the
FESTIVAL platform: Administration API Controller. This sub module of EaaS API controller provides a special
set of APIs accessible only by users with administration rights. These APIs enables administrators to:
manage the platform configuration.
check performances of the federations through the definition and monitoring of KPIs (Key
Performance Indicator).
51
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
manage registered experimenters (e.g.: role assignments).
etc…
Administration API Controller and relative APIs are detailed in section 2.3.7.3.
2.3.7.1. EaaS APIs
This section reports in Table 21 the summary specification of EaaS API; for each API the relative description
and input/output parameters are reported.
API Description Input
Parameters
Output Parameters
get testbeds Return information about available
testbeds; the method will accept in input
the references to requested testbeds (e.g.:
identification codes). If no references are
provided, the method will return
information about all available testbeds.
The method will provide in output the
information about the requested testbeds.
References to
testbeds.
Information about
requested testbeds.
get resources Return the list of resources provided by
specific testbeds (one or more). The
method will return information about
resources obtained from testbeds
specified in the request. The method will
accept in input two parameters: the
references to the testbeds and the search
options containing additional values for
filtering and ordering the list of resources
(e.g.: pagination, ascending or descending
order, keywords, etc...)
References to
testbeds.
Search
options.
Information about
resources.
get resource
status
Verify the status (e.g.: AVAILABLE or NOT
AVAILABLE) of a set of resources (one or
more); the method will accept in input the
references to the resources. The method
will return information about the
requested resources.
References to
resources.
Status of resources.
52
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
get
availability
of resource
Return information about the availability
of a resource; the method will accept in
input the reference to the resource of
interest, the start date and end date
between checking the availability of the
resource itself.
The method checks the availability the
resources only in FESTIVAL platform: no
requests for verifying availability of
resources are sent to testbeds.
References to
the resource.
Start date.
End date.
Information about the
lock status of the
requested resource.
lock
resources
Perform the reservation of a set of
resources (one or more) in a specific range
of dates (start and end dates); the method
will accept in input: the reference to the
experiment that will use the resources, the
references to the resources to be reserved,
the start date and the end date of the
reservation period. The method will return
a message that in case of success confirms
the reservation of the resources and in
case of failure provides information about
the cause of the failure itself.
The method reserves the resources only in
FESTIVAL platform: no reservation
requests are sent to testbeds.
Reference to
the
experiment.
References to
resources.
Start date.
End date.
Message
unlock
resources
Perform the release of a reserved
resource; the method will accept in input
the reference to the experiment that uses
the resources and to the references to the
reserved resources. The method will
return a message that in case of success
confirms the release of the resources and
in case of failure provides information
about the cause of the failure itself.
Reference to
the
experiment.
References to
resources.
Message
53
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
The method release the resources only in
FESTIVAL platform: no requests are sent to
testbeds.
create
experiment
Create a new experiment; this method will
accept in input the information of a new
experiment and it will return the reference
to the new experiment.
Information
about the
experiment.
Reference to the new
experiment.
update
experiment
Update the information of an existing
experiment; this method will accept in
input the reference to the experiment to
be updated and the information of the
experiment itself. The method will return
a message that in case of success confirms
the update of the information related to
the experiment, while in case of failure
provides information about the cause of
the failure itself.
Reference to
the
experiment.
Information
about the
experiment.
Message
delete
experiment
Delete all information related to an
experiment and remove the reference to
the experiment itself; this method will
accept in input the reference to the
experiment and it will return a message
that in case of success confirms the
deletion of the experiment, while in case
of failure provides information about the
cause of the failure itself.
Reference to
the
experiment.
Message
start
experiment
Change the status of an experiment,
placing it "in execution"; the platform
executes the reservation/instantiation of
the resources selected for the experiment
as previously planned by the
experimenter. The method will accept in
input the reference to the experiment. The
method will return a message that in case
of success confirms the start of the
experiment, while in case of failure
Reference to
the
experiment.
Message
54
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
provides information about the cause of
the failure itself.
stop
experiment
Change the status of an experiment
placing it in "stop" and releasing the
resources previously
reserved/instantiated for its execution;
the method will accept in input the
reference to the experiment. The method
will return a message that in case of
success confirms the stop of the
experiment, while in case of failure
provides information about the cause of
the failure itself.
Reference to
the
experiment.
Message
get
experiment
status
Provides information about the status of
an experiment and of resources involved in
its execution; the method will accept in
input the reference to the experiment and
it will return information about its status
and the status of the resources involved in
its execution.
Reference to
the
experiment.
Status of the
experiment.
subscribe to
resource
Executes the subscription to a specific
resource (e.g.: for getting "live" data); the
method will accept in input the reference
to the resource and a call-back URL for
getting the data. The method will return a
message that in case of success confirms
the subscription to the resource, while in
case of failure provides information about
the cause of the failure itself.
References to
the resource.
Call-back URL.
Message
stop
subscription
Terminate the subscription to a specific
resource: the method will accept in input
the reference to the resource and it will
return a message that in case of success
confirms the termination of the
subscription to the resource, while in case
of failure provides information about the
cause of the failure itself.
References to
the resource.
Message
55
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
add
resources to
experiment
Associate a set of resources (one or more)
to an experiment; the method will accept
in input two parameters: the references to
the experiment and the references to the
resources to be associated. The method
will return a message that in case of
success confirms the associations of the
resources and in case of failure provides
information about the cause of the failure
itself.
The method associates the resources to
the experiment only in FESTIVAL platform:
no requests are sent to testbeds.
Reference to
the
experiment.
References to
resources.
Message
remove
resources
from
experiment
Remove the association between a set of
resources (one or more) with an
experiment; the method will accept in
input two parameters: the reference to the
experiment the references to resources to
be disassociated. The method will return a
message that in case of success confirms
the removal of associations between the
resources and the experiment, while in
case of failure provides information about
the cause of the failure itself.
The method remove the associations
between the resources and the
experiment only in FESTIVAL platform: no
requests are sent to testbeds.
Reference to
the
experiment.
References to
resources.
Message
Table 21 EaaS APIs
2.3.7.2. Modules involved in execution of EaaS APIs
This section describes interactions that occur between EaaS APIs Controller and the other components of
FESTIVAL architecture when exposed APIs are invoked and executed. For each API the following information
are reported:
The name of the API.
The list of modules involved in its execution.
56
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
A “sub schema” of the architecture depicting only modules involved in its execution.
A sequence diagram describing the interactions between modules involved in its execution.
API: get testbeds
Modules involved in the execution of this API are:
Resource Access Manager: it is in charge of retrieving available testbeds interacting with the
underlying Drivers (Open Data, IoT Gateway, IT Resources and Living Lab Drivers).
Generic Driver: Generic Driver represents the four drivers defined in FESTIVAL architecture. it is in
charge of retrieving information about the testbeds they manage.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Figure 9 depicts a sub schema of the architecture reporting only the modules involved in the execution of get
testbeds API and relations between them.
Figure 9 API get testbeds architecture sub schema
Figure 10 depicts the sequence diagram that illustrates the execution of “get testbeds” API.
57
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 10 API get testbeds sequence diagram
Sequence diagram description: EaaS API Controller receives a “get testbeds” request from Experimentation
Portal and requires Security module to verify the authorization of the user that made the request. Security
module authorizes the user and EaaS API Controller forwards the “get testbeds” request to Resource Access
Manager. Resource Access Manager requires Security module to verify the role of the user. Security module
confirms the role of the user and Resource Access Manager forwards the “get testbeds” request to the
Generic Driver (which stands for the four drivers defined in the FESTIVAL architecture: Open Data, IoT
Gateway, IT Resources and Living Lab Manager Drivers). Generic Driver returns the list of available testbeds
that is forwarded to EaaS API controller and then to Experimentation Portal.
API: get resources
Modules involved in the execution of this API are:
Resource Access Manager (Resource Discovery): it is in charge of retrieving available resources
interacting with the underlying Drivers.
Generic Driver: Generic Driver represents the four drivers defined in FESTIVAL architecture. it is in
charge of retrieving resources provided by the testbeds they manages.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Figure 11 depicts a sub schema of the architecture reporting only the modules involved in the execution of
get resources API and relations between them.
58
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 11 API get resources architecture sub schema
Figure 12 depicts the sequence diagram that illustrate the execution of create experiment API.
Figure 12 API get resources sequence diagram
Sequence diagram description: EaaS API Controller receives a “get resources” request from Experimentation
Portal and requires Security module to verify the authorization of the user that made the request. Security
module authorizes the user and EaaS API Controller invokes the “get available resources” method Resource
Access Manager (Resources Discovery). Resource Access Manager (Resources Discovery) requires Security
module to verify the role of the user. Security module confirms the role of the user and Resource Access
Manager (Resources Discovery) invokes the “get testbeds” method of the Generic Driver (which stands for
59
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
the four drivers defined in the FESTIVAL architecture. Generic Driver returns the list of available resources
that is forwarded to EaaS API controller and then to Experimentation Portal.
API: get resource status
Modules involved in the execution of this API are:
Resource Access Manager: it is in charge of checking the status of the resources specified in the
request through the interaction with the underlying Drivers.
Generic Driver: Generic Driver represents the four drivers defined in FESTIVAL architecture; it is in
charge of retrieving the status of the resources provided by the testbeds they manage.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Sub schema of the FESTIVAL architecture that illustrates the modules involved in the execution of get
resource status API and relations between them is the same of get testbeds API (Figure 9).
Figure 13 depicts the sequence diagram that illustrates the execution of get resource status API.
Figure 13 API get resource status sequence diagram
Sequence diagram description: EaaS API Controller receives a “get resource status” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “get resource status”
request to Resource Access Manager. Resource Access Manager requires Security module to verify the role
of the user. Security module confirms the role of the user and Resource Access Manager forwards the “get
resource status” request to the Generic Driver (which stands for the four drivers defined in the FESTIVAL
60
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
architecture: Open Data, IoT Gateway, IT Resources and Living Lab Driver). Generic Driver returns the status
of the resource that is forwarded to EaaS API controller and then to Experimentation Portal.
API: get availability of resource
Modules involved in the execution of this API are:
Resource Access Manager (Resource Directory): it is in charge of checking the status of the resources
specified in the request (if the resource is locked or not).
Security: it is in charge of verifying the user that performs the request and the role assigned.
Figure 14 depicts a sub schema of the architecture reporting only the modules involved in the execution of
get availability of resource API and relations between them.
Figure 14 API get availability of resource architecture sub schema
Figure 15 depicts the sequence diagram that illustrates the execution of get availability of resource API.
Figure 15 API get availability of resource sequence diagram
61
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives a “get availability of resource” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “get availability of
resource” request to Resource Access Manager (Resource Discovery). Resource Access Manager (Resource
Discovery) requires Security module to verify the role of the user. Security module confirms the role of the
user and Resource Access Manager checks the availability of the resource. Information about the availability
of the resource is returned to EaaS API controller and then to Experimentation Portal.
API: lock resources
Modules involved in the execution of this API are:
Resource Access Manager (Resource Directory): it is in charge of locking the resources specified in
the request.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Sub schema of the FESTIVAL architecture that illustrates the modules involved in the execution of lock
resources API and relations between them is the same of get availability of resource API (Figure 14).
Figure 16 depicts the sequence diagram that illustrates the execution of lock resources API.
Figure 16 API lock resources sequence diagram
Sequence diagram description: EaaS API Controller receives a “lock resource” request from Experimentation
Portal and requires Security module to verify the authorization of the user that made the request. Security
module authorizes the user and EaaS API Controller forwards the “lock resource” request to Resource Access
62
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Manager (Resource Directory). Resource Access Manager (Resource Directory) requires Security module to
verify the role of the user. Security module confirms the role of the user and Resource Access Manager
performs the lock of the resource. Confirmation about the lock of the resource is returned to EaaS API
controller and then to Experimentation Portal.
API: unlock resources
Modules involved in the execution of this API are:
Resource Access Manager (Resource Directory): it is in charge of unlocking the resources specified in
the request.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Sub schema of the FESTIVAL architecture that illustrates the modules involved in the execution of unlock
resources API and relations between them is the same of get availability of resource API (Figure 14).
Figure 17 depicts the sequence diagram that illustrates the execution of unlock resources API.
Figure 17 API unlock resources sequence diagram
Sequence diagram description: EaaS API Controller receives a “unlock resource” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “unlock resource” request
to Resource Access Manager (Resource Directory). Resource Access Manager (Resource Directory) requires
Security module to verify the role of the user. Security module confirms the role of the user and Resource
63
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Access Manager performs the unlock of the resource. Confirmation about the unlock is returned to EaaS API
controller and then to Experimentation Portal.
API: create experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of verifying the integrity and the coherence of the information
related to the new experiment, to invoke the Storage Service module of registering the new
experiment and to notify the creation of a new experiment to EaaS KPI Monitoring.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Storage Service: it is in charge of registering the new experiment in the “Experiment/Resources
Information” storage.
EaaS KPI Monitoring: it is in charge to handling the notification provided by Experiment Control
module.
Figure 18 depicts a sub schema of the architecture reporting only the modules involved in the execution of
create experiment API and relations between them.
Figure 18 API create experiment architecture sub schema
Figure 19 depicts the sequence diagram that illustrate the execution of create experiment API.
64
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 19 API create experiment sequence diagram
65
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives a “create experiment” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “create experiment”
request to Experiment Control. Experiment Control requires Security module to verify the role of the user.
Security module confirms the role of the user and Experiment Control requires the Storage Service to save
the data of the new experiment. Storage Service confirms that the data of the new experiment are stored
and Experiment Control notifies the EaaS KPI Monitoring about the creation of the new experiment. EaaS KPI
Monitoring confirms delivery of the notification. Experiment Control return the confirmation of the creation
of the new experiment to EaaS API Controller and then the confirmation is returned to Experimentation
Portal.
API: update experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of verifying the integrity and the coherence of the information
related to the experiment, to invoke the Storage Service module of registering it and to notify the
update of an experiment to EaaS KPI Monitoring.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Storage Service: it is in charge of registering the updated experiment in the “Experiment/Resources
Information” storage.
EaaS KPI Monitoring: it is in charge to handling the notification provided by Experiment Control
module.
The sub schema of the architecture reporting the modules involved in the execution of update experiment
API and relations between them is depicted in Figure 18.
Figure 20 depicts the sequence diagram that illustrates the execution of update experiment API.
66
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 20 API update experiment sequence diagram
67
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives an “update experiment” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “update experiment”
request to Experiment Control. Experiment Control requires Security module to verify the role of the user.
Security module confirms the role of the user. After that Experiment Control requires Security module to
verify the rights of the user on the experiment to update. Security module confirms the rights of the user on
the experiment and Experiment Control requires the Storage Service to save the data of the experiment.
Storage Service confirms that the data of the experiment are stored and Experiment Control notifies the EaaS
KPI Monitoring about the update of the experiment. EaaS KPI Monitoring confirms delivery of the
notification. EaaS KPI Monitoring confirms the registration of the KPI. Experiment Control return the
confirmation of the update of the experiment to EaaS API Controller and then the confirmation is returned
to Experimentation Portal.
API: delete experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of executing the deletion of the experiment, to invoke the Storage
Service module of erasing it end its related information and to notify the deletion of an experiment
to EaaS KPI Monitoring.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Storage Service: it is in charge of deleting experiment from the “Experiment/Resources Information”
storage and its related information.
EaaS KPI Monitoring: it is in charge to handling the notification provided by Experiment Control
module.
The sub schema of the architecture reporting the modules involved in the execution of delete experiment
API and relations between them is depicted in Figure 18.
Figure 21 depicts the sequence diagram that illustrates the execution of delete experiment API.
68
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 21 API delete experiment sequence diagram
69
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
EaaS API Controller receives a “delete experiment” request from Experimentation Portal and requires
Security module to verify the authorization of the user that made the request. Security module authorizes
the user and EaaS API Controller forwards the “delete experiment” request to Experiment Control.
Experiment Control requires Security module to verify the role of the user. Security module confirms the role
of the user. After that Experiment Control requires Security module to verify the rights of the user on the
experiment to delete. Security module confirms the rights of the user on the experiment and Experiment
Control requires the Storage Service to erase the data of the experiment. Storage Service confirms that the
data of the experiment are deleted and Experiment Control notifies the EaaS KPI Monitoring about the
creation of the new experiment. EaaS KPI Monitoring confirms delivery of the notification. Experiment
Control return the confirmation of the deletion of the experiment to EaaS API Controller and then the
confirmation is returned to Experimentation Portal.
API: start experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of placing the experiment in the "in execution" status, requesting
the underlying modules to reserve or instantiate on the testbeds the resources selected for the
experiment.
Resource Access Manager: it is in charge of reserving or instantiating the resources selected for the
experiment through the interaction with the underlying Drivers (reserve or instantiate procedures
are admitted only for IoT and IT resources) and to require the registration of a new KPI concerning
the reservation/instantiation of the resources related to an experiment.
Generic Driver: Generic Driver represents the four drivers defined in FESTIVAL architecture; it is in
charge of reserving/instantiating the resources for the execution of the experiment.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Storage Service: it is in charge of storing the new status of the experiment in the
“Experiment/Resources Information” storage.
Figure 22 depicts a sub schema of the architecture reporting only the modules involved in the execution of
start experiment API and relations between them.
70
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 22 API start experiment architecture sub schema
Figure 23 depicts the sequence diagram that illustrates the execution of start experiment API.
71
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 23 API start experiment sequence diagram
72
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives a “start experiment” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the “start experiment”
request to Experiment Control. Experiment Control requires Security module to verify the role of the user.
Security module confirms the role of the user. After that Experiment Control requires Security module to
verify the rights of the user on the experiment to start. Security module confirms the rights of the user on
the experiment and Experiment Control requires the Resource Access Manager to reserve/instantiate
resources for the experiment. Resource Access Manager requires Security module to verify the role of the
user. Security module confirms the role of the user. After that Resource Access manager requires Storage
Service to retrieve the list of resources of the experiment. Storage Service returns the list of resources of the
experiment. Resource Access Manager requires the Generic Driver (which stands for the four drivers defined
in the FESTIVAL architecture) to reserve/instantiate resources. Generic Driver returns the confirmation of the
reservation/instantiation of the resources to Resource Access Manager and Resource Access Manager
returns the confirmation to Experiment Control. Experiment Control requires the Storage Service to change
the status of the experiment placing it “in execution”. Storage Service confirms the change of the status of
the experiment. Experiment Control returns the confirmation of the start of the experiment to EaaS API
Controller and then it returns the confirmation to Experimentation Portal.
API: stop experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of placing the experiment in the "stop" status, requesting the
underlying modules to release resources previously reserved/instantiated for the experiment.
Resource Access Manager: it is in charge of releasing the resources previously reserved/instantiated
for the experiment through the interaction with the underlying Drivers and to require the registration
of a new KPI concerning the releasing of the resources related to the experiment.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
Storage Service: it is in charge of storing the new status of the experiment in the
“Experiment/Resources Information” storage.
The sub schema of the architecture reporting the modules involved in the execution of stop experiment API
and relations between them is depicted in Figure 22.
Figure 24 depicts the sequence diagram that illustrates the execution of start experiment API.
73
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 24 API stop experiment sequence diagram
74
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
In the Figure 24 EaaS API Controller receives a “stop experiment” request from Experimentation Portal and
requires Security module to verify the authorization of the user that made the request. Security module
authorizes the user and EaaS API Controller forwards the “stop experiment” request to Experiment Control.
Experiment Control requires Security module to verify the role of the user. Security module confirms the role
of the user. After that Experiment Control requires Security module to verify the rights of the user on the
experiment to stop.
Security module confirms the rights of the user on the experiment and Experiment Control requires the
Resource Access Manager to release the resources of the experiment. Resource Access Manager requires
Security module to verify the role of the user. Security module confirms the role of the user. After that
Resource Access manager requires Storage Service to retrieve the list of resources of the experiment. Storage
Service returns the list of resources of the experiment. Resource Access Manager requires the Generic Driver
(which stands for the four drivers defined in the FESTIVAL architecture) to release the resources. Generic
Driver returns the confirmation of the release of the resources to Resource Access Manager and Resource
Access Manager returns the confirmation to Experiment Control. Experiment Control requires the Storage
Service to change the status of the experiment placing it in “stop”. Storage Service confirms the change of
the status of the experiment. Experiment Control returns the confirmation of the stop of the experiment to
EaaS API Controller and then it returns the confirmation to Experimentation Portal.
API: add resources to experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of associating the resources specified in the request with the
experiment.
Storage Service: it is in charge of storing the association between the resources and the experiment.
Security: it is in charge of verifying the user that performs the request and the role assigned.
The sub schema of the architecture reporting the modules involved in the execution of add resources to
experiment API and relations between them is depicted in Figure 25.
Figure 25 API add resources to experiment architecture sub schema
Figure 26 depicts the sequence diagram that illustrates the execution of add resources to experiment API.
75
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 26 API add resources to experiment sequence diagram
76
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives a “add resources to experiment” request from
Experimentation Portal and requires Security module to verify the authorization of the user that made the
request. Security module authorizes the user and EaaS API Controller forwards the request to Experiment
Control. Experiment Control requires Security module to verify the role of the user. Security module confirms
the role of the user. After that Experiment Control requires Security module to verify the rights of the user
on the experiment. Security module confirms the rights of the user on the experiment. Experiment control
requires the Storage Service to register the association between the resource and the experiment. Storage
Service confirms the associations. The confirmation of the association is returned to EaaS API control and
then to Experimentation Portal.
API: remove resources from experiment
Modules involved in the execution of this API are:
Experiment Control: it is in charge of disassociating the resources specified in the request from the
experiment.
Storage Service: it is in charge of erasing the association between the resources and the experiment.
Security: it is in charge of verifying the user that performs the request and the role assigned to
him/her.
The sub schema of the architecture reporting the modules involved in the execution of remove resources
from experiment API and relations between them is depicted in Figure 25.
Figure 27 depicts the sequence diagram that illustrates the execution of remove resources from experiment
API.
77
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 27 API remove resources from experiment sequence diagram
78
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Sequence diagram description: EaaS API Controller receives a “remove resources from experiment” request
from Experimentation Portal and requires Security module to verify the authorization of the user that made
the request. Security module authorizes the user and EaaS API Controller forwards the request to Experiment
Control. Experiment Control requires Security module to verify the role of the user. Security module confirms
the role of the user. After that Experiment Control requires Security module to verify the rights of the user
on the experiment. Security module confirms the rights of the user on the experiment. Experiment control
requires the Storage Service to delete the association between the resource and the experiment. Storage
Service confirms the deletion of the associations. The confirmation of the deletion of the association is
returned to EaaS API control and then to Experimentation Portal.
2.3.7.3. Administration API Controller
This section describes “Administration API controller”, the sub module of EaaS API controller dedicated to
the provision of the APIs for the administration of the FESTIVAL platform.
API Description Input
Parameters
Output Parameters
create user Create a new user; this method will accept
in input the information of a new user and
it will return the reference to the new
user.
Information
about the
user.
Reference to the new
user.
update user Update the information of an existing user;
this method will accept in input the
reference to the user to be updated and
the information of the user itself. The
method will return a message that in case
of success confirms the update of the
information related to the user, while in
case of failure provides information about
the cause of the failure itself.
Reference to
the user.
Information
about the
user.
Message
delete user Delete all information related to a user and
remove the reference to the user itself;
this method will accept in input the
reference to the user and it will return a
message that in case of success confirms
the deletion of the user, while in case of
failure provides information about the
cause of the failure itself.
Reference to
the user.
Message
79
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
enable user Enable the user to interact with the EaaS
APIs; this method will accept in input the
reference to the user and it will return a
message that in case of success confirms
that the user is enabled, while in case of
failure provides information about the
cause of the failure itself.
Reference to
the user.
Message
disable user Disable the user to interact with the EaaS
APIs; this method will accept in input the
reference to the user and it will return a
message that in case of success confirms
that the user is disabled, while in case of
failure provides information about the
cause of the failure itself.
Reference to
the user.
Message
create role Create a new role in the platform; this
method will accept in input the
information of the new role and it will
return the reference to the new role. A
role identifies the permission that can be
associated to the users of the platform
(e.g: permission of creating experiment,
permission of using some specific
resources, etc…)
Information
about the
new role.
Reference to the new
role
update role Update the information of an existing role;
this method will accept in input the
reference to the role to be updated and
the information of the role itself. The
method will return a message that in case
of success confirms the update of the
information related to the role, while in
case of failure provides information about
the cause of the failure itself.
Reference to
the role.
Information
about the
role.
Message
delete role Delete all information related to a role and
remove the reference to the role itself;
this method will accept in input the
reference to the role and it will return a
message that in case of success confirms
the deletion of the role, while in case of
Reference to
the role.
Message
80
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
failure provides information about the
cause of the failure itself.
associate
user role
Associate a user to a role; this method will
accept in input the reference to the role
and the reference to user and it will return
a message that in case of success confirms
the association of the user and of the role,
while in case of failure provides
information about the cause of the failure
itself.
Reference to
the user and
reference to
the role.
Message
dissociate
user role
Dissociate a user to a role; this method will
accept in input the reference to the role
and the reference to user and it will return
a message that in case of success confirms
the dissociation of the user and of the role,
while in case of failure provides
information about the cause of the failure
itself.
Reference to
the user and
reference to
the role.
Message
update
platform
configuration
Update the configuration of the platform,
this method will accept in input the
information related to the configuration of
the platform and it will return a message
that in case of success confirms the update
of the configuration, while in case of
failure provides information about the
cause of the failure itself.
Platform
configuration.
Message
get platform
kpis
Retrieve the KPIs collected by the
platform; this method will not accept any
parameters in input and it will return the
collected KPIs.
# KPIs
Table 22 Administration APIs
81
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.4. FESTIVAL Experimentation Portal
The FESTIVAL experimenter portal is the main access point for new experimenters to the platform.
This portal will give the experimenters the possibilities of creating a new account, accessing to the
information of the different resources, and performing experiments (including the creation and
management of experiments).
The FESTIVAL Experimenter Portal provides visual elements that will allow the use of the APIs defined
in section 2.3.7. Along with the direct access to the platform through the defined APIs, the inclusion
of the FESTIVAL portal will provide to the experimenter several options to access to the FESTIVAL
platform. Thus, user interaction with the platform can be made in two different ways:
Accessing the APIs that are provided directly (backend), which aims at offering the ability to
create different applications programmatically by the experimenter.
Accessing the Portal by the Experimenter (frontend).
As mentioned above, the main function of the portal is to provide a graphical user interface to the
experimenter to perform most of the actions identified within the EaaS platform. The use of the
platform must be simple and straightforward, to make the platform as friendly as possible.
To implement the Experimentation Portal, web technologies are taken into account, including HTML
and CSS, as well as technologies for web application development (e.g.: AngularJS [15]). The final
technologies used will be described in the forthcoming deliverables of the work package 2. In this
sense, we can divide the Experimentation Portal in two different sections, depending on the features
offered therein: (1) creating a new user account on the platform and the experimenter's home,
including the creation of a new experiment; (2) documentation of APIs to access to resources, as well
as the information available about them.
Furthermore, the portal will also feature the access for Living Labs administrators, which will provide
a similar graphical interface with exclusive access to a set of features. This access will be used by Living
Lab administrators to meet the demands from experimenters, as well as to modify the details of the
Living Labs (e.g. the inclusion of a new feature in the Living Lab, or a new metric from users who
frequent the Living Lab).
Following sections describe the different part of the Experimentation Portal, including mockups of the
different portal pages, which may vary with respect to the final version.
2.4.1. Experimenter home page and experiment definition
As described in Section 3.3. , the first step for an experimenter that accesses the platform for the first
time is the creation of a user account. The experimenter will be able to access to the experimenter
portal home page and create the account. The account creation will ask for general details, such as
82
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
the user name, organization, the description of his/her work as an experimenter and interests. This
information will serve to the FESTIVAL administrators to grant rights with specific user roles.
Once the account is created, the experimenter can access its home page where he/she will find the
details of the account and the possibility of creating new experiments, as shown in Figure 28. In this
mockup figure, the general information required to create a new experiment is requested. Among this
information we can find the title of the experiments and the description (including optional details
such as the actors, methods, metrics or technical details about the experiment, among others).
Figure 28 Home Page, new experiment definition
83
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
In addition, an experimenter with several experiments created will be able to access the detailed
information of each one of his/her experiments, including the details introduced at the time of the
experiment creation, among others. This information will be modifiable, so the experiment can be
extended or reduced. It is worth mentioning that in both cases, either during the experiment creation
or in the details of the current experiments, it will be possible to include new team members to access
and work in the experiments.
Figure 29 Home page experiment details
While defining a new experiment, the Experimentation Portal will feature a set of tools to ease the
experiment creation. Among these tools we can include the search of available resources in FESTIVAL,
depending on the time period in which the experiment is conducted, or the representation of the
resources depending on their characteristics, such as the geographic location or the resource type.
The pages to choose and select the different resources depend in their types, as described at following.
84
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 30 IoT resource selection view
Figure 30 shows the page for the resource selection. In this section of the portal, the user will access to the
details of the resources by several means as explained before. As shown in the mockup, the resources can
be located in a map with their exact location from the different sensors. A search box is also available to
distinguish the nodes with the required sensors. Additionally, accessing to more details (e.g. sensor range of
measurement, etc.) is also possible. Sensor details also include the periods where the sensor is available or
not, depending if it is locked in the platform.
85
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 31 IT resources selection view
Figure 31 depicts the IT selection view. In this case, the resources can be searched depending on the type of
software components they have included, such as the specific Generic Enabler required for an experiment.
It is also possible to create VMs with a set of specific characteristics, always considering the availability of
hardware resources in the federation.
In the case of the Open Data, the resources are freely available to everyone with the existing Federated Open
Data Platform. In this sense, the Experiment Portal will show the webpage that aggregates the other portals,
showing the already existing possibilities (for instance, searching datasets depending on the name, the
geolocation or the portal). Figure 32 shows a possible mockup of its integration.
86
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 32 Open Data resource selection
2.4.2. Documentation
This section provides documentation of existing APIs for accessing data from the FESTIVAL resources.
This description will help experimenters to develop applications that use the backend directly.
API documentation is presented graphically and will have elements to make calls directly from the
browser. In this regard, the description of the API includes not only the endpoint to make the calls,
but the call type (e.g. if it is GET, POST, etc. in the REST interface), and input and output parameters.
These parameters will be also described with examples using the format defined (e.g. JSON).
For the description of the RESTful APIs, we will use an existing representation engine for RESTful APIs,
such as Swagger [16] or RAML [17]. Figure 33 shows the integration of an API described with RAML in
the portal.
87
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 33 IoT detailed information including RAML documentation of APIs
In addition to the APIs description, the documentation also includes specific details of the resources. For
instance, Figure 34 shows the details of an IoT resource.
88
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 34 IoT information page
2.4.3. Living Lab Administration Portal
The Living Lab Administration Portal will help the Living Lab Administrators to address the requests from the
experimenters that wants to involve the Living Labs from FESTIVAL. This portal is needed as the interaction
between experimenters and Living Labs cannot be automated and direct contact is required (for instance,
physical meetings for deployments of prototypes).
In Living Lab Administration Portal, the Living Lab Administrators will be able to update the details of the
Living Labs, including future events to be held, available rooms to organize meetups, focus groups, etc.
Additionally, the Living Lab Administration Portal will work as a GUI for a message exchange system between
experimenters and managers. Figure 35 and Figure 36 shows the home page of the Living Lab and the mockup
of the features offered by the Living Labs, including the services and expertise.
89
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 35 Living lab general information and contact point
90
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 36 Service and expertise information
Figure 37 shows an example of the management of the requests sent to the experimenter. The requests will
be ordered depending on the experiment and the user that sent the request. These requests will include the
details of what they expect from the Living Lab, taking into account the limitations of the Living Lab and the
needs of the experimenter.
91
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 37 Pending requests to the living lab manager
2.4.4. Administration Portal
Finally, the Administration Portal is the entry point for the FESTIVAL managers where they can manage the
rights of the user, configure certain parameters of the platform (e.g. the maximum number of experiments
at the same time), or to stop experiments that are misusing the platform.
92
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 38 Administration Portal, experiment view
As an example, the Figure 38 shows a possible setup of the administration portal indicating the list of
experiments that are running in the platform at the time being. In this view, the manager can stop the
experiment, or access their details. Regarding to the assignments of the user rights, the administration
portal will show the users in different list: accepted users with an assigned role and users waiting for
a role. Therefore, the FESTIVAL managers will be able to modify the roles of the experimenters.
93
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2.5. Security
2.5.1. General overview of the security architecture
This section aims at providing the architecture of the security component in FESTIVAL. As an important part
of the architecture, the security module is in charge of providing the Authentication, Authorization and
Accounting for the FESTIVAL platform. It is considered as a core module and most of the other software
implementations will depend on this module.
The tasks of this module are listed below:
Authentication: this functionality provides access to the users, comparing the credentials provided
by the users with the existing database. The authentication functionality of this module will confirm
and assess that the user logging into the platform is who he/she says to be.
Authorisation: this functionality determines whether the user can access to a specific set of
resources or not. Furthermore, it also considers what are the commands and actions the users can
perform. So as to assess that the authenticated user is authorised to perform the requests, the
module will check if the assigned roles to the user have the required rights.
Accounting: finally, the security module will monitor the access and requests to keep track and log
the use of the platform and the access attempts, to prevent misuses of the platform.
The security module will be an isolated module with its own databases and software components. This
is to avoid possible security leaks that might result of using common modules such as the Storage
Service module.
The considered options for implementing the security in FESTIVAL are two. On the one hand, the
security mechanisms adopted in Fed4Fire [18], based on a common Certificate Authority (CA) that
provides the credentials to the experimenters. The general CA must be trusted by all the testbeds and
they have to finally confirm the authentication of the experimenter and use their own mechanism of
authorisation and accounting. The method is based on a «speak for» basis, thus including a signature
in the HTTPS requests for every different layer and module in the request process. This allows the
testbeds to identify the user and the components in order to avoid «man in the middle» attacks.
On the other hand, the use of an «identity manager» as a central component to provide the
functionalities, authentication, authorization and accounting is considered. This component will act as
a server for the rest of the modules in the EaaS platform, providing and assuring the identity of the
experimenters. This means that every request made to the platform will go through this component
to verify the identity of the experimenter.
Taking in consideration the two options described above, the architecture considers the use of an
identity manager to perform the security checks. Although that means to include a new component
into the platform, and the testbeds will have to assign specific resources to FESTIVAL, avoid ing specific
implementations in the testbeds and keeping the original concept of using the aggregators to manage
the resources from several testbeds depending on their type. Furthermore, there are several open
94
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
source options that can be used in FESTIVAL to implement this module, providing most of the
requirements in terms of security. Among them we can find Keystone [19], Keyrock [20], OpenIAM
[21] or OpenIDM [22].
Figure 39 Security sequence diagram
In the Figure 39 is depicted the sequence diagram for an EaaS request. A generic request to the EaaS
APIs is considered. As depicted, the first step for a user that wants to perform an experiment in the
platform is to create an account and to log in. Once the user has logged into the EaaS platform, it will
return a credentials or token, valid for a certain period of time. Afterwards, all the requests made to
the platform must use the token to be authorised. This token will be used by the modules providing
the different functionalities, to check through the security module whether it is valid or not. The
security module will check them if the token is not expired and the user role and associated rights. If
the user token is accepted, the EaaS APIs will return the expected response. In case the user token is
not accepted, the EaaS APIs will send an unauthorised response.
The following sections describe in detail the different aspects of the security mechanisms to avoid
malicious access attempts to the platform, as well as securing experimentation data.
2.5.2. Authentication process
The authentication process in the EaaS platform identifies a user that provides a set of credentials
(they can be a classic user and password or Public/Private based certificate).
95
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
The user will need to log into the platform to obtain the token that will allow him /her to perform the
different requests. The method to perform the authentication will be based on the IdM (Identity
Management) included in the security module, which will check the user credentials and return the
token. The authentication method is based on a single sign-on functionality as explained in section
2.5.2.1.
2.5.2.1. Single Sign-on
As part of the FESTIVAL federation, users must be able to log into EaaS platform and to access to the FESTIVAL
resources (always depending on their rights in the platform). Additionally, future components or federated
entities should not require additional login methods for an existing user. In this sense, they should have only
one account and login method, thus providing them access in a homogeneous way to all of the resources
using the FESTIVAL APIs.
So as to provide a single user account for the platform and its components, we need to provide a Single Sign-
on system to identify the user. There are several technologies providing single sign-on functionality, such as
OAUTH2 [23] or OPENID [24] connect, which also relies on OAUTH2. In addition to these SSO (Single Sign On)
technologies, the Security Assertion Markup Language (SAML) [25] is the leading standard for SSO
authentication. SAML is an open standard that defines a XML structure for the exchange of authentication
and authorization data.
In the case of the FESTIVAL EaaS platform, the single sign-on provided by the Identity Manager
implementation will be used. Some proprietary examples of SSO implementations are offered by internet-
based companies such as Google, with Google Sign-in [26], or Facebook with Facebook Connect [27].
Figure 40 Authentication Process
The sequence diagram, depicted in Figure 40, to authenticate into the platform is explained as follows:
1. The user goes to the Experimentation Portal and sends an authentication request.
2. The Experimentation Portal loads the authentication page provided by the IdM, which asks
for the credentials of the user.
3. The user introduces his/her credentials.
96
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
4. If the credentials are right, the IdM will authenticate the user and return a valid token.
The previous sequence can be performed using either the web portal or the EaaS API through HTTPS requests.
Erreur ! Source du renvoi introuvable. reports an example of query for a token using the credentials of a
user. The response to this request, that was accepted, is depicted in Table 24.
POST /oauth2/token HTTP/1.1
Host: account.lab.fiware.org
Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code&code=SplxlOBeZQQYbYS6WxSbIA
&redirect_uri=https%3A%2F%2Fclient%2Eexample%2Ecom%2Fcallback_url
Table 23 Access token request
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Cache-Control: no-store
Pragma: no-cache
{
"access_token":"2YotnFZFEjr1zCsicADFVB",
"token_type":"bearer",
"expires_in":3600,
"refresh_token":"tGzv3JOkF0XG5Qx2TlKWIA",
}
Table 24 Token response
2.5.3. Authorization Process
Once the user has been authenticated into the EaaS platform and has obtained the token, he/she will
be able to perform a set of requests depending on his/her user role. In FESTIVAL, the user rights are
delimited by a set of roles that can be assigned to a user. These roles are delimited by the number of
functionalities and resources that a user can access.
97
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
As aforementioned, the authorisation of the user to use the APIs and to access the resources of the
platform will depend on the security component deployed in FESTIVAL, therefore testbeds will need
to trust on these components.
Figure 41 Authorization Process
The process to authorise a user to access to a specific resource is depicted in Figure 41. The user will
perform the request to the EaaS APIs including the token generated when he/she was authenticated
in the platform. This token have finite life time. Once this time is expired, the token will not be
accepted in the future. The token will be also rejected if the user creates a new token, replacing the
old one. If the token is valid, the EaaS APIs Controller will return the information from FESTIVAL
platform.
2.5.4. Intra-layer communication
The communication between the components in FESTIVAL will be secured by the use a trusted network
SSL encryption. In this regard, the components could be deployed anywhere, but the machines
supporting the different components will need to log into a secured VPN only accessible for FESTIVAL.
Within the VPN, all the communications will be encrypted, thus avoiding information leaks. This
includes all the components between the EaaS APIs Controller and the four aggregators in the bottom
layer.
98
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 42 Architecture components within the VPN
99
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
In the Figure 42 the current architecture is depicted including a red box containing the elements within
the VPN. It is important to highlight that the EaaS APIs Controller will be accessed by outside of the
network as the component will expose the API to the experimenters. In this context, the bottom l ayer
composed by the gateway components, they will have also a connection to the outside of the VPN,
but in this case the IP addresses of the network will be filtered by firewall, as the testbed network
parameters are known (i.e. filtering all the IPs that does not belong to the testbeds).
2.5.5. End-to-end encryption support for experimenters
In order to provide end-to-end encryption for EaaS APIs and portal users, all the requests made against
the platform will use the secure http protocol. This will provide the authentication of the FESTIVAL
webpage to the experimenters, and the encryption of the communications of the requests.
So as to provide this functionality, an external CA trusted from most of the existing web browsers will
be used. The basic functionality will follow the classic HTTPS protocol as shown in Figure 43. The
experimenter will exchange the public keys with the EaaS FESTIVAL platform, and the platform will
send a signed SSL certificate that will be validate with the trusted CA.
Figure 43 Example of the Encrypted HTTPS communication with FESTIVAL EaaS
There are many existing CA providers to perform HTTPS session with a server, but most of them
requires a fee. However, there is a new CA provider, let’s Encrypt [28] from the Linux Foundation
Collaborative projects [29] that could be used for this reason. The main characteristic of this CA is that
it is open and free of charges, what makes it a great candidate for the implementation of the HTTPS.
100
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
3. Experiment workflow
This section describes, as an illustrative use case, the workflow corresponding to a generic experiment
performed over the FESTIVAL platform. It is divided in the following parts: first, a short description of the
common use case is given, to support the experiment workflow discussion; the boundary condition
assumptions for a first time experimenter are also depicted; afterwards, we describe the four experiment
phases: (1) account creation and documentation; (2) experiment definition; (3) execution of the experiment
and (4) experiment finalization.
3.1. Generic use case description
In order to provide the common grounds for the experiment workflow description in FESTIVAL, an
illustrative generic use case is taken into account, in which an experimenter uses the EaaS platform:
An experimenter aims to build an application using a wide range of sensor data from different sources
(e.g. temperature information from a city to correlate it with indoor data, bike usage from different
cities to compare them and to assess broader comparison, including for instance cultural factors, etc.);
he/she thus require a considerable amount of resources (power, computing) to process all the data.
Additionally, the results will be afterwards used to deploy a service, and the experimenter would like
to have feedback from final end-users about their own experience.
The experimenter will therefore need to have access to a wide and heterogeneous set of resources,
coming from different testbeds. Hence, he/she would need to deal with a large number of testbed
managers to get the corresponding credentials. The proposed EaaS platform, allows providing a unique
access to these resources. Additionally, the experimenter will also need to access infrastructure
resources, such as virtual machines and a reliable network substratum able to provide the
requirements posed by the corresponding processing. Finally, granting access to end users would
require the cooperation with living labs and local authorities, which could lead to additional costs.
3.2. Boundary conditions
Before depicting the common experiment workflow, we first need to establish a number of boundary
conditions that are considered for all the experimenters when they start, for the first time, an
experiment in the platform. They are identified for a generic user how has not acce ssed the EaaS
platform earlier and without any relationship with the project. The conditions are enumerated below.
The experimenter knows the main FESTIVAL webpage.
The experimenter does not have any account in the FESTIVAL platform.
The experimenter is not aware of FESTIVAL platform API.
The experimenter requires different types of resources from various testbeds.
The experimenter would like to avoid the interaction with multiple testbeds/endpoints.
101
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
3.3. Account creation and documentation
This is the first phase for any experimenter willing to use the FESTIVAL platform. We assume that an
experimenter is aware of FESTIVAL, thanks to the different dissemination activities. Hence, his
knowledge of the platform would be rather limited and he will first need to learn the federation
composition and the possibilities that are offered. During this first phase, the experimenter accesses
the FESTIVAL portal for the first time. In his/her first sessions, he/she will be able to learn the
description of the testbeds belonging to the federation, the generic description of the resources that
can be found within the federation, and the possibilities and technologies offered by the platform.
Finally, if he/she decides to use the platform to experiment, he/she will create an accoun t.
Figure 44 Account creation and access to documentation workflow
102
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
As shown in Figure 44, this phase can be divided in two different parts: before and after the account
creation.
Before the account creation:
1. Initially, the experimenter wants to perform an experiment, embracing different sensors and
some VMs. He/she has heard about the FESTIVAL possibilities and decides to access the
FESTIVAL portal to learn what is available and how he/she can access the corresponding
resources.
2. The experimenter will find, in the FESTIVAL portal, a description of the existing testbeds in the
federation, including the resources they offer (VMs, sensors, etc), and their short depiction.
Afterwards, if he/she wants to continue with the experiment in FESTIVAL, he/she will be asked
to create an account.
3. The experimenter will be able to create an account in FESTIVAL, to get access rights for the
federation; he/she would be also able to obtain more detailed description of the existing tools.
After the account creation:
4. Once the account is created, the experimenter will need to submit a number of additional
details, such as his/her name, organization, interests, etc. Once the account is finally created,
the experimenter would be able to learn more detailed information about the testbeds,
including availability, detailed descriptions, access methods, etc.
5. Additionally, he/she will be also given information about the steps that are needed to perform
the experiments, together with a succinct description of the corresponding APIs and tools. The
account creation will provide to the experimenter the credentials to access to FESTIVAL
through the experimenter API (RESTful).
6. During the account creation, it is given the minimum rights; once it is approved the definitive
ones are granted by a FESTIVAL manager, who can assign a specific role to the account.
3.4. Experiment definition
During this phase the experimenter will have the opportunity to create a new experiment, define its
characteristics (for instance, title and description), and select the resources that will be used to
perform the experiment. Figure 45 shows the workflow for the experiment definition phase.
103
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 45 Experiment definition workflow
1. As a first step, when the experimenter decides to create an experiment, he/she defines its
name and description, as well as other additional details yet to be defined.
104
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2. Once the experiment is defined, the portal provides the set of resources that will be used
during its execution. These resources are chosen based on certain constraints: the availability
and the specific resources that are required for the experiment; in this sense, the experimenter
is able to accommodate his/her needs to such constraints:
The experimenter establishes a period of time and checks which resources are
available.
The experimenter chooses a set of resources and checks when they will be
available.
3. Finally, once the time period is fixed and the resources are chosen, the experimenter will be
able to access the specific API calls to learn how the data will be accessed:
He/she gets the specific calls for the sensor data (subscription, synchronous
requests, etc).
He/she gets the calls required to access OpenData.
He/she obtains the credentials required to access the VMs.
He/she also gets the possibility to communicate to the Living Lab Administrators.
3.5. Execution of the experiment
Once the experiment has been defined and the resources are locked, the experiment can be started
at any time. The beginning of an experiment actually triggers the final reservation of the resources,
including the instantiation of the VMs and the subscription to different information elements
(sensors). During this phase, the experimenter will be able to add resources or remove those
previously locked. Figure 46 shows the experiment execution workflow.
105
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 46 Execution of the experiment workflow
1. The experiment starts. At this moment, the resources are reserved and instantiated (if needed)
at the different testbeds. Upon this moment, the experimenter can retrieve the credentials
required to access the VMs.
106
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
2. In addition to the previous step, taking into account that the interaction with the Living Lab
Administrators must be made directly, the experimenter can write to the administrator in
order to request available rooms and activities in the Living Labs. This communication will be
done in parallel with the management of the devices.
3. The experimenter can now perform the experiments with the reserved resources
4. The experimenter can deploy services to run for certain time periods within the VMs. If these
services implement the storage functionality provided by FESTIVAL (OML), the experimenter
will be able to access to the logs afterwards.
5. Furthermore, the experimenter will be able to modify the reserved resources and the
experiment itself, by locking and reserving more resources if they are available. He could also
make some of the resources that were previously locked available, if he/she is not using them
any longer.
6. While the experiment is running, the experimenter can monitor the reserved resources. He
can indeed check through the portal whether there is any issue for either the sensors or VMs,
which could have affected his/her experiment (for instance a sensor that stopped sending
values for a certain period of time).
During this phase, the experimenter might need to visit the Living Labs, depending whether he/she wants to
involve real end-users with some prototypes, or just gathering the feedback through surveys or focus groups,
what could be done remotely.
3.6. End of the experiment
The last phase of an experiment over the FESTIVAL platform is its finalization. The experiment can
conclude after two different circumstances:
the experimenter considers that the experiment finished before the time that was originally
planned is over;
the time period finishes, and the experiment is automatically stopped.
In both cases, the experiment will be considered as concluded, which entails closing all the remaining
subscriptions, deleting the VMs that were instantiated, and stopping the FESTIVAL functionalities used
by the experimenter (for example, the storage service).
1. If the experiment time period is about to finish, before closing the experiment and deleting
the involved VMs and subscriptions, the experimenter is notified about this circumstance and
he/she will be given the opportunity to extend the experiment period or to save the
corresponding output data and results, which were obtained during the experiment.
107
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
a. If there are no free available resources to continue the experiment, it will be closed
and the experimenter will be asked to create a new experiment whenever there is
availability.
2. If the experimenter wants to close the experiment before the time period ends, he/she could
send a stop request. Upon receiving it, the EaaS platform will automatically close the
experiment, together with the open tasks and corresponding reservations.
The final phase of an experiment is shown in Figure 47.
Figure 47 End of the experiment workflow
108
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
3.7. Template based experiment use case
In the Deliverable D1.3 [1] the experiment flow is roughly categorized into two patterns, “Template-based
experiment” and “Custom experiment”. Template-based experiments basically provide a preconfigured set
of resources (e.g. a set of sensors or a set of Virtual Machines). If the experimenters are interested in
analyzing data using the provided user interface, they are not required to construct any additional function.
However, if they implement several functions on IT resources to analyze data via the provided APIs, they
need to reserve IT resources and setup their analysis software. In both cases, template-based experiment
and custom experiment, resources reservation is not required prior to an experiment execution. In any case,
customized components depend on the experimenter’s implementation and require the reservation of the
resources where they are deployed.
The Erreur ! Source du renvoi introuvable. following table describes the use case.
Use case ID FESTIVAL Federated use case
Title Template-based experiment use case
Application
domain
Smart Shopping
Smart Energy
Smart Building
Used testbed(s) Federated testbeds including JOSE and FIWARE
Pre-conditions The experimenter is logged into FESTIVAL portal.
Scenario
description
1 Choose Experiment Type Experimenter chooses experiment type between “Template-based experiment” and “Custom experiment”.
(Main flow)
2 Choose Experiment Template If the experimenter chose “Template-based experiment”, he/she will choose template which he/she wants to use.
Templates will be provided as user interface or as documents describing the details of the experiment, resources involved in its execution and output data.
(Alternate flow)
2. Choose Customizing Target If the experimenter chose “Custom experiment”, he/she will choose customizing target between “Utilize SaaS API and customize only integration layer” (semi-custom experiment) and “Use customized components” (custom experiment).
2.1 Choose SaaS API If the experimenter chooses “Semi-Custom experiment”, he/she chooses SaaS API to be integrated by his/her own software component. And then he/she will proceed to step 2.3 for setting up their component into VMs. If
109
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
the experimenter chooses “Custom experiment”, as the same as “Semi-Custom” case, he/she chooses SaaS API and proceeds to step 2.2 for customizing sensor and actuator devices.
2.2 Choose sensor/actuator devices If the experimenter chooses “Custom experiment”, he/she will choose sensor/actuator devices for his/her experiment. This phase can be skipped if the experimenter uses already deployed sensor and actuator devices.
2.3 Customize Virtual Machine If the experimenter chooses “Custom experiment”, he/she will deploy his/her software components into Virtual Machines (IT resources).
Post-conditions Experiment setup completed.
Actors Experimenter
Table 25 Multi-domain use case on JOSE
An implementation example
“Custom experiment” requires that the software components developed by experimenters are deployed into
IT resources. It requires resource reservation and construction of their own experiment environment via EaaS
management APIs; therefore, the customized software deployment handling becomes a key point of EaaS
function. Figure 53 Figure 1describes an example of IT management flow in an experiment using JOSE
platform. JOSE has a possibility to host OpenStack and FIWARE instances and the related development by
ACUTUS is ongoing. Management API of the both two platform interfaces can be operated via Slice-based
110
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Federation Architecture (SFA) tools developed in FIRE project. The detail is described in section 2.2.1.3 but
any kind of required operation is assumed to be executed via that interface.
If an experimenter reuses ORION, Cygnus and IoT Agent for messaging among his/her sensor devices and
storages in OpenData Management System like CKAN, Generic Enablers provided by FIWARE can be utilized.
In this case, the experimenter instantiates those Generic Enablers via SFA API. Some Generic enabler provides
external interfaces to interact with CKAN, other storages; for example an experimenter can collect data from
sensor through ORION using NGSI 9/10 or M2M protocols (e.g. UL2.0/HTTP, MQTT, LWM2M/CoAP).
If software components required for the experiment is not provided as Generic Enabler, the experimenter
must construct his/her environment on Virtual Machines created on OpenStack platform. However, it is still
possible for the experimenter to reuse tools provided by FESTIVAL project for integrating several third parties
software components. For example, an environment monitoring application picked up in D1.3 [1] depends
on other open source software, e.g. mosquitto [30], fluentd [31] and ElasticSearch [32]. While the installation
and configuration procedure can be done by each experimenter because they are all open source software,
it is easier for them to setup the set of software by IT automation engine, e.g. Puppet [33], Chef [34], Ansible
[35] and so on. IT automation engine can implement installation and configuration procedures as a program
written in script languages. It is called DevOps [36] approach (for managing software components and
reducing operational cost for experimenters). The knowledge on software components federation should be
accumulated and implemented as scripts to be deployed into testbeds. As a result, the experimenters can
deploy part of required components by IT automation engines and the rest of the components by manually.
Which implementation will fit the experimenter’s requirement is a difficult problem to solve. For example,
some platforms are already integrated in FESTIVAL (e.g.: sensiNact, MQTT on PIAX, mosquitto, fluentd,
ORION with IoT Agent). The recommendation of each software components can also be documented. They
should be summarized as application template in a portal. The development of those templates is one of the
important tasks to establish user-friendly EaaS portal.
Figure 48 IT resource management use case on JOSE platform
111
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
4. Federation quality monitoring
4.1. FESTIVAL Key Performance Indicators
In Deliverable D4.2 [37], a set of QoS (Quality of Service) and QoE (Quality of Experience) KPIs is identified
within the evaluation framework that will help to evaluate the FESTIVAL federation of testbeds. The subjects
under evaluation (what is to be evaluated?) are testbeds or the federation, however in the current federation
architecture; the focus is only on KPIs that are evaluated on federation level. The feedback given by partners
collected by the circulated questionnaire of their available measurement tools (planned in D4.2 [37]) indicate
that few tools are available, especially for QoE KPIs and some KPIs can be better adapted/merged for reason
of using the same collection method and measurement value. The Table 26 shows the KPIs taken from D4.2
[37] that will be measured at federation level and with possible adaptations.
Id Indicator Description Measurement level
QoS-01 Resource
availability
if asked resources are able to be allocated to a
given experiment Federation
QoS-02,
06
Service
availability and
performance
Federation service availability such as success of
service discovery and allocation, performance
such as response time to requests. It keeps track
on a given experimenter/experiment
Federation
QoS-04 Application/Serv
ice throughput
The monitoring on application developed by
experimenters using federation services Federation
QoS-08 Application/Serv
ice reliability
Services and application's reliability which serves
as a statistic information of the federation and
individual testbeds
Federation and testbed
Table 26 QoS indicators
Id Indicator Description Measurement level
QoE-01 Ease of discovery
Qualitative perception of the experimenter of
the ease of discovery of the resources available
at federation level
Federation
QoE-02 Ease of
allocation
Qualitative perception of the experimenter of
the ease of allocation of resources available at
federation level
Federation
112
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
QoE-03 Ease of
deployment
Qualitative perception of the experimenter of
the ease of deployment of the experiment
(resources, timeline, set-up and run of the
experiment)
Federation and testbed
QoE-04-
5-6
Document
relevance and
availability
Qualitative perception of the experimenter of
the relevance and availability of documentation
(or lack of documentation) at federation level to
discover and set-up easily an experimentation
Federation and testbed
QoE-07 Availability of
tools
Percentage of frequently used tools against the total
number of tools Federation
QoE-08 Tools relevance Perceived percentage of useful tools against the total
number of available tools Federation
QoE-09-
16
Perceived lack of
tools
Number of experimentation tools and Estimation of
the number of missing tools by type Federation
QoE-10 Setup time Perceived relevance of the operation time to prepare
a certain experimentation Federation
QoE-11 Financial cost Financial cost of running experimentation Federation
QoE-12
Relevance of cost
according to
added value
Perception of adaptation of the cost of running the
experimentation with the perceived added value Federation
QoE-13
Access to end-
users
Perception of the access to a sufficient and relevant
pool of end-users Federation and testbed
QoE-14
Participation of
end-users
The number of end-users participated in
experimentations Federation and testbed
QoE-15
Feedback
gathering from
end-users
Existence of feedback tools Federation
QoE-16
Satisfaction to
experiment/applic
ation
Overall appreciation of the testbed, experiment,
application, or service Federation
QoE18 Data Storage Availability of data storage for experimenters Federation
Table 27 List of QoE indicators
113
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
4.2. Collection of KPI data
In order to calculate a correct value for each KPI at the federation/EaaS level, the following point
should be addressed:
- Define what data will be needed.
- How to collect them? Or who will provide them?
- When to collect them (if applicable)? At which frequency?
- How to aggregate them and analyse them (if applicable)?
In this section, the focus is put on data collection that correspond to the 3 first questions. Following the
general approach in D4.2 [37], the KPIs will be regrouped into 2 sets: QoS and quantitative QoE that will be
automatically collected by tools or via EaaS APIs, and qualitative QoE that will more interact with federation
users via the EaaS unique portal during experiment.
4.2.1. Automated KPI data collection
As mentioned just before, automatically collected data are measured for QoS KPIs that include the KPIs of ID
QoS-01, QoS-02, QoS-04 and QoS-08 listed in Table 26. An EaaS module “EaaS KPI monitoring” is proposed in
the architecture in order to centralize the KPI measurement features. This module interacts with other EaaS
modules. The general approach for KPI measurement collection is to implement a “probe” to related EaaS
module that raises the necessary data using EaaS API or other technologies (e.g. sniffer). These data are then
sent to the central EaaS KPI Monitoring module to be analyzed further. When authorized user needs to
visualize the KPI monitoring results, the request is also passed to the “EaaS KPI monitoring” module to get
the stored data about KPIs.
Figure 49 Quality probe
The following section will explain the needed data and how and where these data are collected for each of
the listed KPIs.
114
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
4.2.1.1. Resource availability
When experimenter plan and create an experiment, he wants to know for the given range of target
experiment period, which are the available resources and also if those available resources are useful and
enough for the experiment. The overall KPI will give the percentage of availability of one resource when it is
demanded by experiments, and the percentage of available resources when an experiment is being created.
What is expected to be measured? The status of each resource in a list of asked resources, more
specifically, the availability of each resource in a list of resources asked for an experiment.
How the expected data to be collected? EaaS APIs are available to get this information: get resource
status and get availability of resource (these two APIs can get the status of one or more resources)
together with the resource access manager which defines the accessible resources to a given
experimenter. EaaS modules involved in execution of these APIs that need to implement a probe are
also explained in Figure 49.
When or at what frequency for the measurements? The data are collected every time an experiment
is being created.
4.2.1.2. Service availability and performance
Federation service availability is the result of successful service discovery and allocation. Service performance
can for instance be measured such as response time to requests. It keeps track on a given
experimenter/experiment. High availability and performance of services results in serving the users quickly
and correctly. Following questions need to be answered:
What is expected to be measured? Who (experimenter/experiment) asked which resources and
when he got served with the asked resources.
How the expected data to be collected? A general log or experiment monitoring data will be needed
to be able to consult to get this information. This log will be most probably stored in the EaaS data
storage. The most relevant module to implement the probe will be the “Experiment monitoring”.
When or at what frequency for the measurements? The frequency of measurement will be the same
as the log taking frequency.
4.2.1.3. Application/Service throughput
Throughput or network throughput is the rate of successful message delivery over a communication channel
(definition from D4.2 [37]). EaaS application/service throughput aims to give an indicator of the throughput
of the EaaS services used for a given experiment.
What is expected to be measured? The throughput of a list of services that are involved in the
experiment.
How the expected data to be collected? Through the EaaS module “Experiment monitoring”, quantity
of messages exchanged between the EaaS modules and the experiment application for a given
experiment will be measured for the period of the experiment running time.
115
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
When or at what frequency for the measurements? The data will be collected regularly (frequency
to be defined) during the experiment.
4.2.1.4. Application/Service reliability
This KPI gives an indication of the reliability of federation and testbed level components when an
experimenter chooses resources and services for the planned experiment. This will serve as a statistic
information over a period of historical data.
What is expected to be measured? For each EaaS modules (i.e. storage service, resource discovery),
the ratio of failure time over total time. For the list of all resources, the ratio of failure time over total
EaaS running time. The total time will be fixed to a range of time (e.g. 1 week, 2 weeks).
How the expected data to be collected? This data will be collected through probes deployed inside
the EaaS components, data that is collected on a sliding window, meaning that oldest data recorded
are replaced by the newest data. The window size will to be defined according to the server
limitations.
When or at what frequency for the measurements? The frequency of measurement is the same as
the log taking frequency as the useful data will be found in log.
4.2.2. KPI data collection from the portal
4.2.2.1. KPI Survey
The QoE indicators will be collect through different ways within the portal after each important step done by
the experimenters. At this stage, the plan is to develop and implement different tools in the portal to gather
the feedback from experimenters:
- For some indicators, rapid feedback is pushed to let the experimenter the chance to indicate
immediately of something is getting wrong or if he’d expect having access to other features.
This rapid feedback will be implemented at the end of the corresponding webpage and include a field
enabling to rate the usefulness of information displayed and an open field for any feedbacks, as
shown in Figure 50
Figure 50 Rapid feedback from experimenters
This rapid feedback mechanism will be developed for QoE 1, 2, 3, 4, 5 & 6 listed in Table 27.
116
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
- The second set of tools consists in more detailed survey which questions are related to all other QoE
indicators. A first survey will be proposed to experimenters after the run of the experimentation and
will mainly focus on the feedbacks of the experimenters related to the experimentation set-up
(mainly QoE1 to QoE10). A second survey will be proposed at the end of the experimentation, with
a focus on feedback about the experimentation in itself, and the added value for the experimenter.
These two questionnaires will be developed in the next steps of the project, taking into account the
refinements of the uses cases and will be implemented online on the portal of the federation
4.2.2.1. Workflow for user feedback collection
The figure below shows when and how the feedbacks will be asked to the experimenters inside the workflow
of the experiment.
Figure 51 Account creation workflow
117
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 52 Experiment definition and QoE workflow
118
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
Figure 53 Workflow of the experiment with the feedback requests
119
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
5. Conclusion
This deliverable presented the final version of the FESTIVAL architecture. The definition of the architecture
has been challenging due to the fact that FESTIVAL is a unique project which has the ambitious goal of
federating resources which are not only heterogeneous but also of divers types such as open data, IoT data,
IT resources and living lab resources, which are physically distributed among Europe and Japan testbeds. In
the IoT domain interactions with the real-world and real end-users is essential for validation of research and
developments. All these specific requirements have been taken into account while designing the
architecture.
The different layers of the architecture has been described in detail. The architectural components forming
each layer has been specified and the APIs allowing to interact with them have been formalised. Non-
functional aspects such as security and quality of service monitoring have also been described.
The architecture has been designed to be adaptive and progressive for future considerations. Its core services
respond to the requirements that have been identified in the project. The service interfaces have been
defined in the shape of RESTful APIs that clearly states the capability of each service and the way of
interaction with them in a loosely coupling way, which facilitates the evolution of the architecture. This
loosely coupling allows any components of third parties to become easily compliant with FESTIVAL federation
platform in the future, allowing such third parties’ platform to be part of the federation proposed by the
FESTIVAL platform.
An example experimentation workflow has also been defined in this deliverable along with to illustrate a
typical experimentation scenario in FESTIVAL, as well as a mockup of the experimentation portal which will
be the one-stop-shop for IoT application/service experimenter. The portal will serve as an experimentation
workspace for the end-users, researchers, laboratory and institutions that would be interested in
experimenting their applications in a federated environment.
This deliverable marks the end of the Workpackage 1 which was about building the FESTIVAL architecture.
Implementing the functional blocks identified in this deliverable is the ongoing and future work of the
Workpackage 2.
120
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
6. References
[1] “D1.3,” [Online]. Available: http://www.festival-project.eu/wp-
content/uploads/2015/10/D1.3.InitialArchitecture-V1.pdf.
[2] “FESTIVAL, "D1.3 First Architecture"”.
[3] “What is Open Data?,” Open Knowledge Foundation, [Online]. Available:
http://opendatahandbook.org/guide/en/what-is-open-data/.
[4] “FESTIVAL, "D2.3 Open data guidelines and federated testbed policy"”.
[5] “SFA,” [Online]. Available: https://wiki.confine-project.eu/sfa:start.
[6] “Fed4Fire,” [Online]. Available: http://fed4fire-testbeds.ilabt.iminds.be/asciidoc/rspec.html.
[7] “SFA,” [Online]. Available: http://sfawrap.info/.
[8] “OpenStack,” [Online]. Available: https://www.openstack.org/.
[9] “JSON Service Description Language (JSDL),” [Online]. Available:
https://jsdl.codeplex.com/SourceControl/latest#- Specification/Specification.json.
[10] “OpenAPI Specification,” [Online]. Available: https://github.com/OAI/OpenAPI-
Specification/blob/master/versions/2.0.md.
[11] “OML,” [Online]. Available: http://oml.mytestbed.net/projects/oml/wiki/.
[12] “Kibana,” [Online]. Available: https://www.elastic.co/products/kibana.
[13] “SQL2011,” [Online]. Available:
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=53681.
[14] “ACID,” [Online]. Available: http://www.service-
architecture.com/articles/database/acid_properties.html.
[15] “AngularJS,” [Online]. Available: https://angularjs.org/.
[16] “Swagger,” [Online]. Available: http://swagger.io/.
[17] “RAML,” [Online]. Available: http://raml.org/.
121
This project has received funding from the European Union’s Horizon 2020 research
and innovation program under grant agreement 643275, and from the Japanese
National Institute of Information and Communications Technology
[18] “Fed4Fire safe cloud,” [Online]. Available: http://www.fed4fire.eu/ipcs4fire/.
[19] “Keystone,” [Online]. Available: http://docs.openstack.org/developer/keystone/.
[20] “Keyrock,” [Online]. Available: http://catalogue.fiware.org/enablers/identity-management-keyrock.
[21] “OpenIAM,” [Online]. Available: http://www.openiam.com/.
[22] “OpenIDM,” [Online]. Available: http://openidm.forgerock.org/.
[23] “oauth2,” [Online]. Available: http://oauth.net/2/.
[24] “OpenID,” [Online]. Available: http://openid.net/.
[25] “SAML,” [Online]. Available: https://wiki.oasis-open.org/security/FrontPage.
[26] “Google Identity Platform,” [Online]. Available: https://developers.google.com/identity/.
[27] “Facebook login API,” [Online]. Available: https://developers.facebook.com/docs/facebook-login.
[28] “Lets Encrypt,” [Online]. Available: https://letsencrypt.org/.
[29] “Collaborative Projects,” [Online]. Available: http://collabprojects.linuxfoundation.org/.
[30] “MQTT,” [Online]. Available: http://mqtt.org/.
[31] “FluentD,” [Online]. Available: http://www.fluentd.org/.
[32] “ElasticSearch,” [Online]. Available: https://www.elastic.co/products/elasticsearch.
[33] “Puppet,” [Online]. Available: https://puppetlabs.com/.
[34] “Chef,” [Online]. Available: https://www.chef.io/chef/.
[35] “Ansible,” [Online]. Available: https://www.ansible.com/application-deployment.
[36] “DevelopsTopologies,” [Online]. Available: http://web.devopstopologies.com/.
[37] “D4.2,” [Online]. Available: http://www.festival-project.eu/wp-content/uploads/2015/10/FESTIVAL-
D4.2-Evaluation-Framework-v1.0.pdf.
Top Related