D4.1: Service and Core KPI Specification · 2020. 5. 29. · D4.1: Service and Core KPI...

H2020 5Growth Project

Grant No. 856709

D4.1: Service and Core KPI

Specification

Abstract

The main objective of this deliverable is to provide a formal definition of the KPIs identified by the

5Growth project, with the purpose of evaluating the four different vertical pilots of the project on

Industry 4.0, Transportation, and Energy. To achieve so, this work defines the Core 5G KPIs as the

technical KPIs that can be directly measured on the use cases’ infrastructure. On the other hand,

Service KPIs are defined as the KPIs that allow validating an industrial 5G scenario from the vertical

point of view. In most cases, Service KPIs are not directly measurable, and this is the reason why this

deliverable also defines a formal mapping between Core 5G KPIs and Service KPIs. Furthermore, Core

5G KPIs, Service KPIs and their correspondent mapping are applied to the different 5Growth use

cases to define the parameters that need to be evaluated in order to ensure they fulfill the expected

requirements. Finally, the impact of COVID-19 on the planned KPI evaluation is analysed.

D4.1: Service and Core KPI Specification 2

H2020-856709

Document properties

Document number D4.1

Document title Service and Core KPI Specification

Document responsible Pedro Bermúdez García (TELCA)

Document editor Pedro Bermúdez García (TELCA)

Authors Diego R. López (TID), Sonia Fernández (TID), Luca Valcarenghi (SSSA),

Andrea Sgambelluri (SSSA), Francesco Paolucci (SSSA), Molka Gharbaoui

(SSSA), Koteswarao Kondepu (SSSA), Olivier Tilmans (NBL), Chrysa

Papagianni (NBL), Pedro Bermúdez (TELCA) Pablo Murillo (TELCA),

Sergio González Díaz (UC3M), Jorge Martín Pérez (UC3M), Jesús Alonso

(INNO), Josep Xavier Salvat (NEC), Giovanni Rigazzi (IDCC), Paulo Paixão

(EFACEC_S), Hugo Martins (EFACEC_E), Álvaro Gomes (ALB), Jorge

Carapinha (ALB), José Bonnet (ALB), Claudio Casetti (POLITO), Carla

Chiasserini (POLITO), Corrado Puligheddu (POLITO), Simone Panicucci

(COMAU). Chiara Napione (COMAU), Lucrezia Morabito (COMAU),

Ricardo Martínez (CTTC), Jordi Baranda (CTTC), Luca Vettori (CTTC),

Engin Zeydan (CTTC), Josep Mangues (CTTC), Lorenza Giupponi (CTTC),

Giada Landi (NXW), Juan Brenes (NXW), Daniel Corujo (ITAV), Paola

Iovanna (TEI), Giulio Bottari (TEI), Fabio Ubaldi (TEI), Ioannis Stavrakakis

(NKUA), Nancy Alonistioti (NKUA), Lina Magoula (NKUA), Nikolaos

Koursioumpas (NKUA).

Target dissemination level Public

Status of the document Final

Version 1.0

Delivery date May 31, 2020

Actual delivery date May 29, 2020

Production properties

Reviewers Andres Garcia-Saavedra (NEC), Diego San Cristóbal Epalza (ERC)

Disclaimer

This document has been produced in the context of the 5Growth Project. The research leading to

these results has received funding from the European Community's H2020 Programme under grant

agreement Nº H2020-856709.

All information in this document is provided “as is" and no guarantee or warranty is given that the

information is fit for any particular purpose. The user thereof uses the information at its sole risk and

liability.

For the avoidance of all doubts, the European Commission has no liability in respect of this document,

which is merely representing the authors view.


H2020-856709

Contents List of Figures ......................................................................................................................................................................... 4

List of Tables ........................................................................................................................................................................... 5

List of Acronyms ................................................................................................................................................................... 6

Executive Summary and Key Contributions ................................................................................................................ 8

1. Introduction. .................................................................................................................................................................... 10

2. Core 5G KPIs specification .......................................................................................................................................... 12

3. 5Growth Service KPIs identification ........................................................................................................................ 14

3.1. 5Growth SKPI goals ............................................................................................................................................... 15

4. Core 5G and Service KPI relationship ..................................................................................................................... 17

5. Considered KPIs per Pilot ........................................................................................................................................... 19

5.1. Industry 4.0 Pilot – INNOVALIA ........................................................................................................................ 19

5.1.1. Use Case 1: Connected Worker Remote Operation of Quality Equipment ............................. 19

5.1.2. Use Case 2: Connected Worker: Augmented Zero Defect Manufacturing (ZDM) Decision

Support System (DSS) .............................................................................................................................................. 26

5.2. Industry 4.0 Pilot – COMAU ............................................................................................................................... 31

5.2.1. Use Case 1: Digital Twin Apps ................................................................................................................... 31

5.2.2. Use Case 2: Telemetry/Monitoring Apps .............................................................................................. 33

5.2.3. Use Case 3: Digital Tutorial and Remote Support ............................................................................. 35

5.3. Transportation Pilot – EFACEC Engenharia e Sistemas ........................................................................... 38

5.3.1. Use Case 1: Safety Critical Communications ....................................................................................... 38

5.3.2. Use Case 2: Non-Safety Critical Communications ............................................................................. 41

5.4. Energy Pilot – EFACEC Energia ......................................................................................................................... 45

5.4.1. Use Case 1: Advanced Monitoring and Maintenance Support for Secondary Substation

MV/LV Distribution Substation ............................................................................................................................ 45

5.4.2. Use Case 2: Advanced Critical Signal and Data Exchange Across Wide Smart Metering

and Measurement Infrastructures ....................................................................................................................... 50

6. Conclusions ...................................................................................................................................................................... 55

7. References ........................................................................................................................................................................ 56

8. Appendix A – COVID-19 Impact Analysis on Plans and Roadmaps ........................................................... 57

9. Appendix B – Technical KPIs defined by 5G-PPP TMV-WG, 5G-EVE and 5G-VINNI ............................ 65


H2020-856709

List of Figures

Figure 1: INNOVALIA UC1 logical architecture [4] ................................................................................................. 20

Figure 2: INNOVALIA UC1 data plane logical architecture ................................................................................. 20

Figure 3: Measurement locations of the end-to-end latency Core 5G KPI for the teleworker-CMM

synchronization Service KPI. ........................................................................................................................................... 21

Figure 4: Measurement locations of the reliability Core 5G KPI for the teleworker-CMM

synchronization Service KPI. ........................................................................................................................................... 22

Figure 5: Measurement locations of the QoS Core 5G KPIs for the high-resolution real-time video

quality Service KPI. ............................................................................................................................................................. 23

Figure 6: Overview of Innovalia Use Case 2 .............................................................................................................. 27

Figure 7: Process followed by the CMM and AGV to inspect an industrial part ........................................ 27

Figure 8: AGV-edge control synchronization ........................................................................................................... 29

Figure 9: Architecture of COMAU UC1 ....................................................................................................................... 31

Figure 10: Architecture of COMAU UC2 .................................................................................................................... 34

Figure 11: Architecture of COMAU UC3 .................................................................................................................... 36

Figure 12: EFACEC_S UC1 Overview ............................................................................................................................ 39

Figure 13: EFACEC_S UC2 Overview ............................................................................................................................ 42

Figure 14: COMAU Use Case 1 Roadmap Delay due to COVID-19 ................................................................. 60




H2020-856709

List of Tables

Table 1: Core 5G KPI specification ............................................................................................................................... 12

Table 2: 5Growth Service KPI identification and definition ................................................................................ 14

Table 3 5Growth SKPIs used in 5GPPP Projects ...................................................................................................... 16

Table 4: Mapping between 5Growth Service KPIs and Core 5G KPIs ............................................................. 18

Table 5: INNOVALIA UC1 Mapping between Functional Requirements and Specific Service KPIs .... 25

Table 6: INNOVALIA UC2 Mapping between Functional Requirements and Specific Service KPIs .... 30

Table 7: COMAU UC1 Mapping between Functional Requirements and Specific Service KPIs............ 33



Table 10: EFACEC_S UC1 Mapping between Functional Requirements and Specific Service KPIs ...... 40

Table 11: EFACEC_S UC2 Mapping between Functional Requirements and Specific Service KPIs ...... 44

Table 12: EFACEC_E UC1 Mapping between Functional Requirements and Specific Service KPIs ...... 48

Table 13: EFACEC_E UC2 Mapping between Functional Requirements and Specific Service KPIs ...... 52

Table 14: Estimated COVID-19 Impact on INNOVALIA Use Case 1 ................................................................ 57

Table 15: Estimated COVID-19 Impact on INNOVALIA Use Case 2 ................................................................ 58

Table 16: Estimated COVID-19 Impact on EFACEC_S Use Case 1 .................................................................... 62

Table 17: Estimated COVID-19 Impact on EFACEC_S Use Case 2 .................................................................... 62

Table 18: Estimated COVID-19 Impact on EFACEC_E Use Case 1 .................................................................... 63

Table 19: Estimated COVID-19 Impact on EFACEC_E Use Case 2 .................................................................... 64

Table 20: Technical KPIs defined by the 5G-PPP TMV-WG ................................................................................ 65

Table 21: 5G-EVE KPIs as in 5G-EVE-D1.4 ................................................................................................................. 66


H2020-856709

List of Acronyms

AGV – Automated Guided Vehicle

API – Application Programming Interface

AR – Augmented Reality

CKPI – Core 5G KPI

CMM – Coordinate-Measuring Machine

CP – Control Plane

CPE – Customer Premises Equipment

CUPS – Control and User Plane Separation

DSS – Decision Support System

E2E – End-to-End

eMBB – enhanced Mobile Broadband

FR – Functional Requirements

I4.0 – Industry 4.0

IIoT – Industrial Internet of Things

KPI – Key Performance Indicator

LV – Low-Voltage

LX – Level Crossing

mMTC – massive Machine Type Communications

MTTR – Mean Time To Repair

NS – Network Service

RAN – Radio Access Network

SKPI – Service KPI

SLA – Service Level Agreement

SO – Service Orchestrator

TMV-WG – Test, Measurement and KPI Validation Working Group

UC – Use Case

UE – User Equipment

UP – User Plane

URLLC – Ultra-Reliable Low Latency Communication


H2020-856709

vEPC – virtual Evolved Packet Core

VIM – Virtualized Infrastructure Management

VM – Virtual Machine

VNF – Virtual Network Function

VPN – Virtual Private Network

VS – Vertical Slicer

ZDM – Zero Defect Manufacturing


H2020-856709

Executive Summary and Key Contributions

The main objective of this deliverable is to provide a formal definition of the KPIs identified by the

5Growth project, with the purpose of evaluating the four different vertical pilots of the project on

Industry 4.0, Transportation, and Energy.

The KPIs defined are divided into two categories: Core 5G KPIs and Service KPIs. The former KPIs are

defined as the technical KPIs that can be directly measured on the use cases’ infrastructure and

network. On the other hand, the latter KPIs are defined as the key performance indicators that allow

validating a business and industrial scenario from the vertical point of view.

The Service KPIs are identified taking into consideration the business and functional requirements of

the verticals gathered in D1.1 [1], with the objective of validating the correct operation of the 5G use

cases deployed using the 5Growth platform. However, due to their high-level definition, they are not

directly measurable over the use cases infrastructure. This is the reason why this work achieves to

define a mapping between Core 5G KPIs and Service KPIs, in such a way that by measuring specific

combinations of Core 5G KPIs, certain Service KPIs can be correlated.

Finally, the Service KPIs identified, along with their corresponding mapping to Core 5G KPIs, are

applied to each of the pilot use cases, with the goal of defining the KPIs that need to be fulfilled in

order to demonstrate that the performance delivered by the implementation of the different use

cases satisfy the expected requirements defined in D1.1 [1].

In summary, the main contributions of this deliverable can be listed as:

• Identification of 11 Core 5G KPIs, based on existing 5GPPP KPIs and KPIs defined by the

TMV-WG, 5G-EVE and 5G-VINNI.

• Definition of 11 general 5Growth Service KPIs.

• Mapping between Core 5G KPIs and Service KPIs.

• Application of the Service KPIs and their mapping with Core 5G KPIs to the 5Growth pilots,

obtaining:

o 12 specific Service KPIs applied to the INNOVALIA Pilot on Industry 4.0.

o 6 specific Service KPIs applied to the COMAU Pilot on Industry 4.0.

o 5 specific Service KPIs applied to the EFACEC_S Pilot on Transportation.

o 9 specific Service KPIs applied to the EFACEC_E Pilot on Energy.

• Mapping between the identified Service KPIs and the vertical functional requirements listed

in D1.1 [1] for each use case.

The KPIs identified above will be used as one of the fundamental inputs for task T4.2, mainly focused

on the methodology and tools for assessing these Service and Core 5G KPIs. Furthermore, they will

be used in the pilot validation trials of T4.3, T4.4, T4.5 and T4.6.

Regarding KPI assessment and evaluation, this work provides a roadmap that indicates when the

different KPIs defined will be measured. More specifically, it identifies for each vertical pilot which

KPIs will be reported first in D4.2 First validation and verification report in ICT-17 premises (milestone


H2020-856709

11, month 18 of the project) and which ones will be assessed in D4.4 Validation results in the light of

5Growth layer (milestone 18, month 27 of the project), taking into account the delays caused by the

COVID-19 impact.


H2020-856709

1. Introduction

This document is the result of the work developed under task T4.1 to identify Service KPIs and the

related Core 5G KPIs, with the purpose of evaluating the different vertical pilots of the 5Growth

project on Industry 4.0, Transportation, and Energy. In this work, Service KPIs are defined as the

qualitative indicators that allow validating a business and industrial scenario from the vertical point

of view. On the other hand, Core 5G KPIs are defined as the technical KPIs that can be directly

measured on the use cases’ infrastructure and network. In most cases, Service KPIs are not directly

measurable, and this is the reason why this deliverable also defines a mapping between Core 5G

KPIs and Service KPIs. In this way, combining different Core 5G KPIs (completely transparent for the

verticals), specific Service KPIs can be obtained, which allow verticals to determine whether the use

cases deployed using the 5G technology and the 5Growth platform fulfill the expected performance

and functional requirements.

One of the goals of 5Growth is to provide an automated and sharable 5G End-to-End solution that

will allow the vertical industries to achieve their respective key performance targets. For this reason,

it must be ensured, by leveraging the KPIs identified, that the 5G platform used by the verticals is

compliant with the vertical requirements and their expectations. Evaluating the performance of these

pilot validation trials will allow verifying whether the vertical applications can be considered 5G-ready

or not.

In order to identify and define Service KPIs that adjust as much as possible to the vertical

expectations, this work relies on the business, functional and technical requirements identified in

WP1 and gathered in D1.1[1]. Therefore, the evaluation of the Service KPIs here identified will directly

validate whether the vertical requirements are satisfied.

In relation to other tasks of the project, the KPI definition provided in this document will be used as

one of the fundamental inputs for task T4.2, mainly focused on the methodology and tools for

assessing these Service and Core 5G KPIs. Furthermore, the lists of selected KPIs per use case, along

with the methodology and tools defined in T4.2 to measure these KPIs, will be used in the pilot

validation trials of T4.3, T4.4, T4.5 and T4.6, whose objective is to validate the correct operation of

the different use cases defined under the 5Growth scope.

Going a step further and not only restricted to the 5Growth project scope, the 5Growth Service KPIs

have been defined in a general manner, with the purpose of being applicable to other business and

industrial 5G scenarios, which may be evaluated and validated leveraging the Service KPIs defined

under this work. In a similar manner, the 5Growth Service KPIs may be extrapolated to define realistic

Service SLA parameters that are more suitable to the vertical perspective.

The document is structured as follows.

Section 2 provides a list of identified Core 5G KPIs. As mentioned, these KPIs are technical KPIs that

are directly measurable in the use cases’ infrastructure and network. They are based on existing

5GPPP KPIs, the KPIs defined by the 5G-PPP Test, Measurement and KPI Validation Working Group

(TMV-WG) and the KPIs that can be evaluated by the ICT-17 platforms 5G-EVE and 5G-VINNI.


H2020-856709

Section 3 provides the definition of a set of general 5Growth Service KPIs, which are the KPIs that

evaluate 5G business scenarios from the verticals point of view. These KPIs can be extrapolated to

different 5G industrial scenarios and leveraged to define vertical SLAs.

Section 4 describes how the evaluation of Service KPIs can be performed by measuring a specific

combination of Core 5G KPIs, providing the mapping between the Core 5G KPIs defined in Section 2

and the general 5Growth Service KPIs identified in Section 3.

Section 5 contains the sets of specific Service KPIs selected to evaluate each of the use cases of the

different 5Growth pilots. That is, the general Service KPIs defined in Section 3 are applied to each of

the use cases, explaining how they validate their correct operation and guaranteed performance.

Furthermore, the mapping between the selected Service KPIs and the correspondent Core 5G KPIs is

described, as well as where to measure each of the Core 5G KPIs within the use case architecture.

Additionally, in order to demonstrate that the Service KPIs evaluate accurately the vertical functional

requirements collected in D1.1 [1], a table containing the mapping between Service KPIs and vertical

functional requirements is provided for each of the use cases.

Finally, an analysis of the impact of the COVID-19 on the planned roadmaps for the KPI assessment

is provided in Appendix A – COVID-19 Impact Analysis on Plans and RoadmapsMore in detail, this

analysis identifies for each vertical pilot the KPIs that will be included in the first KPI assessment

report of the project (D4.2 First validation and verification report in ICT-17 premises, milestone 11,

month 18) and those that will be included in the second (D4.4 Validation results in the light of 5Growth

layer, milestone 18, month 27), taking into consideration the availability of the ICT-17 and 5Growth

platforms.


H2020-856709

2. Core 5G KPIs specification

The 5G-PPP Test, Measurement and KPI Validation Working Group (TMV-WG) defined in the White

paper [2] presented at EuCNC 2019 some initial KPIs. This work is the outcome of the collaboration

of many 5G-PPP Phase 1 and Phase 2 projects within the TMV-WG. Details of the specified KPIs are

reported in Appendix B – Technical KPIs defined by 5G-PPP TMV-WG, 5G-EVE.

Moreover, the two ICT-17 projects whose infrastructures will be exploited by 5Growth, namely 5G-

EVE and 5G-VINNI, identified KPIs to be collected in their experimental platforms. The KPIs

considered by 5G-EVE and 5G-VINNI are reported for easy consultation in Appendix A.

Based on the aforementioned KPIs, 5Growth identified a set of Core 5G KPIs. Core 5G KPIs are

performance parameters related to the network and system components of the communications

infrastructure that constitutes the industrial pilots. The core KPIs are the KPIs that contribute to the

service KPIs and that are mainly measured at the network level.

TABLE 1: CORE 5G KPI SPECIFICATION

CKPI Id CKPI Name CKPI Description Units

CKPI-1 End-to-end

Latency

Aggregation of one-way time delays measured

between specific components of the logical

architecture of the use case.

ms

CKPI-2 Packet Loss The number of packets that fail to reach their

destination, measured in specific interfaces of the use

case logical architecture.

%

CKPI-3 Guaranteed Data

Rate

The data rate is the number of bits per unit of time

sent over a specific interface of the use case logical

architecture. The guaranteed data rate is the minimum

expected data ratefor the overall use case to function

correctly.

Mbits/s

CKPI-4 Coverage Radio access coverage area on the pilot premises. m²

CKPI-5 Availability Percentage of time during which a specific component

of the use case (application, server, network function,

etc.) is responding to the requests received with the

expected QoS requirements. That is, it is the ratio

between the up-time of a specific component over the

total time the component has been deployed.

%

CKPI-6 Slice

Creation/adaption

Time

Time elapsed since the creation/reconfiguration of a

slice is triggered until the slice is fully operational.

ms

CKPI-7 Connection

Density

The number of users/devices that can be connected

simultaneously to the use case network infrastructure

without degrading the performance of the

users/devices that are already connected.

1/m²


H2020-856709

CKPI-8 Data Volume The total quantity of information transferred over a

given interface during specific use case operations,

measured in bits.

Gbits

CKPI-9 Jitter Variation of the end-to-end latency forthe

communications between specific components of the

use case. This core KPI is useful to correlate QoE KPIs

for the different video visualizations performed in the

use cases.

ms

CKPI-10 Received Radio

Signal Quality

The quality of the received signal, as an abstract value

compounding multiple low-level signal properties (e.g.,

power, SNR, synchronization).

%

CKPI-11 Buffer occupancy The amount of user-data pending transmission (i.e.,

submitted by applications and waiting in transmission

buffers).

bytes

Finally, within 5G-PPP TMV-WG, Phase II and Phase III projects are currently working together

towards a common approach of mapping 5G core/network KPIs to vertical/service KPIs.


H2020-856709

3. 5Growth Service KPIs identification

To build an infrastructure capable of meeting 5G requirements, it is necessary to describe a set of

service KPIs (SPKIs) that gather the services’ needs. Upon such a description, 5Growth can measure

whether a 5G service deployment is satisfying the expected behavior or not. In this section, we

provide a list of SKPIs that do not only describe the KPIs of those services tackled in 5Growth, but

the KPIs encountered by any service using the 5G technology. Table 2 gives a detailed description of

those SKPIs identified by 5Growth.

TABLE 2: 5GROWTH SERVICE KPI IDENTIFICATION AND DEFINITION

SKPI Id SKPI Name SKPI Description

5GR-SKPI-1 Service Setup Time Time elapsed since the vertical requests a specific

service until the service is completely operational. This

time comprises the provisioning and configuration of

the resources needed for the use case.

5GR-SKPI-2 Synchronization

between

Communication

Components

Synchronization between components evaluates the

ordered and timely execution of operations requested

among components. Out-of-order operation requests

must be near zero in real-time communications

among the involved components.

5GR-SKPI-3 Device Mobility Capacity of the mobile devices of a given use case to

move within the area of applicability of the service

without degrading its performance, independently of

the devices speed.

5GR-SKPI-4 High-resolution Real-

time Video Quality

Specific use cases are supported by a real-time video

streaming service whose high quality resolution is

critical for the correct operation of the vertical service.

This SKPI evaluates the QoE of the users consuming

the video feed that supports a given use case. E.g., in a

I4.0 context, the high-resolution video service is

needed to perform accurate measurements of

industrial pieces remotely and in real-time.

5GR-SKPI-5 Radius of Operation Geographical area of the service applicability. That is,

this SKPI determines wether the service

implementation is able to deliver a correct operation

within a city, a country, a continent or worldwide. All

the requirements of a specific use case are fulfilled for

any device connecting in the area of applicability

specified by the deployed service.

5GR-SKPI-6 Integrated Multitype

Communications

Integration of multiple types of communication and

protocols (e.g., video, voice, control, etc.) with different

performance requirements. This SKPI evaluates that

their interoperability does not degrade the overall


H2020-856709

service performance and meets the same

requirements as met when operating independently.

5GR-SKPI-7 Extensive Network

Coverage in Vertical

Premises

Capability of the elements and actors of a use case

located at different places of the vertical premises to

access the 5G network from anywhere without altering

the performance of the service. This SKPI evaluates

whether the 5G network access is provided in the

complete vertical premises guaranteeing the minimum

expected performance of each specific service.

5GR-SKPI-8 Service Operation Time The service operation time is the minimum time

needed to perform one iteration of the complete

service workflow. E.g., in an industry 4.0 context, the

time that it takes to scan an industrial piece, process

the information and display the results to a QA

operator.

5GR-SKPI-9 Service Operation

Capacity

Number of iterations of the service workflow that are

being performed simultaneously at a specific instant

of time.

5GR-SKPI-10 Service Availability Percentage of time during which the overall vertical

service is working correctly and meeting the expected

vertical requirements.

5GR-SKPI-11 Service Reaction Time Time required by the service to react to changes or

external events. This time comprises any possible run-

time reconfigurations of the service.

3.1. 5Growth SKPI goals The goal of 5Growth is to demonstrate with the Pilots that it is possible to build real experiments

capable of meeting the SKPIs of Table 2. Such compliance will suppose a significant impact on the

Pilots of this project. Section 5 shows which KPIs are present in each of the pilots, and how they relate

to their applications.

5Growth goals do not deal with unrealistic SPKIs, Table 3 provides a quick glimpse of which SPKIs

are actually used in other 5GPPP projects. Showcasing that 5Growth can meet them, implies that

existing research can take benefit of the project work for validation purposes.


H2020-856709

TABLE 3 5GROWTH SKPIS USED IN 5GPPP PROJECTS

5GR-SKPI

1 2 3 4 5 6 7 8 9 10 11

5G VE √ √ √ √ √ √ √ √ √ √ √

5GENESIS √ √ √ √ √ √ √ √ √ √

5G VINNI √ √ √ √ √ √ √

5G!DRONES √ √ √ √ √ √ √ √ √ √

5G HEART √ √ √ √ √ √ √ √ √

5G SMART √ √ √ √ √

5G

SOLUTIONS √ √ √ √ √ √ √ √

5G TOURS √ √ √

5G VICTORY √ √ √ √ √ √ √ √ √

The definition of these SKPIs does not only map to the ongoing work of 5GPPP research projects but

serves as a reference to evaluate the performance of first-phase deployments of 5G business use

cases. For instance, Table 3 might serve as requirements for verticals once they ask to instantiate

their services.

For example, the “mobile broadband and media everywhere” use case presented in [3] requires that

there is mobile broadband in crowded areas like concerts. To realize whether the infrastructure owner

is providing an appropriate service to a vertical, such as a Broadcaster, compliance 5Growth SKPIs

{1,4,5,9,10} would imply that the vertical clients will experience the expected QoS. Consequently, SLAs

will be met as long as the satisfaction of the SKPIs is ensured.


H2020-856709

4. Core 5G and Service KPI relationship

As said before, SKPIs determine those business and operational-oriented metrics and figures of merit

to be fulfilled certifying that the vertical services, once deployed, are perfectly operative (i.e., they

work as expected to satisfy the demanded vertical requirements and agreed SLAs). In a nutshell, the

SKPIs are considered as the guarantees that the underlying management entities promise to the

verticals.

The set of SKPIs for each service deployed is a set of measurable, tangible, and quantifiable

parameters being exclusively relevant and/or meaningful for that specific service. Thereby, the set of

SKPIs for a given service may be orthogonal (not relevant) to other vertical services even if both co-

exist and are rolled out over a common (compute and network) infrastructure. The vertical demands

specify such SKPIs that are processed by the 5Growth VS. The latter then derives a pool of CKPIs

bound to the specific compute and networking resources to be allocated for accommodating the

vertical service. Indeed, recall that CKPIs provide measurable performance parameters associated

with the network and compute infrastructure supporting the functions, applications, connectivity,

etc. required by the vertical service.

Therefore, an unambiguous relationship is made between an SKPI for a vertical service and its

determined set of CKPIs. Those CKPIs can be measured, retrieved, and assessed over different

locations (e.g., edge DC, terminal, etc.), data and control interfaces, functions, systems, and devices

related to the deployed vertical service. To do this, the 5Growth platform (i.e., VS, SO and RL) takes

over of assessing the defined CKPIs for each specific SKPI. As an example, let us assume that a vertical

service demands an SKPI requiring correct synchronization between two remote endpoints (e.g.,

worker and machine). Upon deploying the service (i.e., allocating the resources within the

infrastructure), specific monitoring agents will be used to export selected metrics/monitored info.

Such info will be used to derive a CKPI that allows assessing that the incurred latency contributions

from each element/segment constituting the end-to-end communication between the teleworker

and the remote machine is satisfied.

In the above example, it is important to outline that the vertical service remains completely agnostic

to the set of required CKPIs to assess the targeted SKPI. For the vertical services forming the targeted

pilots to be conducted in 5Growth, a list of SKPIs has been identified (see Section 3). These SKPIs are

decomposed using the CKPIs described in Section 2). Table 4 aims at providing the required CKPI

metric of every SKPI Observe that each SKPI presents its own set of CKPIs. For instance, 5GR-SKPI-10

(i.e., Service Availability) is one-to-one related to 5GR-CKPI-5 (i.e., Availability). On the other hand,

5GR-SKPI-7 (i.e., Extensive Network Coverage in Vertical Premises) needs to be assessed through 6

CKPIs as reflected in Table 4.


H2020-856709

TABLE 4: MAPPING BETWEEN 5GROWTH SERVICE KPIS AND CORE 5G KPIS

5GR-SKPIs Core 5G KPIs

CKPI-1 CKPI-2 CKPI-3 CKPI-4 CKPI-5 CKPI-6 CKPI-7 CKPI-8 CKPI-9 CKPI-10 CKPI-11

5GR-SKPI-1 X X

5GR-SKPI-2 X X

5GR-SKPI-3 X X X X 5GR-SKPI-4 X X X X

5GR-SKPI-5 X X

5GR-SKPI-6 X X X

5GR-SKPI-7 X X X X X X

5GR-SKPI-8 X X

5GR-SKPI-9 X X X

5GR-SKPI-10 X

5GR-SKPI-11 X


H2020-856709

5. Considered KPIs per Pilot

The following subsections are dedicated to the 5Growth pilots. First, an overview of each use case is

given, detailing their components and logical architecture. Second, a set of 5Growth Service KPIs

from Section 3 is selected to evaluate each of the different use cases. That is, the 5GR-SKPIs are

applied specifically in order to explain how they validate their correct operation and expected

performance. Then, the specific Service KPIs are decomposed in several Core 5G KPIs using the

mapping provided in Section 4, with the purpose of detailing how the specific Service KPIs can be

evaluated by measuring the correspondent 5G Core KPIs in specific components and interfaces of

the use case logical architecture. And finally, a table containing the mapping between functional

requirements specified in D1.1 [1] and the specific Service KPIs identified in this deliverable is

provided, highlighting how the application of Service KPIs to the different use cases evaluate the

fulfilment of the vertical requirements.

5.1. Industry 4.0 Pilot – INNOVALIA

5.1.1. Use Case 1: Connected Worker Remote Operation of Quality Equipment

The first INNOVALIA use case involves the remote operation of an industrial machine, called

Coordinate-Measuring Machine (CMM). More specifically, as depicted in Figure 1 [4], an expert from

INNOVALIA located at the headquarter controls the movement of the CMM in a remote location

using a virtual joystick, while receiving visual information through low-latency video stream. Together

with the CMM, an edge device that runs its software controller that translates the movement

commands and the video equipment to record, compress and stream the images are deployed in

the remote location.

Therefore, an end-to-end 5G connection is needed between the remote worker and the INNOVALIA

premises, in order to guarantee the communication requirements of the different elements that

compose the use case.


H2020-856709

FIGURE 1: INNOVALIA UC1 LOGICAL ARCHITECTURE [4]

More in detail, Figure 2 shows the logical architecture of the use case data plane. It specifies the

different data plane components, their connections/interfaces, and the 5G network slices through

which they communicate.

Some elements of the architecture can be highlighted. First, two non-permanent slices (they are

instantiated on-demand by the teleworker only while the remote operation takes place and

terminated on the contrary) are used. Namely, one eMBB slice for the video streaming service and

one URLLC slice for the control of the CMM movements. Second, the M3Box is the edge device that

runs the software components that receive the movements produced by the virtual joystick and

translate them into commands compatible with the CMM controllers, which subsequently execute

the actual movements of the CMM.

FIGURE 2: INNOVALIA UC1 DATA PLANE LOGICAL ARCHITECTURE

The following Service KPI subsections will refer to these architectures with the purpose of relating

the different Service-to-Core 5G KPI mappings with the specific interfaces and elements of the use

case.


H2020-856709

5.1.1.1. P1UC1-SKPI-1: Teleworker-CMM Synchronization

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-2 Synchronization

between Communication Components.

For the correct operation of the use case, whenever the teleworker executes a given movement of

the CMM through the virtual joystick, he must immediately visualize the movement performed on

the display. From the vertical perspective, this teleworker-CMM real-time synchronization is critical

to validate use case. On the contrary, an excessive delay between the movements' execution and

their visualization may disrupt the operation of the teleworker, even causing the crash of the optical

sensors with the industrial piece being measured.

As stated in Section 4, the 5GR-SKPI-2 is mapped to the technical KPIs CKPI-1 End-to-end Latency

and CKPI-2 Packet Loss.

More specifically for this use case, the overall teleworker-CMM-teleworker delay that characterizes

the synchronization of the remote operation can be split into the following one-way network

latencies and component’s processing times (see Figure 3):

Virtual Joystick + UC1-URLLC-if1 + M3Box + UC1-URLLC-if2/3 + CMM controller + UC1-

URLLC-if4/5 + CMM + UC1-eMBB-if2/3 + Video Server + UC1-eMBB-if1 + Display.

FIGURE 3: MEASUREMENT LOCATIONS OF THE END-TO-END LATENCY CORE 5G KPI FOR THE

TELEWORKER-CMM SYNCHRONIZATION SERVICE KPI.

Additionally, the reliability of the URLLC interfaces handling the connection between the virtual

joystick and the CMM has a notable importance, considering that an excessive packet loss rate may

impact the smoothness of the CMM movements. Therefore, the CKPI-2 must be measured in the

following interfaces (see Figure 4):

UC1-URLLC-if1, UC1-URLLC-if2, UC1-URLLC-if3, UC1-URLLC-if4 and UC1-URLLC-if5.


H2020-856709

FIGURE 4: MEASUREMENT LOCATIONS OF THE RELIABILITY CORE 5G KPI FOR THE TELEWORKER-CMM

SYNCHRONIZATION SERVICE KPI.

5.1.1.2. P1UC1-SKPI-2: High-resolution Real-time Video Quality

This Service KPI is directly applied to this use case from the 5GR-SKPI-4 High-resolution Real-time

Video Quality.

The teleworker must be provided with a live video stream that displays the industrial piece to be

measured and the optical sensors of the CMM. Regarding its requirements, the video must have a

resolution high enough to visualize exact shapes and limits in order to perform measurements with

the needed accuracy and error tolerance. Moreover, the real-time video streaming service must be

continuous, as sudden interruptions and stalling events may cause collisions between the sensitive

sensors and the industrial piece.

According to Section 4, this specific service KPI is mapped to some of the technical KPIs identified in

Section 2: CKPI-2 Packet Loss, CKPI-3 Guaranteed Data Rate, CKPI-8 Data Volume and CKPI-9 Jitter.

These KPIs measure the Quality of Service (QoS) performance of the video streaming service of the

use case, that is, based on the logical data plane architecture (Figure 2), they will be evaluated in the

following components and interfaces (see Figure 5):

Display, UC1-eMBB-if1, Video Server, UC1-eMBB-if1 and UC1-eMBB-if3.


H2020-856709

FIGURE 5: MEASUREMENT LOCATIONS OF THE QOS CORE 5G KPIS FOR THE HIGH-RESOLUTION REAL-TIME

VIDEO QUALITY SERVICE KPI.

Once obtained, these technical KPIs will be extrapolated to evaluate the Quality of Experience (QoE)

of the teleworker, to validate the video component of the use case from the vertical perspective.

5.1.1.3. P1UC1-SKPI-3: Service Setup Time

The 5GR-SKPI-1 Service Setup Time can be directly applied to the first INNOVALIA use case. In this

case, whenever the teleworker needs to control the CMM remotely, he is the actor that initiates the

instantiation of the complete vertical service, in order to start the calibration of the CMM. The time

elapsed since the request from the teleworker until the CMM is ready to be remotely controlled is

essential to validate the use case and must not be longer than five minutes, according to the

requirements from INNOVALIA. Excessive delays in this Service Setup Time may cause the

dissatisfaction of INNOVALIA’s clients, as in the manufacturing sector stalling events of even seconds

can incur notable high costs.

This vertical KPI can be extrapolated from the following technical Core 5G KPIs: CKPI-5 Availability

and CKPI-6 Slice Creation Time.

In relation with CKPI-6 Slice Creation Time, two non-permanent 5G network slices are dynamically

created during the vertical service instantiation, one eMBB and one URLLC (see Figure 2). The CKPI-

6 evaluates the time it takes to create these two slices. More specifically, the instantiation and

configuration of the 5G virtual functions that enable the communication among the use case

components, that is, the UPF/vEPG and the 5G Core control plane functions (see Figure 1).

Additionally, CKPI-5 Availability evaluates whether the virtual applications of the vertical use case are

operational at a given point of time. In this case, as stated in Section 5.1.1, these applications are the

displays, the virtual joystick, the video server, and the M3Box (see Figure 2).

By combining these two technical KPIs (that is, measuring the time taken by the 5Growth platform

to establish connectivity between the vertical applications and identifying when the vertical

applications are available) the P1UC1-SKPI-3 Service Setup Time can be extrapolated.


H2020-856709

5.1.1.4. P1UC1-SKPI-4: Radius of Operation

This specific Service KPI is applied to the use case from the 5GR-SKPI-5 Radius of Operation, and it

refers to the maximum end-to-end distance of applicability of the vertical service. From the

INNOVALIA requirements gathered in D1.1 [1], the correct operation of the service needs to be

guaranteed in a radius that comprises distances from Bilbao to any factory within Europe, due to

their headquarters are located in Bilbao and they have clients distributed at different locations within

Europe.

From Section 4, it can be noted that this Service KPI is mapped to the technical KPIs CKPI-1 End-to-

end Latency and CKPI-5 Availability.

Several mechanisms will be applied to validate the use case at different geographical locations,

measuring the variations of the CKPI-1 End-to-end Latency, in order to evaluate whether the use case

is operational (in close relation with P1UC1-SKPI-1 Teleworker-CMM synchronization) independently

of the location.

Moreover, the evaluation of CKPI-5 Availability is needed to determine that the vertical applications

are completely operational at the different locations where they are instantiated. The combination

of both Core 5G KPIs, that is, the increase of the end-to-end latency until the vertical applications are

not available anymore, will establish the limit of the area of the vertical service applicability.

5.1.1.5. P1UC1-SKPI-5: Integrated Multitype Communications

The general 5GR-SKPI-6 Integrated Multitype Communications is directly applied to this use case. The

vertical is consuming and producing two types of traffic at the same time, video (CMM movements)

and data (CMM control commands). Even though they use two different 5G network slices, they both

are transmitted through the same physical channels (see Figure 1). However, their separated

performance (with very different requirements) must be guaranteed, ensuring their isolation and that

they are not affected by other types of communication. Additionally, the integration of multitype

communications must be transparent for the vertical, which must not notice any performance

degradations of the overall vertical service caused by the network.

P1UC1-SKPI-5 Integrated Multitype Communications is decomposed into the following Core 5G KPIs:

CKPI-2 Packet loss, CKPI-3 Guaranteed data rate and CKPI-5 Availability.

This composition is given because the availability of the virtual service applications must be

guaranteed despite the complexity added to the 5G network to support the aggregation and

differentiation of multitype traffic. In order to achieve this, the CKPI-2 Packet loss and CKPI-3

Guaranteed data rate must be measured for each type of traffic in those links where the traffic flows

aggregated so that it can be validated that the CKPIs of one type of data (e.g. video) does not affect

the CKPIs of the other (e.g. CMM commands). By looking at Figure 1, the traffic aggregation points

would be the 5G radio antennas, and the links where the traffic is transmitted aggregated are the

links between the antennas and the UPFs, and the network path between both UPFs.


H2020-856709

5.1.1.6. P1UC1-SKPI-6: Extensive Network Coverage in the Factory Premises

Both the CMM and the Video Server must be provided with a reliable and constant 5G radio access

connection while the CMM is being remotely operated. That is why it is important to apply the 5GR-

SKPI-7 Extensive Network Coverage in Vertical Premises to this use case. Depending on the

manufacturing process of each INNOVALIA customer, the CMM may be portable to different

locations inside the factory premises. In addition, providing a reliable wireless access connection may

be made more complicated due to the specific wall materials and medium conditions of the industry

environment.

With the purpose of evaluating this Service KPI, the following technical KPIs will be measured: CKPI-

1 End-to-end latency, CKPI-2 Packet loss, CKPI-3 Guaranteed data rate, CKPI-4 Coverage and CKPI-5

availability. More specifically and regarding Figure 1 and Figure 2, these 5G Core KPIs must be

evaluated in the radio link that connects the 5G antenna with the M3Box and the Video Server. On

the other side of the connection, the same KPIs can be validated in the 5G access link provided to

the teleworker, namely the display and the virtual joystick.

5.1.1.7. Mapping between Functional Requirements (FR) and Specific Service KPIs.

This section includes the direct relationship between the functional requirements gathered in D1.1

[1] and the specific Service KPIs applied to the use case. In this way, the reader can directly observe

the Service KPIs that will be evaluated in order to validate the fulfilment of the correspondent pilot

functional requirements.

TABLE 5: INNOVALIA UC1 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS

UC Functional Requirements [1] UC Service KPIs

FR-P1UC1-01 The factory manager is the one that

initiates the process

Qualitative verification

FR-P1UC1-02 Security measures to avoid

unauthorised access to the CMM


FR-P1UC1-03 Usable interface for the Metrology

Expert (virtual joystick)

P1UC1-SKPI-1 – Teleworker-CMM

Synchronization

FR-P1UC1-04 Real time video streaming to see the

movement of the CMM

P1UC1-SKPI-2 – High-resolution Real-

time Video Quality

FR-P1UC1-05 Enable 5G connectivity to the M3Box

edge device

P1UC1-SKPI-6 – Extensive Network

Coverage in the Factory Premises

FR-P1UC1-06 Security of the data interchange

(encryption)


FR-P1UC1-07 Radius of action – From Bilbao to

Europe

P1UC1-SKPI-4 – Radius of Operation

FR-P1UC1-08 The industrial environment must be

equipped with 5G-enabled

devices


FR-P1UC1-09 High availability of the remote

connection including redundant

5GR-SKPI-10 – Service Availability


H2020-856709

connection in order to guarantee the

communication in case of

failure

FR-P1UC1-10 Solution shall include additional

capabilities (e.g., specific

features, probes) that will take care of

the monitoring process


FR-P1UC1-11 Solution must support

programmability and virtualization to

enable flexible and dynamic

reconfiguration

Qualitiative verification

FR-P1UC1-12 The solution must guarantee proper

levels of isolation (in the

form of network slice)

P1UC1-SKPI-5 – Integrated Multitype

Communications

FR-P1UC1-13 Simultaneous support of integrated

communications (voice,

data, etc)

P1UC1-SKPI-5 – Integrated Multitype

Communications

5.1.2. Use Case 2: Connected Worker: Augmented Zero Defect Manufacturing

(ZDM) Decision Support System (DSS)

The second INNOVALIA use case aims at semi-automatizing the different stages during the quality

control process. At first, the components used during the metrology process, i.e., the

M3ExecutionEngine, RobotLink, and Data Assembler, will be deployed on an edge-cloud server.

On the one hand, this will enable to develop a Machine-2-Machine (M2M) communication protocol

between an Automated Guided Vehicle (AGV), which carries the specific parts to the Coordinate

Measuring Machine (CMM) for quality testing, and the CMM itself, so that the CMM can start loading

the exact measuring program. This would save valuable time to the quality control workers, as they

would not have to wait for the pieces to start loading the program. On the other hand, while the scan

of a piece takes place, the measurement data is going to be transmitted from the CMM to a Multi-

access Edge Computing (MEC) device which will enable the quality control manager to visualize the

results of the scan in a portable device. That is, there will be cloud support for computational

geometry processing and plotting the measured data. Figure 6 illustrates the complete process.


H2020-856709

FIGURE 6: OVERVIEW OF INNOVALIA USE CASE 2

In summary, this Use Case aims to showcase new collaboration capabilities between systems enabled

by a 5G edge infrastructure, by combining together an edge-cloud deployment of INNOVALIA

software components M2M communications.

5.1.2.1. P1UC2-SKPI-1: Service Operation Time

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-8 Service

Operation Time.

This use case develops a M2M protocol between the AGV and the CMM. This will enable the CMM

to start loading the program for measuring a specific part before the AGV has taken a specific part

to the measuring site. Furthermore, once the inspection finishes the measurements processing and

plotting and the AGV part transportation back to production runs also simultaneously. This service

KPI measures the impact of using 5G on the time of the complete operation, depicted in Figure 7.

FIGURE 7: PROCESS FOLLOWED BY THE CMM AND AGV TO INSPECT AN INDUSTRIAL PART

This KPI is mapped to CKPI-2 Packet Loss and CKPI-3 Guaranteed Data Rate. We can measure the

time for each of the stages of the AGV and CMM as adding up the individual times of each stage.

Once we have the timestamps, we are going to evaluate which ones are simultaneously executed so


H2020-856709

that we can measure the total operation time. These timestamps can also be used to predict how

many times a day we can carry out this operation and evaluate the time gains of using 5G in the

whole process.

5.1.2.2. P1UC2-SKPI-2: Service Operation Capacity

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-9 Service

Operation Capacity.

The service operation capacity measures how many workflows can now be run simultaneously in the

use case. As this use case enables some of the operations between the AGV and the CMM to run

concurrently, it is key to measure what are the gains of these parallel operations. Moreover, it also

enables to predict how many measures could be done during a certain amount of time. This KPI is

mapped to CKPI-1 End-to-end latency, CKPI-3 Guaranteed data rate and CKPI-5 Availability. Beyond

measuring the savings for each for the phases that are executed simultaneously on the overall

execution time, these CKPIs also let us measure the capacity that is required to sustain one workflow,

and eventually takes into account possible downtime on the total capacity in a given timeframe.

5.1.2.3. P1UC2-SKPI-3: Service Creation Time

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-1 Service Setup

Time.

This use case deploys the M3ExecutionEngine, RobotLink, and Data Assembler in an edge-cloud

server to carry out the operations between the AGV and the CMM. This SKPI evaluates the impact of

this by measuring the time needed to deploy and start the components before the service is

considered operational. This KPI is mapped to CKPI-5 Availability and CKPI-6 Slice Creation /

Adaptation Time. Therefore, we measure the boot time for each of the components. This KPI can be

used to determine whether there is any problem when deploying the components for the first time

in the edge-cloud server.

5.1.2.4. P1UC2-SKPI-4: AGV-Edge Control Synchronization

AGV-Edge control synchronization is fundamental to ensuring accurate AGV operations in the

factory, avoiding potential crashes and unexpected maneuvers. Figure 8 shows the interactions

between the AGV controller and the AGV. The AGV control software is located at the edge

infrastructure and sends commands to the AGV instructing on the next movement. Simultaneously,

the AGV reports location and context information back to the control software. As a result, end-to-

end latency and packet loss are the main core 5G KPIs impacting P1UC2-SKPI-4. High end-to-end

latency may compromise the AGV operations by delaying the maneuvers, while packet loss may

generate crashes as the AGV may drive in the wrong direction. In other words, such core KPIs can

impact service reliability as well as cause accidents and hazards, thus affecting the safety of the

working environment.


H2020-856709

FIGURE 8: AGV-EDGE CONTROL SYNCHRONIZATION

5.1.2.5. P1UC2-SKPI-5: Network support for user mobility

To ensure correct and safe operations, AGVs and the quality control operator must stay connected

at any time and in any conditions. Device mobility (5GR-SKPI-3) is a service KPI influenced by

availability (CKPI-5), coverage (CKPI-4), radio signal quality (CKPI-10) and buffer occupancy (CKPI-

11). CKPI-4, CKPI-10 and CKPI-11 are a set of metrics related to the mobile device features, including

speed limits, reaction time to changes, handovers, etc. On the other hand, service availability must

be guaranteed at any time and in any location of the factory floor while the end-devices are moving,

without degrading the overall performance of the service and meeting the use case requirements.

5.1.2.6. P1UC2-SKPI-6: Concurrency of simultaneous users in a given area

The maximum number of served users is also a key metric in this use case. The SKPI “Concurrency of

simultaneous users in a given area” intends to capture the impact of the number of users in a given

area. Three core KPIs can impact this SKPI: the connection density, indicating the number of

connections served per unit of area, the service availability, and the data volume, as the capability of

meeting the service requirements of a number of users depends on the amount of data transferred.

Nevertheless, we do not expect that all the potential users will be connected at the same time, and

therefore we can consider a concurrency rate. In any case, users must perceive the same service

availability regardless of the total data volume.





H2020-856709



TABLE 6: INNOVALIA UC2 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P1UC2-01 Externalization of the Edge Computing

system

P1UC2-SKPI-1 – Service Operation

Time


Capacity

P1UC2-SKPI-3 – Service Creation

Time

FR-P1UC2-02 M3 software deployed in the manager

device (laptop, tablet)


Time


Capacity


Time

FR-P1UC2-03 Deployment of Robotlink,

DataAssembler and AGV control

services on a Cloud Edge System


Time


Capacity


Time

FR-P1UC2-04 Enable 5G connectivity on the CMM

controller


FR-P1UC2-05 CMM-AGV communication protocol

working over 5G connection

P1UC2-SKPI-5 – Network support

for user mobility

FR-P1UC2-06 Enable 5G connectivity on the AGV P1UC2-SKPI-4 – AGV-Edge Control

Synchronization


for user mobility

FR-P1UC2-07 Enable 5G connectivity on the manager

device


for user mobility

FR-P1UC2-08 Security: end-to-end encryption Qualitative verification

FR-P1UC2-09 The industrial environment must be

equipped with 5G-enabled

solutions


FR-P1UC2-10 High availability of the remote

connection including redundant

connection in order to guarantee the

communication in case of failure

5GR-SKPI-10 – Service Availability

FR-P1UC2-11 Solution must include additional

capabilities (e.g., specific features,

probes) that will take care of the

monitoring process



H2020-856709

FR-P1UC2-12 Solution must support programmability

and virtualization to enable flexible and

dynamic reconfiguration


FR-P1UC2-13 The solution must guarantee proper

levels of isolation (in the form of

network slice)

5GR-SKPI-6 – Integrated Multitype

Communications

5.2. Industry 4.0 Pilot – COMAU

5.2.1. Use Case 1: Digital Twin Apps

The Digital Twin is essentially a virtual representation of something which exists in the real world as

physical assets, processes, people, places, systems and devices connected in real-time thanks to

continuous data streaming.

To demonstrate 5G low latency performance and its feasibility in URLLC, this use case has the

objective to create a digital twin of an actual robot with both the aim to enable visibility and remote

insights with no delays about robot status and its performance (without the need to be

geographically close to it); and enabling remote controlling using an interface to receive a command

from external applications or virtual machines thanks to high-level program languages.

FIGURE 9: ARCHITECTURE OF COMAU UC1

The use case has as major objective the demonstration of 5G low latency features (i.e., URLLC profile).

As Figure 9 shows, the use case architecture includes an industrial robot, which reaches the 5G

network thanks to a user equipment (5G CPE), and a PC, similarly connected, where robot instructions

are generated and sent from. HoloLens glasses are connected to the PC via Wi-Fi as it is the only

connection supported by this device.

The use case workflow consists in remotely control the robot using the PC, which means send to it

time by time the position it has to assume, gathering real-time position data from the robot


H2020-856709

controller (which takes the actual position of the robot thanks to encoders installed on axes motor),

and replicate the behavior via a virtual version on the augmented reality device (i.e. the HoloLens).

Remotely send robot positions from IT devices (PCs or even VMs) is important in industry scenario

since it aims at changing the current paradigm which requires to offline program every single robot,

each with its program language.

In order to further investigate this aspect, characterized by low latency requirements, a remote

controlling of the robot has been developed as well using a 5G smartphone. With this handy device,

it is possible to instruct and drive the robot using a sort of “joystick” as an alternative to the

instructions sent by the PC.

The following subsections describe the specific Service KPI for this UC (i.e. P2UC1-SKPI-x), mentioning

from what 5GR-SKPI the Service KPIs applied, how it validates the service implemented by the UC,

the decomposition of the Service KPI into Core 5G KPIs and where to measure each Core 5G KPI

within the UC infrastructure/logical architecture.

5.2.1.1. P2UC1-SKPI-1: Digital Twin – Delay Guarantees

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-11 Service Reaction

Time. It represents the time between the instant a problem occurs and the moment in which the

plant manager decides how to mitigate it. The delay must be minimal, up to being not perceptible.

The plant manager must be able to see immediately the results of his/her decisions. For this reason,

there should be no delay in the representation of the virtual twin. As stated in Section 4, 5GR-SKPI-

11 is mapped onto the core KPI CKPI-1 End-to-end Latency.

Measurements of this SKPI basically amount to the quantification of the End-to-end latency. As

shown in Figure 9, measurement consoles will be deployed at the CPE and at the backend of the

vEPC, allowing the quantification of latency through the injection of test traffic.

5.2.1.2. P2UC1-SKPI-2: Support of Industrial Protocols over 5G networks

This specific service KPI is applied from the general 5Growth Service KPI 5GR-SKPI-11 Service Reaction

Time. Plant components use dedicated industrial communication protocols that currently work over

cable. These protocols are used to interconnect sensors, stations and PLCs. Industrial protocols, such

as Profinet or Ethernet/IP, are strongly based on time synchronization of data exchange cycles. They

can be tunneled over a 5G link if requirements about latency, data rate, packet loss and reliability are

met. In this sense, the 5G network should behave as a wired network as much as possible. Similarly

to SKPI-1, 5GR-SKPI-11 is mapped onto the core KPI CKPI-1 End-to-end Latency.

Measurements of this SKPI basically amount to the quantification of the end-to-end latency, so the

same observations as in P2UC1-SKPI1 can be applied.


H2020-856709

5.2.1.3. P2UC1-SKPI-3: Extensive on-Plant Coverage

This specific service KPI is primarily applied from the general 5Growth Service KPI 5GR-SKPI-7

Extensive Network Coverage in Vertical Premises. A steady connection must be ensured to all

connected stations in the plant. To do so, the network must be broadly available throughout the

plant and proper QoS must be ensured to all devices. A minimum link quality, with no packet loss,

measured from peers connected to the 5G network, is required to guarantee the service. Since a

multitude of sensors will be connected in a limited area, the network must offer a minimum

connected device density and support integrated multitype communications, as provisioned in the

general 5Growth Service KPI 5GR-SKPI-6 Integrated Multitype Communications. As shown in Figure

9, measurement consoles will be deployed at the CPE and at the backend of the vEPC. The CPE will

then be moved over all the area of the plant, providing a mapping of performance in the different

regions.

5GR-SKPI-7 is then mapped onto the core KPIs CKPI-1…5 as well as CKPI-7, while 5GR-SKPI-6 is

mapped onto CKPI-2, -3 and -5.

5.2.1.4. Mapping between Functional Requirements (FR) and specific Service KPIs.





TABLE 7: COMAU UC1 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P2UC1-01 Wi-Fi coverage up to 400m² for office

and double for open factory


FR-P2UC1-02 Cellular indoor coverage of up to 800

m² for office and double in open

factory

P2UC1-SKPI-3 – Extensive on plant

coverage

FR-P2UC1-03 Implement industrial protocols based

on cable via wireless connection

P2UC1-SKPI-1 – Delays guaranteed

P2UC1-SKPI-2 – Support of industrial

protocols over 5G

FR-P2UC1-04 Guarantee updated information with

no delays

P2UC1-SKPI-1 – Delays guaranteed

FR-P2UC1-05 Use the COMAU internal tool to

collect and analyse data in order to

predict future problems in production

lines


5.2.2. Use Case 2: Telemetry/Monitoring Apps

In this use case an extensive sensor deployment is in place to monitor and prevent failures of

machineries and equipment through massive data collection (i.e., vibration, pressure, temperature


H2020-856709

and so on). 5G facilitates the installation of a wide range of different wireless sensors easy to attach

on machineries without the rigidity of cable which is also more sensible to attrition.

COMAU has been developing its own IIoT (Industrial Internet of Things) platform which gathers data

directly from machinery as well as sensor data. With respect to the state of the art of this platform,

this use case aims at empowering the fault predictive capabilities with more data collected via 5G.


As shown in Figure 10, the architecture involves some industrial robots and a gateway on the COMAU

premises. COMAU IIoT platform is hosted in the TIM site. The two sites are connected via a VPN

channel.

Detailing the architecture of the testbed, we have used two different user equipment, actually CPEs,

in order to test two different communications. Indeed, the piece of software which gathers the

information and sends them to the platform in TIM is the gateway; a local connection (behind the

same router) to the industrial machinery connects a first robot (in the bottom part of the figure) to

the gateway. The other robot (in the upper part of the figure) is connected through a dedicated 5G

CPE. This second robot, however, is in the same address space of the first one and so, logically, it is

connected to the same gateway (i.e. same network).

Having gathered information from all the robots via those two different communications, everything

is sent via 5G to the COMAU IIoT platform, hosted in TIM site, through the VPN channel.

The following subsections describe the specific Service KPI for this UC (i.e. P2UC2-SKPI-x), mentioning

from what 5GR-SKPI the Service KPIs applied, how it validates the service implemented by the UC,




H2020-856709

5.2.2.1. P2UC2-SKPI-1: Extensive Network Coverage for High-density Sensors

This specific service KPI is primarily applied from the general 5Growth Service KPI 5GR-SKPI-7

Extensive Network Coverage in Vertical Premises. Equipment monitoring for fault detection and

predictive maintenance requires a network able to support all the sensors that are simultaneously

reporting data from each station of the production line. As shown in Figure 10, and similarly to what

is foreseen for P2UC1-SKPI3, measurement consoles will be deployed at the CPE and at the backend

of the vEPC. The CPE will then be moved over all the area of the plant, providing a mapping of

performance in the different regions.

5GR-SKPI-7 is then mapped onto the core KPIs CKPI-1…5 as well as CKPI-7.




the functional requirements that are validated when the selected Service KPIs are evaluated.








factory

P2UC2-SKPI-1 – Extensive coverage for

high density networks

FR-P2UC2-03 Guarantee updated information with

no delays



FR-P2UC2-04 Enable 5G connectivity on as much

machineries as possible



FR-P2UC2-05 Implement industrial protocols over

wireless channel


FR-P2UC2-06 Use the COMAU internal tool to

collect and analyse data in order to

predict future problems in

machineries


FR-P2UC2-07 High availability and low latency of

connection in order to avoid machine

stops


5.2.3. Use Case 3: Digital Tutorial and Remote Support

This use case aims at providing technicians and maintenance staff with digital tutorials and remote

support by means of high definition videos and live connections to remote technical offices. The

main objective is to reduce the MTTR (Mean Time To Repair) using real-time video streaming with a

skilled technician in remote locations to support maintenance and repair operations in the


H2020-856709

production line of the factory. Another advantage is the possibility to access on-demand tutorials

and instructions for training purposes.


This scenario, depicted in Figure 11, illustrates a remote factory operation. Here some machinery is

affected by a fault, but the local staff needs advanced support to rapidly fix the problem. Technicians

in the remote factory can use the HoloLens device connected via Wi-Fi to a 5G CPE. On the other

side of the connection, geographically separated from the factory, there is an expert with a remote

maintenance application. Such expert has the “full picture” of the fault and can provide remote

support to the onfield technician.

The application requested in this use case must enable:

• A high definition video streaming.

• The possibility to set up a video call between the technician and the expert.

• The possibility to assist the technician with detailed instructions and procedures (i.e., a digital

tutorial) enhanced by exploiting the AR capabilities of the HoloLens device. By adopting the

AR technology, it would be feasible to superimpose graphical elements to the reality so as

to assist the technician with the identification of the various components affected by the

fault.

The following subsections describe the specific Service KPI for this UC (i.e., P2UC3-SKPI-x), detailing

from what 5GR-SKPI the Service KPIs is applied, how it validates the service implemented by the UC,



5.2.3.1. P2UC3-SKPI-1: High-resolution Real-time Video Quality

This specific service KPI is mapped onto the general 5Growth Service KPI 5GR-SKPI-4 High-resolution

Real-time Video Quality. Applications in support of remote assembly and maintenance operators run

on consumer devices such as tablets or smartphones and are based on high-quality video feed.

Similarly, tutorials and assembly instruction videos are expected to require HD quality. In addition,


H2020-856709

for the correct operation of the use case, the priority is to immediately assist the technicians in case

of an emergency. From the vertical perspective, the real-time HD video quality is critical, however, in

order to assure a smooth factory operation in case of disruption there should be several factors that

need to be taken into consideration. A real-time HD video streaming requires high bandwidth and

low packet loss. Moreover, in order for the vertical to enable a real-time satisfactory interaction, there

must be a requirement for low latency and jitter. As shown in Figure 11, measurement consoles will

be deployed at the CPE and at the backend of the vEPC, allowing the quantification of latency and

throughput through the injection of test and background traffic.

As stated in Section 4, the 5GR-SKPI-4 is mapped onto the technical KPIs CKPI-2 Packet Loss, CKPI-

3 Guaranteed Data Rate, CKPI-8 Data Volume and CKPI-9 Jitter.

More specifically for this use case, in order for the vertical to establish a reliable QoE-aware

connection between the remote technical experts and the local technicians with no service

interruptions, a low packet loss rate should be guaranteed, considering that otherwise it may impact

the smoothness of the remote support.

Finally, due to the real-time nature of the Use Case scenario for remote operation, jitter should

substantially decrease.

Once obtained, these technical KPIs will be used to evaluate the QoE of the service offered to the

onfield technicians from the vertical perspective.

5.2.3.2. P2UC3-SKPI-2: Network Support for Device Mobility

This specific service KPI is mapped onto the general 5Growth Service KPI 5GR-SKPI-3 Device Mobility

as well as onto KPI 5GR-SKPI-7 Extensive Network Coverage in Vertical Premises. In order to ensure

the reliability of the use case operations, the provided remote support service should be available all

over the factory area in order to enable remote maintenance activities. Maintenance operators’

devices should stay connected to the network all over the facility, even when they move across the

plant. Service should be available all over the plant to allow remote maintenance activities close to

every station of the facility. Based on the above-mentioned requirements, device mobility should be

taken into account so as to ensure service availability. Measurement consoles will be deployed at the

CPE and at the backend of the vEPC. The CPE will then be moved across all the area of the plant,

tracking the performance as the CPE is being moved.

As stated in Section 4, 5GR-SKPI-3 is mapped onto the core KPIs CKPI-4 Coverage, CKPI-5 Availability,

CKPI-10 Radio Signal Quality and CKPI-11 Buffer Occupancy, while 5GR-SKPI-7 is mapped onto the

core KPIs CKPI-1…5 as well as CKPI-7 Connection Density.

5.2.3.3. Mapping between Functional Requirements (FR) and specific Service KPIs.




H2020-856709










factory

P2UC3-SKPI-2 – Network Support for

Device Mobility

FR-P2UC3-03 High quality stream video with low

delay

P2UC3-SKPI-1 – Realtime High

Definition Video Quality

FR-P2UC3-04 Provide the possibility to interact

using the same tool


FR-P2UC1-05 Have a storage where save

procedures and 3D models of

machineries


5.3. Transportation Pilot – EFACEC Engenharia e Sistemas

5.3.1. Use Case 1: Safety Critical Communications

The main goal of this Use Case involves the use of 5G communications to support railway signaling

operations; in particular, to meet railway level crossing communication requirements (M2M), namely

regarding safety-critical communications from approaching train detectors (Strike In detectors) to

the level crossing controllers.

Figure 12 depicts the use case overview, illustrating the relevant interface requiring URLL slice

capabilities.


H2020-856709

FIGURE 12: EFACEC_S UC1 OVERVIEW

5.3.1.1. P3UC1-SKPI-1: Communication Latency between LX Detectors and LX Controller.

Communication between these critical components must be established with low latency according

to the railway standards1, in order to assure the proper behavior of the Level Crossing; otherwise,

The Level Crossing, including the half-barriers will enter into protection mode, blocking the road

traffic. The network needs to ensure a minimum end-to-end latency in order to reduce the

communication time between railway protection devices.

As such, this use case SKPI applies to the general 5Growth 5GR-SKPI-2 “Sync between

communication components”, which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss”.

Regarding the “CKPI-1 – e2e latency” packets (data messages) between the rail sensor (strike in

detector) and the railway crossing controller need to be exchanged with very low latency (< 10 ms)

in order to detect that a train is approaching the Level Crossing and to start the safety signaling

operation. If the data messages do not reach the destination with low latency a failure will occur and

once again the system will operate in protection mode. The e2e latency can be measured at the level

crossing controller.

For the “CKPI-2: packet loss”, the communication between the strike in detector and the level crossing

controller involves two different kinds of packets (data messages): i) events that occur when a new

1 CENELEC SIL4 and EN 50159 – for transmission systems)


H2020-856709

train is approaching the level crossing and ii) other protocolary messages such as keep-alive

messages to guarantee a stable communication between the devices. Packet loss must be measured

at the level of the crossing controller. In order to be compliant with Railway CENELEC standards and

thus assuring a Certified Level crossing, this Core KPI must be guaranteed. If the availability or a

failure that occurs doesn’t recovery automatically in the proper time will represent a failure in the

Level Crossing, starting this way to an operation in a protection mode (road traffic will be blocked).

This situation has a huge impact in operation, can motivate penalties and requires the presence of

maintenance people to manually recover from this operation mode.

5.3.1.2. P3UC1-SKPI-2: Communication Availability between LX Detectors and LX Controller.

Communication between these critical components must be established with availability according

to the railway standards1, in order to assure the proper behavior of the Level Crossing; otherwise,

The Level Crossing, including the half-barriers will enter into protection mode, blocking the road

traffic. The network needs to ensure high levels of availability and resilience, in order to ensure that

the communication between railway protection devices is always up and running.

As such, this use case SKPI applies to the general 5Growth 5GR-SKPI-10 “Service availability”, which

maps to the core CKPI-5 “Availability”. As indicated in section 5.3.1.2, in order to be compliant with

Railway CENELEC standards and thus assuring a Certified Level crossing, this Core KPI must be

guaranteed. If the availability or a failure that occurs doesn’t recovery automatically in the proper

time (<500ms) will represent a failure in the Level Crossing, starting this way to an operation in a

protection mode (road traffic will be blocked). This situation has a huge impact in operation, can

motivate penalties and requires the presence of maintenance people to manually recover from this

operation mode. The availability of this critical communication can be measured at level crossing

controller






TABLE 10: EFACEC_S UC1 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P3UC1-01 The communications between the

components (track lines equipment’s

and LX controller) must use standard

protocols both for safety and security


FR-P3UC1-02 A Strike-In detector/axel counter train

detector system will be needed to

detect trains approaching the Level

Crossing



H2020-856709

FR-P3UC1-03 A Train detector/ axel counter train

detector system will be needed to

detect if the train is occupying the

Level Crossing section or to the

detect the absence of the train in the

section

P3UC1-SKPI-1 – Communication

Latency


Availability

FR-P3UC1-04 A LX controller will be needed to

receive sensors and equipment

information and to assure the LX

actions (railways signalling

operation)


Latency


Availability

FR-P3UC1-05 The communication between the

Strike-in detectors and the LX

controller must be supported over IP


FR-P3UC1-06 In a presence of communication

failures, the LX must be set to its safe

mode (protection mode)


FR-P3UC1-07 The communication between the axel

counters and the LX controller must

be 5G


Latency


Availability

FR-P3UC1-08 Traffic lights will be needed to inform

car driver’s


FR-P3UC1-09 Protection Lx Signals will be needed

to inform train drivers that the Level

Crossing is free


FR-P3UC1-10 5G CPE devices will be needed to

assure the 5G connectivity of Lx

devices to 5G network


Latency


Availability

FR-P3UC1-11 5G CPE devices must support

Ethernet interfaces


FR-P3UC1-12 The same Level of safety and security

must be assured by the 5G network


Latency


Availability

FR-P3UC1-13 The site pilot must be covered by a 5G

network

5GR-SKPI-7 – Extensive Network

Coverage in Vertical Premises

5.3.2. Use Case 2: Non-Safety Critical Communications

This use case is broken down into two different scenarios. involving (i) HD video image transmission

from level crossing surveillance camera to the approaching trains’ on-board tablets and/or


H2020-856709

maintenance agents’ handheld tablets and (ii) Level crossing controller status and alarm events

transmission to maintenance agents’ tablets.

The main objective of the first scenario is to transmit, supported by 5G communications, HD video

images to the approaching train, allowing the driver to get images of the level crossing area, thus

preventing accidents caused by cars trapped between the level crossing gates. In the scope of this

UC, it is also desired to transmit the same video images to maintenance agents/Command Centre,

providing this way, mechanisms to reinforce the level crossing safety.

Regarding the second scenario, the main objective is to allow maintenance agents to monitor (status

and alarm event transmission) and manage, remotely, the level crossing controller, through a tablet

device (5G communications).

Figure 13 depicts the use case overview, illustrating the relevant interface requiring URLL and eMBB

slice capabilities.

FIGURE 13: EFACEC_S UC2 OVERVIEW

5.3.2.1. P3UC2-SKPI1: High-definition Surveillance Video.

The control center and mobile devices (Train and maintenance agents) need to be able to visualize

the HD surveillance video with good quality and low latency; that means MOS of at least 42 and in

2 Mean Opinion Score (MOS) is a well-known measure of video quality (5-Excellent, 4-Good, 3-Fair, 2-Poor, 1-

Bad).


H2020-856709

accordance with Video Surveillance Systems3. The network needs to ensure the appropriate resources

to transmit HD video with a good level of quality and without delay. That means a sustained

bandwidth, low end-to-end latency and jitter.

As such, this use case SKPI applies to the general 5Growth 5GR-SKPI-4 “High-Resolution Real-Time

Video Quality”, which maps to the core CKPI-2 “Packet Loss”, CKPI-3 “Guaranteed Data Rate”, CKPI-

8 “Data Volume” and CKPI-9 “Jitter”.

Regarding CKPI-2 “Packet Loss”, the link needs low packet loss in order to avoid any loss of quality

to the video or its freeze. This UC does not apply to safety critical communications but is used to

reinforce the safety conditions of the level crossing. However, to have real-time video images to help

train drivers or maintenance agent to make decisions, no packet loss in the transmission must be

assured. This KPI can be measured at tablet devices (train).

Regarding CKPI-3 “Guaranteed Data Rate”, the link needs to guarantee enough data rate to support

the video stream. A guaranteed data rate of 160 Mbps is needed to accommodate high video

resolution and M-JPEG compression assuring that 2 trains in the same level crossing can receive,

simultaneously the LX images. This KPI can be measured at tablet devices (train).

Regarding CKPI-8 “Data Volume”, this KPI is not considered due to its low relevance regarding real-

time video. The link needs to have enough capacity to hold the transmission of the video at the

highest quality possible.

Regarding CKPI-9 “Jitter”, interarrival delays need to be avoided in order not to impact the quality of

the video. According to the best practices for real-time video systems applicable to railway scenarios,

the latency should be less than 100 ms, in order to have optimal performance. This latency as a direct

relationship with the train speed and the 5G network performance. Latency between 100 and 200 ms

causes impact in images visualization and latency higher than 500 ms are unacceptable and any

image shall be removable from the displays. This jitter can be measured at tablet devices (train) and

at the command center.

5.3.2.2. P3UC2-SKPI2: Real-time Sensors Monitoring Latency.

Real-time status and alarm information need to reach the control center and maintenance team

devices (Tablet) in useful time (less than 100ms). The network needs to ensure that the monitoring

is performed with a minimum end-to-end latency so that the information can be collected with the

appropriate accuracy.


communication components”, which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss”.

3 IEC62676 Video Surveillance Systems for use in security application in terms of security, integrity, availability

and latency


H2020-856709

Regarding the “CKPI-1 – e2e latency” packets (data messages) between the Lx Controller sensors and

the maintenance teams need to be exchanged with low latency (< 100 ms) in order to allow the team

to work without perceivable delay regarding the sensed information. If the data messages do not

reach the destination with enough low latency, the quality of the maintenance team can be impacted.

The latency can be measured at Tablet Devices.

For the “CKPI-2: packet loss”, the communication between the sensors and the maintenance crew

involves two different kinds of packets (data messages): i) events that occur when a new train is

approaching the level crossing and ii) other protocolary messages such as keep-alive messages to

guarantee a stable communication between the devices. The latency can be measured at Tablet

Devices (maintenance teams).

5.3.2.3. P3UC2-SKPI3: Real-time Sensors Monitoring and Visualization Data Connection

Availability.

The network needs to ensure that the monitoring services are continuously provided with a high

level of reliability and availability in order to manage the status and alarms in a proper way, allowing

them to make decisions in near-real-time. As such, this use case SKPI applies to the general 5Growth

5GR-SKPI-10 “Service availability”, which maps to the core CKPI-5 “Availability”.

Regarding the “CKPI-5 – Availability”, the KPI can be measured at Tablet Devices (maintenance

teams).






TABLE 11: EFACEC_S UC2 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P3UC2-01 At least a HD camera must be

installed in the LX area


FR-P3UC2-02 Geolocation coordinates must be

received by GPS


FR-P3UC2-03 Tablet devices must support 5G

communications

P3UC2-SKPI-1 – High-definition

surveillance video

P3UC2-SKPI-3 – Real-time sensors

monitoring and visualization data

connection availability

FR-P3UC2-04 5G communication must available to

exchange information between the

system components


surveillance video


monitoring latency


H2020-856709


monitoring and visualization data

connection availability

FR-P3UC2-05 Video Images must be visualized in

Tablet Devices


surveillance video

FR-P3UC2-06 Video Camera must support IP

protocols and interfaces


surveillance video

FR-P3UC2-07 5G CPE devices will be needed to

assure the 5G connectivity of system

devices to 5G network


FR-P3UC2-08 A command centre will be needed to

aggregate information (video images

and to broadcast alarms)


FR-P3UC2-09 An app for mobile devices (Tablet)

must be supplied mainly for

supervision purposes and to visualize

video images regarding

Level Crossing areas


surveillance video

FR-P3UC2-10 The site pilot must be covered by a 5G

network

5GR-SKPI-7 – Extensive Network


5.4. Energy Pilot – EFACEC Energia

5.4.1. Use Case 1: Advanced Monitoring and Maintenance Support for

Secondary Substation MV/LV Distribution Substation

The main goal of this Use Case is to implement an efficient control system in a secondary substation

of the University of Aveiro, supporting both the real-time network operation in the Control Centre

and the maintenance activities executed by field teams.

Upon receiving a work order from the Control Center, the field team will be capable to immediately

watch the live video feed from the secondary substation and, when arriving at the site, will be assisted

with the necessary maintenance actions by an augmented reality functionality.

With these tools, it is possible to aid and optimize the repair and replacement of damaged

equipment, upon a fault detection in a secondary substation.

5.4.1.1. P4UC1-SKPI-1: Monitoring Sensors Information Collection and Visualization End-

to-end Latency.

An alarm triggered in a secondary substation with the relevant sensor information needs to reach

the control center and the maintenance team in less than 100ms. The 5G network needs to ensure


H2020-856709

minimum end-to-end latency, allowing the telemetric information to reach the control center and

the mobile crew with no perceived delay associated.


communication components”, which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss.

Regarding the “CKPI-1 – e2e latency”, the different measurement devices, control center and

maintenance crew devices need to receive and send their information with low latency.

In what concerns to the visualization in the Control center (operator workstation), the KPI can be

measured at the Control Centre, by inspection and comparison of the data source time and data

processed time of each telemetered event.

In what concerns to the visualization in the mobile interface (tablet device, handheld by the

maintenance team), the KPI can be measured at the Tablet Device, by inspection and comparison of

the data source time, data processed time (server) and data processed time (tablet) of each

telemetered event.

For the “CKPI-2: packet loss”, the different measurement devices, control center and maintenance

crew devices need to receive and send their information.

5.4.1.2. P4UC1-SKPI-2: Monitoring Sensors Information Collection and Visualization

Availability.

An alarm triggered in a secondary substation with the relevant sensor information needs to reach

the control center and the maintenance team with high availability to ensure timely failure detection.

The network needs to provide maximum reliability to ensure that monitoring information is

transmitted continuously and with no packet loss, to prevent retransmissions and information not

timely reached.


maps to the core CKPI-5 “Availability” as well as the 5GR-SKPI-2 “Synchronization between

Communication Components”, which maps to the core CKPI-2 “Packet Loss” (it also maps to the

CKPI-1 “Latency” but it is not considered relevant for P4UC1-SKPI-2).

Regarding “CKPI-5: Availability”, the service needs to be available for continuously sending the

monitoring data and video surveillance.

The KPI can be measured at the Control Centre, by inspection of the logs of the “keep-alive” function

in the SCADA Frontend module.

Regarding “CKPI-2: Packet Loss”, the different measurement devices, control center and maintenance

crew devices need to receive and send their information.

The KPI can be measured both at the Control Centre and at the tablet device.


H2020-856709

5.4.1.3. P4UC1-SKPI-3: High-definition Surveillance Video.

The control center and the maintenance crews need to be able to visualize the HD surveillance video

with good quality and low latency. The network must sustain an ensured required bandwidth for the

video service, in order to allow the delivery of HD video with no glitches and enable remote crew

assistance to work effectively.

As such, this use case SKPI applies to the general 5Growth 5GR-SKPI-4 – “High-Resolution Real-

Time Video Quality”, which maps to the core CKPI-2 “Packet Loss”, CKPI-3 “Guaranteed Data Rate”,

CKPI-8 “Data Volume” and CKPI-9 “Jitter”.

Regarding the CKPI-2 “Packet Loss”, the link needs low packet loss in order to avoid any loss of

quality to the video or its freeze. Regarding the CKPI-3 “Guaranteed Data Rate”, the link needs to

guarantee enough data rate to support the video stream. Regarding the CKPI-8 “Data Volume”, the

link needs to have enough capacity to hold the transmission of the video at the highest quality

possible.

Regarding the CKPI-9 “Jitter”, the interarrival delays need to be avoided in order not to impact the

quality of the video.

In what concerns to the visualization in the Control center (operator workstation), the KPI can be

measured at the operator workstation. In what concerns to the visualization in the mobile interface

(tablet device, handheld by the maintenance team), the KPI can be measured at the Tablet Device.

5.4.1.4. P4UC1-SKPI-4: Augmented Reality Information Real-time End-to-end Latency.

The augmented reality (AR) application will need to use real-time video from the mobile device (ex.

Smartphone) and real-time information from the installation through the control center. If the

information does not come on time, the AR service will lose integrity, representing wrong or not

timely information on top of live video streaming. The network needs to ensure minimum end-to-

end latency, allowing the telemetric information to reach the control center and the mobile crew with

no perceived delay associated, enabling a correct representation of data over the video.

Therefore, this UC1 SKPI shares the same considerations as presented in P4UC1-SKPI-1 “Monitoring

sensors information collection and visualization must have an end-to-end latency below 100ms

(<100ms).” in section 5.4.1.1. Concretely, it also applies to the general 5Growth 5GR-SKPI-2 “Sync

between communication components”, which maps to the core CKPI-1 “e2e latency” and CKPI-2

“packet loss.

The KPI can be measured at the Tablet Device, by inspection and comparison of the data source time,

data processed time (server) and data processed time (tablet) of each telemetered event.

5.4.1.5. P4UC1-SKPI-5: Augmented Reality information Real-time Availability.

The augmented reality (AR) application will stream real-time video from the mobile device (ex.

Smartphone) and must have high availability and low packet loss to ensure that data is timely


H2020-856709

collected and represented on top of live video streaming. The network needs to ensure high

availability, ensuring a continuous service for data delivery. It also needs low packet loss, preventing

information re-transmission, which will cause undesirable effects on the AR service.

Therefore, this UC1 SKPI shares the same considerations as presented in P4UC1-SKPI-2 “Monitoring

sensors information collection and visualization must be available more than 99.9% of the time

(>99.9%).” in section 5.4.1.2. Concretely, it also applies to the general 5Growth 5GR-SKPI-10 “Service

availability”, which maps to the core CKPI-5 “Availability” as well as the 5GR-SKPI-2 “Synchronization

between Communication Components”, which maps to the core CKPI-2 “Packet Loss” (it also maps

to the CKPI-1 “Latency” but it is not considered relevant for P4UC1-SKPI-2). The KPI can be measured

at the tablet device.






TABLE 12: EFACEC_E UC1 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P4UC1-01 The communications between the

components (Low voltage sensor, LV

controller, and HD video camera)

must use standard protocols


FR-P4UC1-02 The communication between the LV

sensor and LV controller must be RS

485 using Modbus protocol


FR-P4UC1-03 The connection between the HD

video camera and LV Controller must

be Ethernet or wireless


FR-P4UC1-04 The LV controller and HD video

camera must be connected to a

router with 5G communication


FR-P4UC1-05 A digital input in the LV controller

must be available to receive the

intrusion alarm


FR-P4UC1-06 The connection between the

secondary substation (LV controller

and HD video camera) and the

control centre must be 5G

P4UC1-SKPI-1 – Monitoring sensors

information collection and visualization

end-to-end latency



availability

P4UC1-SKPI-3 – High Definition

surveillance video


H2020-856709

FR-P4UC1-07 5G communication must be used

between the smartphone and control

centre

P4UC1-SKPI-4 – Augmented reality

information real-time end-to-end

latency


information real-time availabillity

FR-P4UC1-08 The fault detection functionality must

be implemented in the low voltage

sensor


FR-P4UC1-09 The fault must be transmitted to the

control centre whenever is detected



end-to-end latency

FR-P4UC1-10 Transmission of HD video from the

secondary substation to the control

centre must be implemented

P4UC1-SKPI-3 – High Definition

surveillance video

FR-P4UC1-11 A SCADA system must be

implemented for monitoring and

control of the low voltage network


FR-P4UC1-12 The intrusion sensor must be

available



availability

FR-P4UC1-13 The intrusion alarm must be

transmitted to the control centre

whenever is activated



end-to-end latency

FR-P4UC1-14 The control centre must have the

capacity to send information to the

maintenance crew whenever is

needed



availability



FR-P4UC1-15 The maintenance crew must have a

smartphone with capacity to receive

messages from the control centre


FR-P4UC1-16 The control centre must have the

capacity to send a command to the

secondary substation to start the

transmission of the HD video


FR-P4UC1-17 An augmented reality application

must be available to be installed in

the smartphone for maintenance

support



FR-P4UC1-18 The LV sensor must be connected to

the LV network using Rogowski coils

for currents and direct connection for

the voltages


FR-P4UC1-19 The LV controller must be connected

to the LV network using current



H2020-856709

transformers for currents and direct

connection for the voltages

FR-P4UC1-20 An intrusion sensor must be installed

in the substation door


FR-P4UC1-21 The control centre must be installed

in the cloud with connectivity to the

5G network


5.4.2. Use Case 2: Advanced Critical Signal and Data Exchange Across Wide

Smart Metering and Measurement Infrastructures

The goal of this Use Case is to develop a new supervision architecture for the low voltage network,

in order to reduce downtime/outage duration and the number of customers affected by the outage.

Relying on an enhanced level of automation in the secondary substations and low voltage

distribution network, and also on the LV device connectivity through the 5G network, it will be

possible to detect and locate an electric grid fault occurring in the low voltage distribution network

in real-time, which will be immediately notified to the control center operator. The Control Centre

operator will then be able to send precise information to the maintenance team identifying the

electric grid fault location, which will lead to a significant reduction on the time taken to restore the

power to the affected costumers.

5.4.2.1. P4UC2-SKPI1: Last-gasp Information End-to-end latency.

In a power outage situation, last-gasp capable devices must be able to send their latest information

in memory to the control center, via 5G network, before the devices cannot send it anymore (because

of a power outage). The network needs to send the last-gasp capable devices information as soon

as it is generated with very low latency, to ensure that the device still has power.


communication components”, which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss.

Regarding the “CKPI-1 – e2e latency”, the last gasp information needs to be sent out as soon as it is

generated.

The KPI can be measured in the GSmart device, by inspection and comparison of the data source

time and data reception time (GSmart) of each last gasp.

For the “CKPI-2: packet loss”, the last gasp information needs to be sent with no packet loss.

The KPI can be measured in the GSmart device, by inspection and comparison of the received last

gasps with the set of triggered and timestamped power outages at the source. This measurement

has the particularity that, since transmission is done using UDP, there can be potential unrecoverable

packet loss.


H2020-856709

5.4.2.2. P4UC2-SKPI2: Last-gasp Information Connectivity Availability.

In a power outage situation, the last-gasp capable devices must be able to send their latest

information in memory to the control center, via 5G network, ensuring that the network is available

and has no packet loss. The network needs to send the last-gasp capable devices information in a

continuous manner requiring that the network service is highly available, and without packet loss,

as there will be no chance for retransmission.


maps to the core CKPI-5 “Availability” as well as the 5GR-SKPI-2 “Synchronization between

Communication Components”, which maps to the core CKPI-2 “Packet Loss” (it also maps to the

CKPI-1 “Latency” but it is already considered in P4UC2-SKPI1).

Regarding “CKPI-5: Availability”, the system needs to be available whenever there is a need to send

last gasp information.

The KPI can be measured in the GSmart device, by inspection of the logs of the “keep-alive” function

over the connected low voltage sensors.

Regarding “CKPI-2: Packet Loss”, the KPI can be measured in the GSmart device.

5.4.2.3. P4UC2-SKPI3: Secondary Substation End-to-end Latency.

While analyzing the power outage event, the control center crew can issue further queries to the

monitoring system, requesting more information. In this case, the latency requirement can be least

stringent when compared to the live power outage. As the reconstruction of the power outage event

requires very precise analysis, the analysis crew needs to be able to collect information quickly. The

information needs to be sent with very low delay when it is requested.

Therefore, this UC1 SKPI shares the same considerations as presented in P4UC2-SKPI-1 “Last-gasp

information must reach the control center in less than 20ms (<20ms)” in section 5.4.2.1. Concretely,

it also applies to the general 5Growth 5GR-SKPI-2 “Sync between communication components”,

which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss. This SKPI, however, can be

revisited in the future regarding its usefulness, as it ends up being covered by the remaining SKPI’s

in UC1.

5.4.2.4. P4UC2-SKPI4: Secondary Substation Availability.

While analyzing the power outage event, the control center crew can issue further queries to the

monitoring system, requesting more information. For that purpose, the network needs to be

continuously available. As the reconstruction of the power outage event requires very precise

analysis, the analysis crew needs to be able to collect information, requiring that the network service

is always available to fulfill the communication need, and without packet loss.

Therefore, this UC SKPI shares the same considerations as presented in P4UC2-SKPI-2 “Last-gasp

information connectivity with the control center must be available more than 99.9% of the time


H2020-856709

(>99.9%)” in section 5.4.2.2. Concretely, it also applies to the general 5Growth 5GR-SKPI-10 “Service

availability”, which maps to the core CKPI-5 “Availability” as well as the 5GR-SKPI-2 “Synchronization

between Communication Components”, which maps to the core CKPI-2 “Packet Loss” (it also maps

to the CKPI-1 “Latency” but it is not considered relevant for P4UC1-SKPI-2). This SKPI, however, can

be revisited in the future regarding its usefulness, as it ends up being covered by the remaining SKPI’s

in UC1.

5.4.2.5. P4UC2-SKPI5: Time Synchronization between all Sensors.

The time synchronization commands sent by the control center to the smart sensors will need to

guarantee the synchronization between all sensors equal to or less than 1ms. In order to keep the

devices all timely synchronized, the connectivity among them needs to be of low latency and jitter

should be as lower as possible in order to enable a predictive latency, which is an important aspect

of time synchronization.

Therefore, this UC SKPI shares the same considerations as presented in P4UC2-SKPI-1 “Last-gasp

information must reach the control center in less than 20ms (<20ms)” in section 5.4.2.1. Concretely,

it also applies to the general 5Growth 5GR-SKPI-2 “Sync between communication components”,

which maps to the core CKPI-1 “e2e latency” and CKPI-2 “packet loss.

The KPI can be measured in the GSmart device, where the source time of the received low voltage

event can be compared against the time configured on the LV Generator for firing that event, to

measure the time synchronization accuracy.






TABLE 13: EFACEC_E UC2 MAPPING BETWEEN FUNCTIONAL REQUIREMENTS AND SPECIFIC SERVICE KPIS


FR-P4UC2-01 The connection between the

distribution cabinet (LV sensor) and

LV Controller must be 5G

P4UC2-SKPI-1 – Last-gasp information

end-to-end latency

P4UC2-SKPI-3 – Secondary substation

end-to-end latency

FR-P4UC2-02 The control centre must run an

algorithm to detect the location of

the fault


FR-P4UC2-03 The control centre must send controls

to the LV sensors in order to

reconfigure the LV network



H2020-856709

FR-P4UC2-04 The control centre must send date

and time to all the LV sensors

installed in the low voltage network

P4UC2-SKPI-5 – Time synchronization

between all sensors

FR-P4UC2-05 The fault detection functionality must

be implemented in the low voltage

sensors


FR-P4UC2-06 The fault must be transmitted to the

control centre whenever is detected


end-to-end late ncy


end-to-end latency

FR-P4UC2-07 A SCADA system must be

implemented for monitoring and

control of the low voltage network


availability


availability

FR-P4UC2-08 The SCADA system must have the

capacity to send information to the

maintenance crew whenever is

needed


FR-P4UC2-09 The SCADA system must know the

low voltage network topology to

identify the location of the fault


FR-P4UC2-10 The LV sensor must be connected to

the low voltage network using

Rogowski coils for currents and direct

connection for the voltages


FR-P4UC2-11 The SCADA system must be installed

in the cloud with connectivity to the

5G network


FR-P4UC2-12 LV sensor power supply with storage

capability to ensure the conditions for

the last-gasp message (before

turning off)


availability

FR-P4UC2-13 5G communication services available P4UC2-SKPI-2 – Last-gasp information

availability


availability

FR-P4UC2-14 Low latency communications:

a. < 5ms for last gasp

b. < 1ms for synchronization

c. < 20ms for control plane latency


end-to-end latency


end-to-end latency

P4UC2-SKPI-5 – Time synchronization

between all sensors.

FR-P4UC2-15 Wide Area Coverage at least 10 km2 5GR-SKPI-7 – Extensive Network



H2020-856709

FR-P4UC2-16 The self-healing algorithm can run on

the edge


FR-P4UC2-17 A LV Controller must be installed in

the secondary substation to

concentrate all the information from

the low voltage network



H2020-856709

6. Conclusions

Deliverable D4.1 consolidates the work performed in T4.1 with respect to KPI identification and

definition, with the purpose of setting the basis for the evaluation of the vertical pilots implemented

by the 5Growth project. Given the industrial and business nature of the 5G use cases implemented,

an innovative classification of KPIs has been applied, dividing the KPIs identified into two different

levels: Core 5G KPIs and Service KPIs. Core 5G KPIs are technical KPIs that evaluate performance

paramaters at the network and infrastructure level of the use case. In most cases, these technical KPIs

are transparent for the verticals, who are not aware of the underlying infrastructure of their services.

Therefore, the need for Service KPIs that evaluate end-to-end 5G industrial scenarios from the vertical

point of view arises. More in detail, Service KPIs evaluate qualitative aspects of the pilots that are not

directly measurable, to ensure that the functional and business requirements [1] of the use cases are

met. Due to Service KPIs are not directly measurable, a mapping between Service and Core 5G KPIs

has been defined in this deliverable. This mapping states the specific components and interfaces

where the Core 5G KPIs can be measured in the use cases infrastructure, in addition to how the

Service KPIs are extrapolated from different Core 5G KPI combinations.

Regarding future work, task T4.2 of the project is dedicated to define the tools and methodology

needed to measure the KPIs here identified. Furthermore, the validation trials for the 5Growth vertical

pilots will leverage the KPIs and tools defined to evaluate whether the deployment of the different

use cases satisfy the expected performance requirements. Ultimately, the objective is not restricting

the KPIs defined only to evaluate the 5Growth use cases, but use them as well to define realistic

vertical SLAs and extrapolate them to other 5G industrial scenarios.


H2020-856709

7. References

[1] 5Growth, D1.1, Business Model Design, August 2019.

https://5growth.eu/svn/5growth/execution/WP1/deliverables/D1.1/D1.1-

Business_Model_Design.pdf

[2] TMV-WG, Validating 5G Technology Performance Assessing 5G architecture and

Application Scenarios, June 2019. https://5g-ppp.eu/wp-content/uploads/2019/06/TMV-

White-Paper-V1.1-25062019.pdf

[3] Ericsson, 5G Use Cases. https://www.ericsson.com/4a8d61/assets/local/5g/5g-use-cases-

ericsson.pdf

[4] 5Growth, D3.2, Specification of ICT17 in-house deployment, May 2020.

https://5growth.eu/svn/5growth/execution/WP3/deliverables/D3.2/D3.2-

Specification_of_ICT17_in-house_deployment.pdf

[5] 5G-EVE, D4.1, Experimentation tools and VNF repository, October 2019.

https://doi.org/10.5281/zenodo.3628201

[6] 5G-VINNI, D4.1, Initial report on test-plan creation and methodology, and development of

test orchestration framework, July 2019. https://doi.org/10.5281/zenodo.3345626

https://5growth.eu/svn/5growth/execution/WP1/deliverables/D1.1/D1.1-Business_Model_Design.pdf

https://5growth.eu/svn/5growth/execution/WP1/deliverables/D1.1/D1.1-Business_Model_Design.pdf

https://5g-ppp.eu/wp-content/uploads/2019/06/TMV-White-Paper-V1.1-25062019.pdf

https://5g-ppp.eu/wp-content/uploads/2019/06/TMV-White-Paper-V1.1-25062019.pdf

https://www.ericsson.com/4a8d61/assets/local/5g/5g-use-cases-ericsson.pdf

https://www.ericsson.com/4a8d61/assets/local/5g/5g-use-cases-ericsson.pdf

https://5growth.eu/svn/5growth/execution/WP3/deliverables/D3.2/D3.2-Specification_of_ICT17_in-house_deployment.pdf

https://5growth.eu/svn/5growth/execution/WP3/deliverables/D3.2/D3.2-Specification_of_ICT17_in-house_deployment.pdf




H2020-856709

8. Appendix A – COVID-19 Impact Analysis on Plans and

Roadmaps

In the scope of WP3, the deliverable D3.2 [4] contains a dedicated section evaluating the impact

analysis related to the COVID-19. According with this analysis, the COVID-19 restrictions may cause

a worst-case scenario of four month of unavailability for on-site (COMAU premises, TIM premises,

SMEs and universities, regarding vertical COMAU Pilot; 5TONIC premises, regarding vertical

INNOVALIA Pilot; University of Aveiro, EFACEC premises and ALB laboratories, regarding vertical

Energy EFACEC_E Pilot; and University of Aveiro, Aveiro Harbour, EFACEC premises and ALB

laboratories, regarding vertical Transportation EFACEC_S Pilot) activities, motivating an impact in the

original plan for all vertical pilots.

Therefore, in order to evaluate further implications in related Work Packages, in this section, a pilot

basis impact analysis is performed related to WP4 activities (verification and validation campaigns)

focusing on KPI measurements.

Industry 4.0 Pilot – INNOVALIA

As described in D3.2 [4], the COVID-19 restrictions cause three major impacts in INNOVALIA Pilot: i)

Equipment availability for performing initial integration and testing with 5TONIC ii) 5G equipment

with Stand Alone option and iii) 5Growth integration with 5G-EVE at 5TONIC. Before COVID-19 the

initial plan estimation was to achieve this goal at March 2020, June 2020 and December 2020

respectively. The COVID-19 restrictions have motivated a change in the goal related to the realization

of preliminary tests at 5TONIC, by performing tests at INNOVALIA premises but considering a

commercial 4G network, also have motivated an estimated delay of three months in the second goal

(Stand Alone option) and finally have motivated to relax the ambition of the 5Growth integration

with 5G-EVE at 5TONIC , performing the integration at INNOVALIA premises but only in April 2021.

These considerations affect the activities of KPI measurements causing delays in the initial plan and

roadmap and cause some changes in the initial goals.

Use Case 1: Connected Worker Remote Operation of Quality Equipment

Table 14 shows the SKPIs of the INNOVALIA use case 1 and the estimated impact (delays/initial goals)

motivated by COVID-19.

TABLE 14: ESTIMATED COVID-19 IMPACT ON INNOVALIA USE CASE 1

UC-SKI- ID KPI description Impact on M18 Impact on initial goal

P1UC1-SKPI-1 Teleworker-

CMM

Synchronization

To be performed at

M18

To be performed at 5TONIC, even

though 5Growth solution will not be

fully integrated with 5G-EVE

platform.


H2020-856709

P1UC1-SKPI-2 High-resolution

Real-time Video

Quality

To be performed at

M18




platform.

P1UC1-SKPI-3 Service Setup

Time

To be performed

after M18

Not possible to be performed at

M18, since it requires 5Growth

integration with 5G-EVE and first SW

release

P1UC1-SKPI-4 Radius of

Operation

To be performed at

M18




platform.

P1UC1-SKPI-5 Integrated

Multitype

Communications

To be performed at

M18




platform.

P1UC1-SKPI-6 Extensive

Network

Coverage in the

Factory Premises

To be performed at

M18




platform.

Use Case 2: Connected Worker: Augmented Zero Defect Manufacturing (ZDM)

Decision Support System (DSS)

Table 15 shows the SKPIs of the INNOVALIA use case 2 and the estimated impact (delays/initial goals)

motivated by COVID-19

TABLE 15: ESTIMATED COVID-19 IMPACT ON INNOVALIA USE CASE 2


P1UC2-SKPI-1 Service

Operation Time

To be performed after

M18




release.


Operation

Capacity


M18




release.


Creation Time


M18




release.

P1UC2-SKPI-4 AGV-Edge

Control

Synchronization

To be performed at

M18




platform.


H2020-856709

P1UC2-SKPI-5 Network

support for

user mobility

To be performed at

M18




platform.

P1UC2-SKPI-6 Concurrency of

simultaneous

users in a given

area

To be performed at

M18




platform.

Industry 4.0 Pilot – COMAU

Taking in consideration the original plan, the KPI measurements of COMAU Use Cases will start at

month 10 (M10) and will end by month 24 (M24). Even considering the COVID-19 restriction, this

plan will be kept for Use Case 1 and 2 and it is assumed that all KPI will be provided according to

the initial plan. Regarding the Use Case 3, a two month delay is estimated regarding M24.

Additionally, taking in consideration month 18 (M18) a gap is being considered. At M18, the

measurements will be provided without monitoring platform implementation and and all

measurements related to KPI planned for M18 will be done using a point-to-point transport

network. In parallel, E2E latency measurements on the target transport network (based on a ring)

will be done to assess the impact on transport before integration with the E2E infrastructure which

is planned in M24.

Next subsections show, for the three COMAU Use Cases, the plan and estimated roadmap impact.

Use Case 1: Digital Twin Apps

Figure 14 illustrates the delays (in red) caused by the COVID-19 impact on the original roadmap (in

green) planned for the COMAU use case 1.


H2020-856709

FIGURE 14: COMAU USE CASE 1 ROADMAP DELAY DUE TO COVID-19

Use Case 2: Telemetry/Monitoring Apps




Use Case 3: Digital Tutorial and Remote Support




H2020-856709


Transportation Pilot – EFACEC Engenharia e Sistemas

As detailed in D3.2 [4] the COVID-18 restrictions caused several major impacts; i) Pilot availability at

EFACEC_E premises ii) 5G network infrastructure availability at IT Aveiro premises iii) 5G network full

functionalities and iv) Pilot complete integration at IT Aveiro premises.

Before COVID-19, the initial plan estimation was to achieve these goals at June 2020, September

2020, March 2021 and August 2021 respectively. Except the final Pilot integration, which is expected

to keep the initial planned date, the COVID-19 restrictions had motivated a four month delay in the

other goals. Nevertheless, motivated by these delays in previous activities it is expected less time for

final integration, less time for verification and validation and less time for global performance

evaluation of the Pilot. Because of these limitations and since according these dates the 5G network

is not available at IT Aveiro premises in month 18 (M18) it was decided to perform the preliminary

integration and KPI’s measurements at IT Aveiro Lab (instead of being performed in the physical

secondary substation).



Use Case 1: Safety Critical Communications

Table 16 shows the SKPIs of the EFACEC_S use case 1 and the estimated impact (delays/initial goals)



H2020-856709

TABLE 16: ESTIMATED COVID-19 IMPACT ON EFACEC_S USE CASE 1


P3UC1-SKPI-1 Communication

Latency between LX

Detectors and LX

Controller

To be

performed at

M18

To be performed at IT

Aveiro/lab environment (ICT-

17/5Growth) instead of Aveiro

Harbour premises. Safety

critical components and

simulations will be used

instead of a real Pilot scenario.

P3UC1-SKPI-2 Communication

Availability between

LX Detectors and LX

Controller

To be

performed after

M18

Service availability will be

performed in a real Pilot

scenario.

Use Case 2: Non-Safety Critical Communications

Table 17 shows the SKPIs of the EFACEC_S use case 2 and the estimated impact (delays/initial

goals) motivated by COVID-19.

TABLE 17: ESTIMATED COVID-19 IMPACT ON EFACEC_S USE CASE 2


P3UC2-SKPI-1 High-Definition

Surveillance Video

To be

performed at

M18




Harbour premises. An

automotive solution instead of

a train

P3UC2-SKPI-2 Real-time Sensors

Monitoring Latency

To be

performed at

M18




Harbour premises

P3UC2-SKPI-3 Real-time Sensors

Monitoring and

Visualization Data

Connection

Availability

To be

performed after

M18


performed in real Pilot scenario

Energy Pilot – EFACEC Energia

As detailed in D3.2 [4] the COVID-18 restrictions caused several major impacts; i) Pilot availability at

EFACEC_E premises ii) 5G network infrastructure availability at IT Aveiro premises iii) 5G network full

functionalities and iv) Pilot complete integration at IT Aveiro premises.


H2020-856709

Before COVID-19, the initial plan estimation was to achieve these goals at June 2020, September

2020, March 2021 and August 2021 respectively. Except the final Pilot integration, which is expected

to keep the initial planned date, the COVID-19 restrictions had motivated a four month delay in the

other goals. Nevertheless, motivated by these delays in previous activities it is expected less time for

final integration, less time for verification and validation and less time for global performance

evaluation of the Pilot. Because of these limitations and since according these dates the 5G network

is not available at IT Aveiro premises in month 18 (M18) it was decided to perform the preliminary

integration and KPI’s measurements at IT Aveiro Lab (instead of being performed in the physical

secondary substation).



Use Case 1: Advanced Monitoring and Maintenance Support for Secondary

Substation MV/LV Distribution Substation

Table 18 shows the SKPIs of the EFACEC_E use case 1 and the estimated impact (delays/initial goals)


TABLE 18: ESTIMATED COVID-19 IMPACT ON EFACEC_E USE CASE 1

UC-SKI- ID KPI’s description Impact on M18 Impact in initial goal

P4UC1-SKPI-1 Monitoring Sensors

Information

Collection and

Visualization End-to-

End Latency

To be performed

at M18



17/5Growth) instead of IT

physical secondary

substation

P4UC1-SKPI-2 Monitoring Sensors

Information

Collection and

Visualization

Availability

To be performed

after M18


performed in a real Pilot

scenario

P4UC1-SKPI-3 High Definition

Surveillance Video

To be performed

at M18



17/5Growth) instead of IT

physical secondary

substation

P4UC1-SKPI-4 Augmented Reality

Information Real-

time End-to-End

Latency

To be performed

after M18

EFACEC_E-UC1 is not fully

deployed by M18. It lacks the

interactions with the

maintenance team mobile

interface (HD Video

Streaming, Augmented

Reality)


H2020-856709

P4UC1-SKPI-5 Augmented Reality

Information Real-

time Availability

To be performed

after M18

EFACEC_E-UC1 is not fully

deployed by M18. It lacks the

interactions with the

maintenance team mobile

interface (HD Video

Streaming, Augmented

Reality)

Use Case 2: Advanced Critical Signal and Data Exchange Across Wide Smart Metering

and Measurement Infrastructures

Table 19 shows the SKPIs of the EFACEC_E use case 2 and the estimated impact (delays/initial

goals) motivated by COVID-19.

TABLE 19: ESTIMATED COVID-19 IMPACT ON EFACEC_E USE CASE 2

UC-SKI- ID KPI’s description Impact on M18 Impact in initial goal

P4UC2-SKPI-1 Last-gasp

Information End-to-

End Latency

To be performed

after M18

No impact, since the initial

plan was considering to

perform KPI’s measurements

after M18

P4UC2-SKPI-2 Last-gasp

Information

Connectivity

Availability

To be performed

at M18

No impact is expected, since,

in line with the initial plan,

KPI’s measurements will take

place only after M18

P4UC2-SKPI-3 Secondary

Substation End-to-

End Latency

To be performed

after M18





P4UC2-SKPI-4 Secondary

Substation

Availability

To be performed

after M18





P4UC2-SKPI-5 Time

Synchronization

between all Sensors

To be performed

after M18






H2020-856709

9. Appendix B – Technical KPIs defined by 5G-PPP TMV-WG,

5G-EVE and 5G-VINNI

As stated in the White paper4, the priority for the 5GPPP TMV group was to identify those technical

KPIs that were supporting the 5G PPP contractual KPIs validation5. The initial identified KPIs support

mostly the following two contractual KPIs:

• P1: Providing 1000 times higher wireless area capacity and more varied service capabilities

compared to 2010.

• P4: Creating a secure, reliable and dependable Internet with a “zero perceived” downtime for

services provision.

Those two contractual KPIs can be translated in a series of Technical KPIs, summarized in Table 20.

In the table, the KPIs are classified into two types: Service Level Agreement (SLA) or Technology

validation. In the former meaning, they are related to performance specified in the SLA in the latter

meaning they refer to testing technology capabilities. In addition to the chosen KPI their definition,

the measurement points in the 5G architecture are provided. Finally, the last column reports the

contractual KPI to which a specific KPI is contributing.

TABLE 20: TECHNICAL KPIS DEFINED BY THE 5G-PPP TMV-WG

Type KPI name KPI measurement points 5G PPP

KPI

Validated

SLA Minimum Expected

Upstream Throughput

UE transmitting IP packets to the

N6 interface.

P1

SLA

Minimum Expected

Downstream

Throughput

UE receiving IP packets from the

N6 interface P1

SLA Maximum Expected

Latency

RTT of UE IP packets transmitted

to the N6 interface.

P1, P4

SLA Network Reliability Transport layer packets are lost

between the UE and the N6

interface

P4

SLA, Technology

Validation

Quality of Experience Measured at the UE side at

application or application API

level

P1, P4

4 5G-PPP Test, Measurement, and KPIs Validation Working Group (5G-PPP TMV), “Validating 5G Technology

Performance Assessing 5G architecture and Application Scenarios”, 25/06/2019 5 "Contractual Arrangement Setting up a Public Private Partnership in the Area of Advanced 5G Network

Infrastructure for te Future Internet between the European Union and the 5G Infrastructure Association",

December 17, 2013 (https://5g-ppp.eu/contract/ latest access 07/05/2019)


H2020-856709

Technology

Validation

UL Peak Throughput Single UE transmitting IP packets

to the N6 interface.

P1

Technology

Validation

DL Peak Throughput Single UE receiving IP packets

from the N6 interface

P1

5G-EVE and 5G-VINNI, identified KPIs to be collected in their experimental platforms are reported

in [5] and in [6]. For easy consultation the 5G-EVE KPIs are reported in Table 21.

TABLE 21: 5G-EVE KPIS AS IN 5G-EVE-D1.4

5G-EVE KPI Definition Measurement methodology

User Data

Rate or

Speed

Minimum user experienced

data rate required for the

user to get a quality

experience of the targeted

application/use case (it is also

the required sustainable date

rate).

To measure this KPI, for each individual

UE and end-user application, the

application throughput across time is

measured and recorded. For DL, UE

device application must be able to

provide this info in some format (e.g.

Built-in KPI log file managed by android

smartphones). For UL, several alternatives

exist such as a PC equipped with

Wireshark sniffing traffic.

Peak Data

Rate -

Broadband

connectivity

It is the high data rate

provisioned during high

traffic demand periods (It is

also a measure of the peak

data rate required).

The measurement procedure for Peak

Data Rate is the same as User data Rate

or Speed measurement procedure

described above.

Capacity It is measured in bit/s/m2 is

defined as the total amount

of traffic exchanged by all

devices over the considered

area.

The KPI requirement on the minimum

Traffic Volume Density/Areal Capacity for

a given use case is given by the product:

[required user experienced data rate] x

[required connection density]. “Number

of 5G System Connected Users”: Counter

available in NG-RAN node (gNB).

Counter reports average number of

connected users in the cell in the

reporting period (typically 15 minutes)

multiplied by the user data rate or speed,

as defined above.

Latency

(end-to-

end or E2E

Latency)

Measures the duration

between the transmis-

sion of a small data packet

from the application layer at

the source node and the

successful reception at


UE and end-user application, the

application latency across time must be

measured and recorded. An alternative

way is to deploy an application that can

send ping commands to a UE (RTT is


H2020-856709

the application layer at the

destination node plus the

equivalent time needed to

carry the response back.

measured). This application could be

deployed as a VNF close to Application

Server.

Device

Density

It is the number of

simultaneous active

connections per square

kilometer supported for

massive sensor deployments.

Here, active means the

devices are exchanging data

with the network.

“Number of 5G System Connected

Users”: Counter available in gNB. Counter

reports average number of connected

users in the cell in

the reporting period (usually 15 minutes).

Reliability It is the amount of sent

packets successfully delivered

to the destination within the

time constraint required by

the targeted service, divided

by the total number of sent

packets.


UE and end-user application, the packet

loss rate across time must be measured

and recorded. For UL, UE device

application must be able to provide this

info in some format (e.g. Built-in KPI log

file managed by android smartphones).

For DL, several alternatives exist like a PC

equipped with Wireshark sniffing traffic

from a mirrored port in a switch or

Wireshark on a Linux machine involved

with connectivity to Application server.

Availability It is the network availability

characterized by its

availability rate X, defined as

follows: the network is

available for the targeted

communication in X% of the

locations where the

network is deployed and X%

of the time.

“Cell Availability”: Counter available in

gNB. Counter reports the length of time

that a cell is available for service (usually

every 15 minutes).

Mobility It refers to the system’s ability

to provide seamless service

experience to users that are

moving. Seamless service

experience means that UE

achieves QoS required by the

service/ application.

To measure this KPI, the following PIs will

be used: Latency, Availability, Reliability,

User Data Rate or Speed, Capacity and

Device Density as defined above. In

addition, the “UE speed” that will be

provided by UE application or measured

through alternative means.

D4.1: Service and Core KPI Specification · 2020. 5. 29. · D4.1: Service and Core KPI...

Documents

Transcript of D4.1: Service and Core KPI Specification · 2020. 5. 29. · D4.1: Service and Core KPI...