KPI Assessment Framework - Transforming...

DELIVERABLE

D3.8 – KPI Assessment Framework

Project Acronym TT

Project Title Transforming Transport

Grant Agreement number 731932

Call and topic identifier ICT-15-2016-2017

Funding Scheme Innovation Action (IA)

Project duration 30 Months [1 January 2017 – 30 June 2019]

Coordinator Mr. Rodrigo Castiñeira (INDRA)

Website www.transformingtransport.eu

Project Acronym TT

http://www.transformingtransport.eu/

Page | 2


Version 5.0, 28/07/2017

Document fiche

Authors: Víctor Corral [ATOS], Germán Herrero [ATOS], Guillermo

Velázquez [UPM], Alfonso Román [UPM], Andrés Monzón

[UPM]

Contributors -

Internal reviewers: Mauro Gil (INDRA), Stephane Lorin (THALES).

Work Package: WP3

Task: T3.1

Nature: R

Dissemination: PU

Document History

Version Date Contributor(s) Description

1.0 15/05/2017 ATOS Deliver the first Table of Content of the document

(ToC)

2.0 07/07/2017 ATOS First draft version circulated

3.0 14/07/2017 ATOS Final draft version to be submitted for internal review

process

4.0 24/07/2017

INDRA

THALES

UPM

UDE

Inputs and comments received from internal review

process

5.0 28/07/2017 ATOS Implementation of the comments and final version to

be submitted

Page | 3


Version 5.0, 28/07/2017

Keywords: Assessment framework, KPI, Performance Targets,

Pilot Requirements, Market Impact.

Abstract (few lines): This deliverable reports on the work performed in

WP3/T3.7 “KPI Assessment Framework” continuing

the work performance in T2.2 where UPM and Atos

deliver the first common assessment framework for

assessing the performance in the 6 categories

selected to measure the results obtained in each of

the 13 TT pilots, in this occasion, we have completed

the work by delivering the methodology for the

assessment of the performance of the Strategic

Assessment of TT by using Project Level KPIs.

DISCLAIMER

This document does not represent the opinion of the European Community, and the European

Community is not responsible for any use that might be made of its content. This document

may contain material, which is the copyright of certain TT consortium parties, and may not be

reproduced or copied without permission. All TT consortium parties have agreed to full

publication of this document. The commercial use of any information contained in this

document may require a license from the proprietor of that information.

Neither the TT consortium as a whole, nor a certain party of the TT consortium warrant that the

information contained in this document is capable of use, nor that use of the information is free

from risk, and does not accept any liability for loss or damage suffered by any person using this

information.

ACKNOWLEDGEMENT

This document is a deliverable of Transforming Transport project. This project has received

funding from the European Union’s Horizon 2020 research and innovation programme under

grant agreement Nº 731932

Page | 4


Version 5.0, 28/07/2017

Table of Contents DEFINITIONS, ACRONYMS AND ABBREVIATIONS ................................................................................................... 6

EXECUTIVE SUMMARY ........................................................................................................................................... 7

1 INTRODUCTION ............................................................................................................................................ 8

1.1 JOINT EXPLOITATION, MARKET IMPACT & KPI ACHIEVEMENT ....................................................................................... 9 1.2 PROCESS FOR REPORTING KPI ACHIEVEMENT FROM TT PILOTS .................................................................................... 12

2 KPI ASSESSMENT FRAMEWORK .................................................................................................................. 14

2.1 EVALUATION FRAMEWORK STRUCTURE-LEVELS FOR TT .............................................................................................. 16 2.2 OBJECTIVES OF THE STRATEGIC ASSESSMENT ........................................................................................................... 17 2.3 SPECIFIC METHODOLOGY OF THE STRATEGIC ASSESSMENT LEVEL AND MARKET IMPACT ANALYSIS ....................................... 18

3 STRATEGIC ASSESSMENT LEVEL DEFINITION ............................................................................................... 23

3.1 ANALYSING THE STRATEGIC TT ASSESSMENT CATEGORIES .......................................................................................... 23 3.2 IMPLEMENTATION OF THE STRATEGIC ASSESSMENT LEVEL .......................................................................................... 25 3.3 OVERVIEW OF KPIS PER STRATEGIC ASSESSMENT GLOBAL TARGETS .............................................................................. 27 3.4 KPI PERFORMANCE ACCORDING TO SCENARIOS ........................................................................................................ 31

4 MARKET IMPACT ........................................................................................................................................ 32

4.1 STAKEHOLDERS MAPPING AMONG PILOT DOMAINS VALUE CHAIN ................................................................................. 33 4.1.1 SMART HIGHWAYS ......................................................................................................................................... 35 4.1.2 CONNECTED VEHICLE ..................................................................................................................................... 35 4.1.3 PROACTIVE RAIL INFRASTRUCTURE .................................................................................................................... 36 4.1.4 PORTS AS INTELLIGENT LOGISTICS HUBS ............................................................................................................. 37 4.1.5 SMART AIRPORT TURNAROUND........................................................................................................................ 37 4.1.6 INTEGRATED URBAN MOBILITY ......................................................................................................................... 38 4.1.7 DYNAMICS SUPPLY NETWORKS ........................................................................................................................ 39 4.2 IDENTIFICATION OF TT OUTCOMES OF INTEREST ....................................................................................................... 39

5 CONCLUSIONS AND NEXT STEPS ................................................................................................................. 50

6. REFERENCES ............................................................................................................................................... 51

ANNEX 1- STRATEGIC ASSESSMENT KPI LEVEL ..................................................................................................... 52

1. SUPPORTING ECONOMIC GROWTH ......................................................................................................................... 53 2. BOOSTING THE BUSINESS PERFORMANCE OF OPERATIONS IN PILOTS .............................................................................. 56 3. IMPROVED ENVIRONMENTAL AND ENERGY CONSUMPTION .......................................................................................... 60 4. ENHANCE THE TRANSPORT BIG DATA RESEARCH AND POLICY ECOSYSTEM ....................................................................... 62

Page | 5


Version 5.0, 28/07/2017

List of Figures Figure 2: Updated logic between Tasks and WPs related to T2.2. and T3.1 ................................ 10

Figure 3: Comparison between the assessment objectives of KPIs tasks .................................... 11

Figure 4: Pilot and project level KPIs alignment ........................................................................... 15

Figure 5: Four levels of assessment within TT .............................................................................. 16

Figure 6: Pilot Assessment Level ................................................................................................... 16

Figure 7: Methodology stages for assessing the impact of TT ..................................................... 19

Figure 8: Strategic TT assessment categories ............................................................................... 23

Figure 9: List of topics per Strategic Assessment Category .......................................................... 25

Figure 10: Transport system diagram .......................................................................................... 34

Figure 11: Stakeholder mapping in Smart Highways .................................................................... 35

List of Tables Table 1: Evaluation Results Template ........................................................................................... 13

Table 2: Objectives of Stage 1 of the methodology for the Specific Assessment ........................ 20




Table 6: Group of selected outcomes of interest of TT project. Source: Adaptation from SUMMA

....................................................................................................................................................... 40

Table 7: Definition of TT Economic outcome................................................................................ 42

Table 8: Definition of TT Technical outcome ................................................................................ 43

Table 9: Definition of TT Environmental outcome ....................................................................... 44

Table 10: Definition of TT Social outcome .................................................................................... 45

Table 11: Expected impacts at TT project level ............................................................................ 48

Page | 6


Version 5.0, 28/07/2017

Definitions, Acronyms and Abbreviations Acronym Title

AM Asset Management

CO2 Carbon Dioxide

DL Deliverable Leader

DoA Description of Action

Dx Deliverable (where x defines the deliverable identification number e.g. D1.1.1)

EC Energy Consumption

EEA European Environmental Agency

EF Economic

EQ Environmental Quality

EU European Union

GHG Green House Gas

KPI Key Performance Indicator

LCV Light Commercial Vehicle

NOx Nitrogen Oxides

OE Operational Efficiency

PC Project Coordinator

PO Project Officer

PT Priority Topic

PU Public

R Report

SF Safety

TL Task Leader

TT Transforming Transport

Tx Task (where x defines the task identification number e.g. T1.1)

Page | 7


Version 5.0, 28/07/2017

Executive Summary The overall goal of the Transforming Transport project (TT) is to demonstrate in a realistic,

measurable, and replicable way the transformative effects that Big Data will have on the

mobility and logistics sector. The project is designed to validate Big Data as capable of

reshaping transport processes and services significantly increasing operational efficiency,

improving customer experience, and fostering new business models. All of this will be achieved

by demonstrating, evaluating and validating, in real operational scenarios (Pilots) belonging to

different transportation domains, the capability of Big Data innovations to develop more

efficient solutions.

This deliverable reports on the work performed in WP3, Task T3.1, specifically focussing on the

aspect of KPI assessment. The goal of the task is to develop the framework that will perform

the technical, operational, economical, and societal evaluation of the aggregated results

obtained in each of the 13 pilots in TT. The deliverable covers the process of design,

development and actual deployment of the TransformingTransport Key Performance Indicator

(KPI) Assessment framework, which will be used to assess the Pilots involved in the project and

their results.

This deliverable provides a definition of a holistic set of KPIs arranged in a common assessment

framework for TT. The KPI assessment framework helps determining and estimating as

accurately as possible the benefits that different innovations in each pilot domain might bring,

and the impact occasioned in different assessment categories to multiple stakeholders of the

Pilots. It is not intended to halt any of the innovations at the R&I stage, in case we don’t get the

performance we estimated, but to frame in what circumstances and scenarios they can be

useful in order to widen the portfolio of solutions available for decision-makers. It should be

noted that not only the numerical results of the KPIs are important, but also the frame and the

interpretation of the results according to the circumstances. Also, experience gained by

measuring the KPIs may lead to further improvements of the KPIs’ definitions and application

methodology from pilot domain as well as from project perspective.

This deliverable is complement by an exploitation methodology presented in the TT Impact plan

(deliverables D3.7 and D3.10), which will allow us to identify which commercial opportunities

arise from pilot and project execution to leverage the possibility the transferability of the

results from the research programs to the transport market.

Page | 8


Version 5.0, 28/07/2017

1 Introduction

The overall goal of TT is to demonstrate in a realistic, measurable, and replicable way the

transformative effects that Big Data will have on the mobility and logistics sector. The project is

designed to validate Big Data as capable of reshaping transport processes and services by

significantly increasing operational efficiency, improving customer experience, and fostering

new business models. All of this by demonstrating, evaluating and validating, in real operational

scenarios (Pilots) belonging to seven different transportation domains, the capability of Big

Data innovations to develop more efficient solutions. To achieve this objective, the project is

based on a consolidated methodological approach characterised by three main activities:

- Defining both global and pilot domain performance targets.

- Testing the innovations in real operation conditions.

- Evaluation and validation of the potential impact of Big Data innovations.

Section 1: summarizes the main objectives of T3.1 Joint Exploitation, Market Impact & KPI

achievement, concretely in the scope of D3.8 KPI Assessment Framework. This section provides

an integrated framework overview of both tasks, T2.2 and T3.1. The alignment, objectives and

outcomes are presented in this section.

Section 2: It has been presented the fourth level of Assessment which is the Strategic

Assessment. The concrete objectives of this level and also, the methodology stages of this level

have been defined.

Section 3: presents the assessment categories that will be evaluated under the Strategic

Assessment Level. It has been indicated which formula will be used to evaluate the aggregated

performance obtained in each of the categories. As well as, it has been selected, defined and

explained which KPIs will be used to assess the performance of TT at a project level. The way to

measure the degree of target achievement is also defined.

Section 4: provides a preliminary overview of the main TT impacts, categorised depending of

the outcomes that TT is seeking to produce. The impacts have been fed by the preliminary

identification carried out by Pilot domains in their priority topics.

Page | 9


Version 5.0, 28/07/2017

1.1 Joint Exploitation, Market impact & KPI achievement

The main objective of T3.1– Joint Exploitation, Market Impact & KPI achievement described in

the DoA is “Set up the TransformingTransport KPI assessment framework, which will satisfy a

set of basic needs of the private sector intending to investigate the adoption of

TransformingTransport solutions”. On top of that, there are additional objectives that T3.1 is

aiming to reach:

Objective 1: Implementation of the Strategic Assessment Level (KPI Project Level)

based on the Four Level of Assessment within TT KPI Framework.

o The final result arisen from the co-creation between Atos an UPM (leading

partners in KPIs) has been reflected in the delivery of the final integrated KPI

Assessment Framework between T2.2 and T3.1. Due to the importance to build

up a common framework for assessing the performance of TT at a pilot and

project level, it has been decided to start working on such task since M3, and

finalise the implementation and methodology of the Strategic Assessment level

(Project Level) in this deliverable, as explained afterwards in sections 2.1 and 2.2.

Another important outcome has been delivered such as setting up the

Methodology or Structure of Work for the Assessment framework development

(Figure 3).

Objective 2: Selection of the project KPIs that will depict which is the penetration and

performance of the TransformingTransport Big Data ecosystem. In addition to

indicating the value delivered by TransformingTransport with the different pilots, these

KPIs will be provided as input to T3.1 “Joint Exploitation, Market Impact & KPI

Achievement”, in order to determine higher-level KPIs (as stated in the work

programme) and thus determine potential market impact. Furthermore, for M18, we

will collect the results achieved concerning which is the degree of KPI achievement

accordingly to the performance targets set up for each of the KPIs selected.

Objective 3: Performance of the methodology needed for the identification of the

main market impacts achieved by TT project as well as, which stakeholders could

benefit for the positive outcomes in transport and logistics areas achieved through the

implementation of big data solutions in pilot location. Some of the areas under

assessment are business improvement, socio-economic aspects, environmental and

energy impacts, etc.

Page | 10


Version 5.0, 28/07/2017

Taking as a basic the “Figure 2: Logic between Tasks and WPs related to T2.2” included in D2.2,

we have updated it in order to include which are the connections with, and rationale behind,

T2.2 and T3.1. Specifically, this figure shows the flow of KPI-related information. Concretely,

between both deliverables related with the project’s KPIs, D2.2 – Analysis of Pilot Requirements

for Big Data Use and D3.8 – KPI Assessment Framework. Furthermore, we have advanced in

the identification of liaisons with other WPs and Tasks of the project to understand better the

interconnections with the overall project management structure.

Figure 1: Updated logic between Tasks and WPs related to T2.2. and T3.1

As described in the above figure, T2.2, and T3.1, through its dedicated deliverables, will set up

the methodology and the framework needed to report the consolidated figures and results

obtained by the pilot in terms of KPI achievement. Furthermore, it will set up the necessary

process steps to implement the Strategic Assessment level evaluation using “Project level KPIs”.

D3.8, which will serve as a guideline for the evaluation and validation activities delivered later

in D3.11 and D3.12. Also, the work carried out in D*.2 – Performance Assessment Plan (for each

pilot domain) [M9] will be considered to build up the performance targets at project level. In

addition, D*.2 will explain concretely how pilot-specific KPIs will be measured and assessed.

Task 2.2 will also feed the development of the day to day monitoring of pilots (to be

implemented in T2.1).

Page | 11


Version 5.0, 28/07/2017

Concretely, in order to provide better insights to the whole KPI Assessment Framework created

for the TT project, the objectives and expected outcomes for each document and task are

clearly defined under D2.2 and D3.8 boxes in Figure 2. These are as follows:

Tasks: T2.2; D2.2 – Analysis of Pilot

Requirements for Big Data Use T3.1; D3.8 – KPI Assessment Framework

Objectives:

To define domain-specific and transversal KPIs and their corresponding baselines, ensuring that they remain comparable to perform a cross pilot assessment.

To perform a thorough intra-domain and cross-pilot analysis on the use of Big Data technologies.

To offer an analysis of applicability of TT Big Data solutions in post-project replications.

Implementation of the Strategic Assessment Level (KPI Project Level) based on the Four-Level Assessment methodology within TT KPI Framework.

Selection of the project’s KPIs that will depict which is the penetration and performance of the TransformingTransport Big Data ecosystem.

Performance of the methodology needed for the identification of the main market impacts achieved by the TT project.

Outcomes:

Domain specific and transversal KPIs.

Perform intra-domain and cross-pilot assessment.

Analysis of post replication strategies.

Assessment framework level; Strategic Level.

Selection of project-level KPIs.

Market impact.

KPI achievement reporting in:

D3.11 – Period 2 Impact & KPI achievement report and Plan (M18)

D3.12 – Period 3 Impact & KPI achievement report and Plan (M30)

Figure 2: Comparison between the assessment objectives of KPIs tasks

Thus, based on the performance outcomes resulting from D*.2 delivered in M9, it will be

feasible to identify which performance targets will be utilised in the Strategic Assessment Level

in order to evaluate it according to the expected impact of TT already allocated in DoA, also

complemented with the different impact tables (such as Table 12, Table 13, etc.). The process

for setting up the performance targets at the Strategic Assessment Level will be carried out

between M10-M11, and its achievements and progresses will be reported in M18 and M30.

Page | 12


Version 5.0, 28/07/2017

1.2 Process for reporting KPI achievement from TT pilots

As indicated in Figure 3 (also in Figure 2), the KPI achievement reporting expected in D3.11 and

D3.12 will be undertaken under the T2.2. and T3.1 task. For this purpose, we have been aligned

with T2.1 to introduce the required inputs needed in D*.2 to establish the quantification of the

performances achieved by the pilot domains. The alignment obtained is based on the inclusion

of the following sections in the prior KPI tables (included in Annex I of D2.2):

Data sources: this deliverable intends to advance in the KPI definition provided in D2.2.

in order to include in close collaboration with Data ID card to understand which is the

main data source inside the pilot’s architecture that will feed the required inputs

needed to compute the proposed equation(s) to obtain the assessment results.

Measurement procedure: in this section, the information required from partners is

aimed at understanding which process has been set up within the pilot domains in order

to measure the results obtained during project execution.

This additional information will support the achievement of the complete KPI Framework

Assessment, concretely to pilot domains to have a wide overview of the data needed for the

dedicated KPIs in each of the WP scenarios, as well as, to have a common measurement

procedure to analyse the performance within the selected evaluation timeframe. In relation

with this point, one important aspect that is worth being mentioned is the need to establish an

agreed due date for submission of the performance assessment of the KPIs defined for each of

the pilots before M18 in order to arrange the time required by T2.2. and T3.1 to prepare D3.11

and D3.12 reports in close collaboration with each of the Pilot domain leaders. For this purpose,

it has been established that pilot leaders must report the full figures and results measuring the

performance of their activities by March 2018 (M15).

Therefore, by M15, Pilot leaders should report by using preliminary template to collect the results of the planned evaluation activities which have been created to ensure a uniform and high level of consistency of results across the evaluation of TT pilot domains. The template for the Measure Evaluation Results is presented in Table 1. The objectives of this template are:

Facilitate KPI performance results collection in just one single template structure.

Ensure that Pilot leaders will report all the evaluation-relevant information needed to prepare the KPI achievement reports in M18 and M30.

Provide clear findings to assess which is the potential exploitation/replication of the big data solution in other transport scenarios by identifying the innovation’s novelties and the improvements accomplished in the priority topics established by Pilots domains.

The first draft/attempt of the template to collect the results for assessment process is reflected in the following table:

Page | 13


Version 5.0, 28/07/2017

Table 1: Evaluation Results Template1

At this time, a project lifetime advances; this table will be improved continuously in order to reach the highest quality reports based on the findings and inputs from pilot domain execution. Information collected by Pilot domain leaders (leader and replica) in the process of the assessment activities will be imported to the results template. Pilot leaders are responsible for completing this Evaluation Result Template. The completed templates are then sent to the KPI Assessment team (T2.2. and T3.1) who will perform a basic quality and completeness review process. This will be used as input when the KPI Assessment team summarises the results in the project evaluation report to complete the dedicated deliverables for reporting KPI Achievement D3.11 and D3.12.

1 Adapted from Box 1: Measure Evaluation Results Template; D4.10 – Evaluation Framework in CIVITAS

Page | 14


Version 5.0, 28/07/2017

2 KPI Assessment Framework

The main objective of the evaluation framework is to structure the overall assessment of the

project and the pilots, taking into consideration the specific impacts per assessment category

and the overall effects on the level of efficiency of the pilot domain and the transport sector.

From the early stages of the project development, UPM-Transyt and Atos have realised the

importance of co-creating the KPI assessment Framework in order to reach the highest level of

alignment between the two tasks (T2.2 and T3.1) which are mainly involved in TT KPI definition.

The similarities and convergence points identified between both contributing parties requires

comprising a common framework to assess the performance of the TT project in four levels.

Three of these are mainly related with Pilot KPI levels and, the remaining stage is associated to

the Project KPI level.

The main objective of the evaluation framework is to structure the overall assessment of the project and the pilots, taking into consideration the specific impacts per assessment category and the overall effects on the level of efficiency of the pilot domain and the transport sector. In this sense, the rationale behind the KPI Assessment Framework at project level is to provide holistic and aggregated figures on the performance obtained with the development and deployment of big data solutions within each of the pilot domains. An initial outline of the alignment that TT is aiming to reach between the KPIs tasks (T2.2 and T3.1) is presented below:

PIlot KPIs defininition

Project-level KPIs initially expressed in section 2.1 of

DoA

Overall TT Assesment Framework

D2.2: Pilot

Requirement Analysis

D3.8: KPI Assessment

Framework

Page | 15


Version 5.0, 28/07/2017

The result derived from the combination of both tasks in just one single integrated Assessment Framework, from the pilot perspective as well as from the project level side, is represented in Figure 4:

Figure 3: Pilot and project level KPIs alignment

Starting from the left side, the figure taken from D2.2 (Figure 4: Transforming Transport

Evaluation Process) represent the process to be undertaken for the assessment at pilot level.

Afterwards, in the right-hand side of the illustration, the five main categories used for assessing

the performance of TT at project level are presented. It is relevant to highlight that the project

level categories have been selected based on the inputs at pilot-level, due to the fact that

(some) of the KPIs selected for the project-level are fed by the results and performances

achieved in the pilot domains. Finally, the main outcomes that once aggregated will build-up

the TT KPI Assessment Framework required for D3.8 have been presented and explained. This

figure will allow understanding which is the resulting alignment between the two different KPIs

dimensions.

Moreover, the TT Assessment framework will establish the guidelines that will subsequently be

used during the assessments performed in Months 9, 18 and 30, and the lesson learned analysis

planned for month 26, in order to identify synergies among Pilots, implement a market

potential analysis of the innovations, and define the transferability guidelines for the

implementation of Big Data solutions in other European sites.

The results arisen from this co-creation between multiple project partners has been reflected in

the release of the Evaluation Framework structure-levels for TT depicted in Figure 4: Four levels

Page | 16


Version 5.0, 28/07/2017

of assessment within TT. The need to understand how TT will perform the assessment process

from the bottom to the top level is identified.

2.1 Evaluation Framework structure-levels for TT

The evaluation will be performed at four levels (Figure 4: Four levels of assessment within TT):

Figure 4: Four levels of assessment within TT

Below, it will be specified which are the implications of each of the Four levels of assessment

within TT Framework already included in D2.2:

Pilot-Category and Pilot-Objective assessment (covered in D2.2): based on local

analysis and focused on the expected impacts at each individual pilot in the different

categories. When the use of Big Data technology has effects on only a segment of the

pilot’s activity, an upscale exercise to the whole pilot’s activity will be attempted. For

this assessment level, a combination of the General Assessment Categories through

both transversal and specific sectorial PTs and KPIs, and of a qualitative assessment of

the pilot objectives will be used.

Figure 5: Pilot Assessment Level

Page | 17


Version 5.0, 28/07/2017

Domain-Category assessment (covered in D2.2): above the Pilot Assessment level, an

assessment based on a cross analysis for each pilot domain, crossing the impact attained

at both the initial and the replica pilots for each of the assessment categories. for this

assessment both transversal and specific sectorial PTs and KPIs will be used.

Horizontal Category assessment (covered in D2.2): by General Assessment Category, to

identify and evaluate the impact of Big Data use on the Category throughout the

different Pilot domains. For instance, the Energy Consumption Category evaluates the

influence of Big Data use in transportation fuel consumption, or in any other energy

related variables. For this assessment only transversal PTs and KPIs will be utilised. The

economic category is the categorization that will not include a transversal assessment

across pilot domains.

For the purpose of this deliverable, progress in the definition of the strategy, methodology, and

roadmap needed to set up the Strategic Assessment are necessary. Contributing parties’ efforts

during the first 6 months of the project have focused in developing the remaining Three levels

of the assessment which are related with the definition of Assessment for Pilot domains. Taking

this into consideration, the new stage intends to perform the Strategic Assessment which is

considered as the Strategic (Project) Assessment level. Based on this perspective, TT’s last and

fourth level of assessment is described as follows:

Strategic assessment (covered in D3.8): This appraisal combines the contribution of all

Pilots to reach the global strategic targets determined from the expected impact of TT.

This level of evaluation will quantify the foreseen effects if TT technologies were jointly

tested and applied by also identifying and taking into consideration factors which

provide synergic achievements and better results. For this assessment only global PTs

and KPIs will be used (section 3).

2.2 Objectives of the Strategic Assessment

The objectives of the Strategic Assessment are now described:

1) To support the effective deployment of the Strategic Assessment Framework to depict

which performance of the overarching goals of the R&I activities designed in the

transport domains of the TT, and embedded within the TT Assessment Framework

(section 3). The TT KPIs aims at qualifying the outcomes of R&I activities in transport and

logistics sector to boost the adoption of Big Data solutions in both, in accordance with

the expected impacts allocated in DoA. The results of the performance analysis of those

KPIs will help policy makers, regulators and network operators in utilising the results of

R&I activities to organise and manage the large-scale deployment of innovative

solutions. Scalability and replicability of the results are two of the main factors that TT is

Page | 18


Version 5.0, 28/07/2017

intending. So, for this purpose, the aggregated benefits provided by the demonstrations

made in pilot domains must be highlighted, and this will be granted by evaluating the

results achieved in the performance targets at both project and pilot level.

2) In parallel, the document will set up the processes and mechanisms that depict the

market penetration of TT solutions (section 4) in the big data environment of transport

projects. The first step is to identify the expected impacts of each pilot domain based on

the objectives and requirements established in Pilot Design (delivered in M3)

complemented with the Pilot KPI definition exercise already performed in D2.2, as well

as, carrying out a preliminary analysis of which stakeholders are placed in each pilot

domain. However, at this stage, due to the lack of operational results coming from

pilots, the objective is to suggest the initial conditions to collect, quantify and present

the aggregated TT market penetration. In parallel, TT is connecting the market

penetration conditions of TT (expected impacts, stakeholders and benefits) with the

exploitation strategy defined among the consortium partners. One of the key challenges

of TT is enhancing the market uptake of Big Data solutions in transport, so it is required

to perform the market penetration process and its mechanisms to discover which are

the most promising technical and business knowledge/solutions developed by TT pilots,

demonstrating to the transportation sector stakeholders that the implemented

solutions are scalable and replicable within the concrete transport domain.

2.3 Specific methodology of the Strategic Assessment level and Market impact analysis

The methodology should be described as a process including several steps of information gathering, evaluation, validation within targeted stakeholders, as well as the usage of the outcomes obtained to transform intro joint exploitation scenarios, and promote the communication of the benefits behind TT to raise the awareness of the deployment of Big Data solutions within Transport and Urban Mobility environment, specially to the wide range of transport modes and value chains represented in TT pilots. For this purpose, TT is aimed towards setting up the following methodology to be carried out in order to guarantee that the expected impacts described in DoW are being achieved through the development of the project. The stages which are mainly divided this methodology are depicted below:

Page | 19


Version 5.0, 28/07/2017

Figure 6: Methodology stages for assessing the impact of TT

STAGE 1: Design of the Impact Assessment Roadmap for KPIs as a project level: Firstly, the design of an Impact Assessment Methodology for KPIs as a project level is required to present in early stages of project development which indicators will conform the main framework, followed by a concrete work plan which partners should comply in order to feed the Project KPIs aggregated data. TT should guarantee the quality of the data collected as well as preserve its accuracy by providing to the Pilot domains the detail of concrete activities that should implemented in order to gather the data and transform it into accurate units for measuring the performance in terms of Project level categories. Most of the information needed from partners related with aggregated KPIs data will be collected through the coordination and alignment between WP2, concretely D2.2-Analysis of Pilots requirements in Big data use and WP3, concretely by D3.8-KPI Assessment Framework. The objectives of this phase are now summarised:

STAGE 1: Design of the Impact Assessment Roadmap

for KPIs as a project level

Cod Objectives Timeframe

Obj1 Identification and categorization of the impacts effecting TT in each of the pilot domains

M6

Obj2 Gather and aggregate the impacts M6

Design of the Impact

Assesment Roadmap

(project level)

Performance of the KPI collection

activities

Evaluate the KPI performance

targets reached

Exploitation, communication

and transferability of

the assesment results obtained

STAGE 1

STAGE 2

STAGE 3

STAGE 4

Page | 20


Version 5.0, 28/07/2017

affecting TT pilot level to contribute to the configuration of impacts at project level

Obj3 Discover the benefits derived from the effects caused by project impacts

M18

Obj4 Identify the list of stakeholders based on the specific value chain of the pilot domains

M6

Table 2: Objectives of Stage 1 of the methodology for the Specific Assessment

STAGE 2: Performance of the KPI collection activities After the identification of which impacts, benefits and stakeholders are placed in each of the value chain network of pilot domains, advances in the performance of assessment reporting activities are now. At this stage, the objective is to perform the assessment reporting by gathering the data resulting from Pilot domains to aggregate into the KPI tables in order to compile this information into the aggregated KPIs used at project level. Guidelines will also be provided to partners to identify those areas and aspects which should be monitored to afterwards translate into the KPI aggregated list. The objectives of this phase are summarised below:

STAGE 2: Performance of the KPI collection

activities


Obj4 Open a call to action to collect the information from the pilot domain leaders by using Table 1: Measurement Evaluation Results Template

M14

Obj5 Aggregate the data collected from pilot domains into the KPI project level

M15


STAGE 3: Evaluate the performance targets reached As a continuation of the performance activities, once the innovations have been implemented and the tests are completed, the contribution of Big Data in each pilot case is assessed by comparing the results for the KPI variations with the expected/planned outcomes expressed by its related Performance Target. So, the evaluation will be performed by analysing relative indicators and by aggregating the results into a global rating.

Page | 21


Version 5.0, 28/07/2017

The refinement of impact strategy, the information gathering on the achievements of TT, its assessment in terms of impact, and the relevant strategy decided according to the results will be reported (mainly on M18 and M30) through the dedicated deliverables. The objectives of this phase are summarised below:

STAGE 3: Evaluate the performance targets

reached


Obj6 Evaluate the results obtained in order to draft the conclusions resulting from pilot execution

M16 Obj7

Document the results achieved as well as draft a first set of conclusions to be included in D3.11 Period 2 Impact & KPI Achievement Report and Plan

Obj8 Present the results of TT KPIs project level evaluation to the project partners

M17

Obj9 Submission of the deliverable into EC portal M18 Table 4: Objectives of Stage 3 of the methodology for the Specific Assessment

STAGE 4: Exploitation and Communication of the assessment results obtained The final stage of the methodology is focused on communicating and exploiting the results achieved after the assessment process. The whole methodology should be focused not only on the achievements of the task but also on disseminate and communicate the global results obtained among our target audience; Public Bodies, End-users, Transport Operators, mainly to the audience identified in D3.1-Initial Impact Plan following the guidelines allocated in this document to follow-up the joint strategy settle down. The dissemination and communication of the results provide us the necessary tools to spread and raise the awareness of the benefits of deploying big data measures and techniques into concrete transport sectors (Ports, Airports, Railway, etc.) where TT pilot domains are focusing. The multiple benefits arisen from business performance, economic growth, environmental and energy, as well as society categories, will be used to spread the innovations within our target audience and will allow them to understand which positive impacts have effected into the assessment categories identified. Furthermore, this information will allow us to sketch a set of lessons learned from the pilot’s execution that will enhance the transfer knowledge and cross-fertilization strategies related to the adoption of big data measures and techniques into the different transport domains. The objectives of this phase are summarised as follows:

Page | 22


Version 5.0, 28/07/2017

STAGE 4: Exploitation and communication of the

assessment results obtained


Obj10 Identification of the main outcomes achieved at TT project level

M18

Obj11 Select and communicate the outcomes and benefits obtained to our targeted audience for this first KPI reporting milestone in M18

M19

Obj12 Draw a lessons learned sketch from first KPI reporting milestone in M18

M19-20


Page | 23


Version 5.0, 28/07/2017

3 Strategic Assessment Level definition

3.1 Analysing the Strategic TT Assessment Categories

The Strategic TT Assessment Categories is composed of five main global targets. In each of the

categories indicated, there is a set of concrete objectives which define the objectives followed

by the consortium to evaluate the wide range of effects encountered when developing Big data

measures and technologies in the transport and urban mobility value chain environment.

Figure 7: Strategic TT assessment categories

Below, a description of each of the Strategic assessment global targets will be provided:

GT1: Supporting Economic Growth: this category can be defined as the evaluation of the effects on the level of economic activity in a given area. In the case of TT project, the main areas under study are being covered by the KPIs defined in the pilot domains which are related with the innovation’s operation efficiency, cost reduction, etc. It’s important to monitor the performance and results obtained in economic aspects, since cost is a key driver in transport operation activities. It will also serve as a guideline for

Economic Growth

Energy and enviromental

Research and policy

Business performance

Page | 24


Version 5.0, 28/07/2017

the exploitation strategy to increase the business potential of the TT outcomes and its replicability in transport and logistic big data solutions.

GT2: Boosting the business performance of operations in pilots: The objective is to evaluate if the innovation and the technology designed and deployed in the TT project are aligned with the technical expectation of stakeholders and with the predictable impact described in Table 13: Improvements with respect to technological constraints allocated in DoW.

GT3: Improved environmental and energy consumption: This category assesses one of

the main objectives of the TT project which consists in preserving the environmental

aspects as well as improving the energy efficiency during the operation of different

transport networks/modes and, therefore, the energy requirements of those systems.

GT4: Enhance the transport big data research and policy environment: These have

been set up with the goal of bringing together transport stakeholders to foster

innovation and integration in research, including the development of long-term visions

for transport infrastructure at a European level. This category will contribute to address

and retrieve the information allocated in DoW related to technological constraints from

pilot domains.

Each of the aforementioned assessment categories underlies a concrete set of selected topics

which provides a better understanding of which expected impacts must be measured in order

to consequently establish the more appropriated KPIs. There are strong links between the

topics of all the categories due to their nature and area covered. However, each TT categories

topics are relevant for the discussion and are analysed separately in the following figure:

Page | 25


Version 5.0, 28/07/2017

Figure 8: List of topics per Strategic Assessment Category

3.2 Implementation of the Strategic Assessment Level

The last level of the assessment approach is the Strategic Assessment which combines the

contribution of all Pilots to reach the Global Targets set by the expected impact. This step will

allow quantifying the predictable impacts when all pilot domains are applied together by also

considering which influential factors contribute to achieve more synergies and better results.

For this purpose, we have identified, based on the TT objectives and expected impact from a

project and pilot perspective, which are the main Core Performance Targets that will best

measure the level of achievement of the global targets established in TT.

The selection of the Core Performance Targets is based on the identification made by Pilot

Leaders in their Pilot Priority Topics, as well as the expected impacts indicated in the DoA. As an

initial approach, the selected Core PTs are:

GT1: Supporting Economic Growth

Overall transport and logistics productivity

Investements leveraged

Market share of TT

GT2: Boosting the business

perfomance of operation in pilots

Improvement of transport/mobility

operations

Perform the current product/service

portfolio

External-Joint collaboration within

Third parties

GT3: Improved environmental and

energy consumption

Pollutant emissions Energy savings

GT4: Enhance the transport big data

research and policy environment

Knowledge transfer from research to

business

Page | 26


Version 5.0, 28/07/2017

Global strategic targets Core PTs

Supporting Economic Growth


Reduction of the average travel times / Time

savings

Investments leveraged

Market share of TT

Boosting the business

performance operation in pilots

Improvement of transport/mobility operations

Reduction of operating cost

Perform the current product/service portfolio

New services/business models

External-Joint collaboration within Third parties

Number of external collaborations

Improved environmental and

energy consumption

Pollutant emissions

Energy savings

Enhance the transport big data

research and policy environment

Knowledge transfer from research to business

Thus, a set of Objective Functions (OF) will be developed which will try to maximise all the

strategic targets. An OF is a function of a sub-set of CORE PTs, to be used for strategy appraisal

or for optimization purposes.

Our OF is defined as follows:

OF = f(CPTs)

Where CPTs are the Core Performance Targets selected to measure the global strategic targets.

The measurement of the CPTs will be based on the variation of corresponding KPIs between

scenarios.

Thus, the objective functions related to each strategic target are:

- Supporting economic growth: EG = f (PT xx, PT xx, PT xx)

- Boosting the business performance operation in pilots: BPO = f (PT xx, PT xx, PT xx)

- Improved environmental and energy consumption: EE= f (PT xx, PT xx, PT xx)

Page | 27


Version 5.0, 28/07/2017

- Enhance the transport big data research and policy environment: RP= f (PT xx, PT xx, PT

xx)

The Global impact of all the technologies implemented together in the project is:

Where α, β, γ and δ are weights assigned by a representative group of stakeholders for the

different strategic targets. A consultant process will be carried out to obtain the corresponding

weights.

3.3 Overview of KPIs per Strategic Assessment global targets

The last level of the assessment approach is the Strategic Assessment which combines the

contribution of all Pilots to reach the global targets set by the expected impact. It will allow

quantifying the predictable impacts when all TIs are applied together by also considering which

influential factors contribute to achieve more synergies and better results.

The proper assessment of the interventions made during the project against the project objective is based on criteria, which are considered representative for the decision to be taken. Criteria usually refer to certain parameters that should be taken into account during the evaluation, such as production, input use and conditions of the project beneficiaries. The criteria are then translated into indicators, which represent yardsticks to measure the impacts. Adequate indicators are able to (1) assess the project’s performances, (2) measure progress, including the identification of potential problems or successes and (3) ensure knowledge transfer and cross-fertilization strategies to selected targeted stakeholders. Transforming Transport has elaborated a set of indicators including environmental, research

and policy and other socioeconomic impacts. The TT assessment procedure gives to policy,

research, business and society environment surrounding TT transport modes, the possibility to

evaluate the wide range of benefits provided by Big Data measures and technologies, and their

consequences on transportation network performance. It is applicable to both ex-ante

assessments and ex-post evaluations. The objective is to perform an assessment baseline to

demonstrate the benefits and impacts of applying Big Data analysis to transportation activities.

The comparison achieved will allow us to evaluate the results accomplished in each of the TT

categories to draft a lessons-learned statement from project and pilot competition.

The final aim of the selected KPIs is to ensure that, by defining an appropriate quantity of such

indicators, the objectives of the project are duly covered (from operational efficiency, to energy

management, economic sustainability and improvement, and raising the awareness of business

TT Global Impact = ∑(α·EG + β·BPO + δ·EE + γ·RP)

Page | 28


Version 5.0, 28/07/2017

community specially SMEs). With these suppositions under consideration, the KPIs for this

research work have been selected in accordance with the preceding composition of Project-

level KPIs initially expressed in Section 2.1 of DoA and the Pilot KPIs (see D2.2) whose

aggregated results will support the accomplishment of the performance objectives.

Page | 29


Version 5.0, 28/07/2017

TT Project KPI Assessment Category Related subtopics Code KPIs Liaison with DoA Units



EG-PR-1 Aggregated productivity increase in the TT pilot domains

“Demonstrated increase of productivity

in target sector by at least 15%” [DoA, Sec

2.1.1]

%

Investments leveraged

EG-IL-1 Total amount of additional target sector investments of TT industrial partners

“Leveraging additional target sector

investments of more than 6 times the EC

investment” [DoA, Sec 2.1.1]

€

Market share EG-MS-1 Total percentage of market share obtained by TT Industry Members

“Estimated increase of market share and size

of TransformingTransport

industry members” [DoA, Sec 2.1.2, Tab.

16]

%

Boosting the business performance of operations in pilots

Improvement of Transport/Mobility

operations BPO-IO-1

Total accumulated operations costs of existing processes and services reduced by TT pilot deployment

Fed by Operation Efficiency and

Economic Assessment Categories of pilot

domain "Table 12:

Improvements in cost structure of existing

processes and services" [DoA, Sec 2.1.2, Tab.

14]

€/%

Page | 30


Version 5.0, 28/07/2017

Perform the current

product/service portfolio

BPO-PS-1

Total number of new products/services/processes with big data features launched into the market

“Cost and performance of novel processes and

services” [DoA, Sec 2.1.2, Tab. 14]

Unit

BPO-PS-2

Total number of existing improved products/services/processes with big data features launched into the market

“Cost and performance of novel processes and

services” [DoA, Sec 2.1.2, Tab. 14]

Unit

External Joint-collaborations within Third

parties

BPO-JC-1

Total number of Involved organizations which are participating actively in Big Data demonstrations

“At least 120 organizations

participating actively in Big Data

demonstrations” [DoA, Sec 2.1.1]

Unit

Improved environmental and energy consumption

Pollutant emissions

EE-PE-1 Total direct CO2 emission reduction obtained

Obtained from Environmental Quality Assessment Category

(t/year)

Energy savings EE-ES-1 Total direct energy savings obtained

Obtained from Energy Consumption

Assessment Category kWh/100 km

Enhance the transport big data research and policy environment

Knowledge transfer from research to

business

RP-RTB-1 Total number of big data components integrated into pilot domain platforms

“Doubling the use of Big Data technology in

the mobility and logistics sector from

the currently 19% to at least 38%” [DoA, Sec

2.1.1]

%

The detailed description of each of the KPIs expressed in the above table, it’s included the Annex 1-KPI Strategic Assessment KPI

level.

Page | 31


Version 5.0, 28/07/2017

3.4 KPI performance according to scenarios

As described in D2.2, “The assessment of the results provided by any investment in a

transportation system is based on its validation and comparison in different scenarios. Within

this methodology, the benefits are evaluated by means of the variation between the “No-TT”,

control or ex-ante, scenario (situation without the implementation of the Big Data Technology)

and the “TT” scenario (according to the data collected when the corresponding innovations are

being implemented).” In accordance with this definition of the measurement process to

evaluate KPI performance, we are also reutilising this concept to measure the performance of

TT in the defined KPIs.

One key aspect still pending to be defined consists in setting up the performance targets at the

Strategic Assessment Level. Based on the objectives of D3.8, the calculation and definition of

the project’s performance targets is fed by the inputs coming from pilot domains. These specific

KPI performance targets will be included in the dedicated deliverables referenced as D*.2. After

the evaluation of the results and findings gathered in the referred deliverable, the Strategic

Assessment Level will allow defining the performance targets at project level. This process will

be undertaken between M10-12 in order to guarantee the accurateness and coherence of the

performance targets from both pilot and project perspectives.

The process consists in calculating the variation in KPI values before and after Big Data

techniques are integrated in pilot management decisions, as expressed in Equation (1):

𝐾𝑃𝐼𝑉𝑎𝑟 = (𝐾𝑃𝐼2 − 𝐾𝑃𝐼1

𝐾𝑃𝐼1) 𝑥100

With KPI1 reporting the performance in the No-TT scenario (before or control) and KPI2 the

performance during the TT scenario.

For each of the selected KPIs, its related definition, units of measurement and required sources

for the data provision have been preliminarily established to avoid misunderstandings during

the data procurement phase.

(1)

Page | 32


Version 5.0, 28/07/2017

4 Market impact

The Market impact of TT intends to identify which are the main aggregated impacts and

benefits from the multiple transport domains. These will be later categorised depending on the

outcomes of interests (presented in Table 6) at a project level. The stakeholder mapping will

allow us to grasp the opportunity for connecting the impacts and benefits from each pilot to

the value chain of the stakeholders’ domain. As an innovation action, the post-replication

strategies are essential to reach the highest level of transferability from R&D environment to

business scenarios. The results achieved in this phase will be used to configure which set of

project KPIs will be utilised to assess and demonstrate the pilot performance in the transport

and logistic domains.

From the supply side, for instance, the need to understand which are the impacts and benefits

of applying Big Data solutions and techniques is a prerequisite to boost its adoption by end-

users. It will enhance the decision making of the management bodies when decisions are

required concerning the implementation of big data innovations among similar organizations.

From the Demand Side perspective, the pilot deployment will be used as a test-bench to

demonstrate the effectiveness and sustainability adoption of big data solutions in the

transportation industry. For instance, the internal business portfolio of the partners involved in

each pilot could be performed by adding this kind of Big Data features into their product or

service offers, thus resulting in a market share gain for the company.

Thus, in this deliverable, we have complemented the development of the KPI Assessment

Framework by including a preliminary definition of the intend market impacts that TT is aiming

to reach. For this purpose, we have taken the information related to the Priority Topics

stablished in D2.2, concretely in section 3.2 Overview of Priority Topics. We consider that each

Priority Topic (PT) identified should be quantified through the establishment of a Performance

Target by the Pilot leader in order for the assessment to introduce a comparison between the

expected result (target) and the actual impact of the solution in the operation or business model.

So, in order to build up the initial definition of the Market impacts of TT as a whole, we have been

analyzing and defining which PTs are horizontal to all the pilot domains in order to have a holistic

composition of the impacts at a project level. Furthermore, we have also taken into account the

current expected impacts indicated in the DoA for start making the necessary comparisons for

demonstrating the potential of the adoption of Big data in transport.

The KPI Assessment Framework is more than a tool to evaluate the performance within the

Assessment Objectives, and the expected impacts, the rationale behind is to develop the necessary

resources for the pilot locations to understand how the big data has been upgrade the performance

within the priority topics as well as, a framework that depicted which impacts will be occasioned by

the pilot development.

Page | 33


Version 5.0, 28/07/2017

In the methodology presented of the Strategic Assessment, the Market impact at this project

phase is composed of the following objectives indicated in the STAGE 1: Design of the Impact

Assessment Roadmap for KPIs as a project level.

4.1 Stakeholders mapping among Pilot domains value chain

According to the transport article published by the New Zealand Government2, the supply of

transport services occurs through a combination of providing and using infrastructure across a

range of modes. Provision of infrastructure incurs capital costs and covers, for example, roads,

railway lines, airports and ports, while usage is made possible through vehicles (for example,

cars, trucks, trains, airplanes and boats) and management systems (for example, traffic lights,

signals, air traffic control, navigational aids). It is evident that the transport system is broader

than just physical transport networks and vehicles. It includes institutional settings and the soft

systems that underpin the coordination of transport services. Figure 6 provides a diagrammatic

representation of the transport system.

2 Contribution of transport to economic development – International review with New Zealand perspectives

Page | 34


Version 5.0, 28/07/2017

Figure 9: Transport system diagram3

As represented in Figure 6, within the transport and logistic value there are a set of multiple

players exchanging, sharing and selling a wide range of services, products and all are

surrounded by external actors which are part of the macroeconomic externalities of the

transport modes. The interconnections and dependencies between transportation modes

makes possible that implementing an innovative solution based on Big Data could positively

affect the complete logistic processes in the value chain among all transport modes, or could

have multi-side effects for the different stakeholders.

The wide range of transport modes, end-users, corridors and networks intervening in TT Pilot

domains reveals the need to map which concrete stakeholders are present in order to perform

an ad-hoc identification of the main impacts and benefits foreseen after pilot execution. The

individual stakeholder value chain mapping allows us to connect both sides of the business

cases presented in each D*1, the requisites established by demand and supply.

The main stakeholder groups who could benefit from the innovations related with Big Data

techniques/algorithms deployed in the pilot scenario are presented in their respective figures.

In order to perform the stakeholders’ analysis per case study, the following steps are proposed:

1) Identify transport domain stakeholder groups: the main stakeholders groups for each

of the pilot domains will be analysed. The different features related with the diverse

transportation modes require providing ad-hoc identification in order to connect these

in later project stages (in M18), when the expected impacts among groups are

identified. This activity will allow us to present and communicate to the targeted

stakeholders which are the main results, impacts and benefits that Big Data deployment

would deliver, and how could these innovations influence their daily operations.

2) Identify concrete stakeholders: the groups obtained in the previous stage will be used

to identify concrete enterprises, institutions that could be beneficiaries of the

deployment of the Big data technologies used within TT pilot domains. The efforts of

identifying concrete stakeholders will be undertaken within the region/country of the

pilot domain. In this sense, the aim is to follow the statement of “Act Local, Think

Globally” in order to replicate and cross-fertilise strategies within local actors among the

3 Adapted from Lakshmanan and Anderson (2002), Figure 9, p.21

Page | 35


Version 5.0, 28/07/2017

region/country, but thinking globally in order to set up the conditions to export the

innovation to third countries.

4.1.1 Smart Highways

The main stakeholders involved in the Smart Highways Value Chain are:

Figure 10: Stakeholder mapping in Smart Highways

4.1.2 Connected Vehicle The main stakeholders involved in the Connected Vehicles are:

Highways operators

Managing authority of the highway

Policy Makers

National governments

PPP entity (contractor)

End-users (drivers)

Customers of the highway

Service providers of

Toll areas

Equipment manufacturers

IT providers

Service stations

Research

Research centres related to traffic and

roads.

Page | 36


Version 5.0, 28/07/2017

4.1.3 Proactive Rail Infrastructure

The main stakeholders involved in the Proactive Rail Infrastructure are:

End-users / data providers

Car Manufacturers

/ OEMS

Fleet Industry

Vehicle users

IT Infrasctructure providers

IT Service providers

Mobile network

operators

Vehicle service providers

Meteorological organisations

Automotive suppliers

Vehicle repair shops

Navigation Systems

Car Insurances

Financial Services

Public Authorities

City councils

Regional and national

goverments

Transport operators

Startups/developers

Developers/adopters

High tech specialized

SMEs

End-users/Customers

Users of the trains network

Cargo handling

Industrial suppliers


Equipment manfacturers

Train manufacturers

Network operators

Operational and asset

management

Public Authorities


goverments

Supervisory authorities

Innovation and Research

Research centres

Page | 37


Version 5.0, 28/07/2017

4.1.4 Ports as Intelligent Logistics Hubs The main stakeholders involved in the Ports as Intelligent Logistics Hub are:

4.1.5 Smart Airport Turnaround The main stakeholders involved in the Smart Airport Turnaround are:

End-users/Customers

Shipping agents

Seller/exporters

Buyer/importers




Bulk cargo

Terminal operators

Operational and asset

management

Port Authority


goverments


Custom departament


Research centres

Page | 38


Version 5.0, 28/07/2017

4.1.6 Integrated Urban Mobility The main stakeholders involved in the Integrated Urban Mobility are:

Passengers

Travellers

Cargo handling

Airlines




Airport management and

operations

Airport infrastructure

Service providers (air carriers, air

traffic control, etc.)

Port Authority


goverments


Custom departament

Regulators


Research centres

Civil society groups

Inhabitants

Travellers

Communities

Infrasctructure providers

Multi-sectorial IT providers

Smart grids





Navigation Systems

Car Insurances

Financial Services

Public Authorities

City councils

Local, regional and national goverments

Transport operators

Business ecosystem

Large companies

Developers/adopters


SMEs

Page | 39


Version 5.0, 28/07/2017

4.1.7 Dynamics Supply Networks The main stakeholders involved in the Integrated Urban Mobility are:

4.2 Identification of TT outcomes of interest

The formulation and identification of the impacts at project level are built up according to the

objectives, expected effects and targets allocated in D*1 and D2.2 documents already

submitted. These documents describes the Domain categories and the (Specific and Horizontal)

priority topics which allow us to establish quantitative objectives that may be used to evaluate

the results achieved by each Pilot in particular.

To understand the factors influencing the impacts at project level, it is necessary to

comprehend the linkages and interactions among all impacts in the involved pilot domain

categories. For this purpose, the current project level impacts will be identified, thus supporting

the formulation of the project’s high-level impacts. The result will complement and enrich the

Civil society groups

Inhabitants

Travellers

Communities

Infrasctructure providers

Multi-sectorial IT providers

Smart grids





Navigation Systems

Car Insurances

Financial Services

Public Authorities

City councils

Local, regional and national goverments

Transport operators

Business ecosystem

Large companies

Developers/adopters


SMEs

Page | 40


Version 5.0, 28/07/2017

current KPI process development, and will be used by TT project participants to evaluate which

performance of TT as a whole in the transportation and mobility market may be delivered.

Below, the impacts arisen from each of the WP pilot domain depending on the outcomes

reached by them will be categorised. The outcomes of interest4 of the transport system are a

formulation of the general goals of transport domains in more concrete terms. They describe

the issues that are relevant when observing the impacts of transport systems on society. To

define the outcomes of interest we need to identify the necessary components, which all

together influence the sustainability performance of the transport domain value chain and

which must be monitored in order to assess its status.

The final list of outcomes of interests is shown in Table 6: Group of selected outcomes of interest of

TT project. Source: Adaptation from SUMMA:

Category

Economic

outcomes of

interest

Technical

outcomes of

interest

Environmental

outcomes of

interest

Social outcomes

of interest

Sub-c

ateg

ory

Transport operation cost

+ Productivity /

Efficiency

Big data techniques and

algorithms adoption

+ Data volume

management and processing

Emissions to air +

Energy savings +

Nosie

Safety and security

Table 6: Group of selected outcomes of interest of TT project. Source: Adaptation from SUMMA

Below, we will present which is the explanation of each of the outcomes of interest that TT

project is seeking to reach:

Outcome

category

Economic Definition

4 Source: SUstainable Mobility, policy Measures and Assessment SUMMA. GMA2/2000/32061-S07.14497

Page | 41


Version 5.0, 28/07/2017

Sub

-cat

ego

ry

Transport operation

cost

From industry transport users’ perspective (i.e. manufacturers, retailers, wholesalers and logistics service providers), the expectation is to increase profit while meeting both people and social expectations. To address these expectations, they look forward to increase return on assets and working capital targeting full asset utilization, increase service levels while reducing costs, energy consumption and environmental impacts in their supply chains5. Big Data can help mitigate these problems. Transforming Transport will integrate, adapt and evolve existing Big Data architectures, services and technologies, and will validate these through highly relevant conceptual prototypes. High impact scenarios have been chosen, to demonstrate the large-scale applicability of the proposed solution. As an example, described in the DoW, a 10% efficiency improvement will lead to EU cost savings of 100 B€ (source: ALICE ETP). Thus, in TT, we have identified 3 main sub-categories which TT is expecting to deliver impacts based on the results achieved by pilot domain during the S2 and S3 validation stages. Those ones are;

Transport operation costs: Operating costs are those expenses derived from the acquisition of the resources needed by an organization to accomplish its activities. For instance, cost of goods sold, office rental and labour costs are examples of operating costs. By the integration of big data solutions, TT is looking to improve the operation costs in each of the pilot domain in order to increase the competitive advantage, improves profitability while an optimization of the cost structure is achieved.

Productivity/efficiency: the definition which applies to TT case; it is the ratio of total output to the inputs used in the production process. Inputs can include labour, capital, land, etc. If productivity increases, a business can produce the same output using fewer inputs. The business may then choose to produce more output, lower prices, invest in the business, or return income to shareholders6. For instance, clear examples of this

Productivity / Efficiency

5 A truly integrated transport system for sustainable and efficient logistics: http://www.ectri.org/Documents/2017-

03-13_SETRIS_Truly%20integrated%20Final%20Edition%20(WEB).pdf 6 Bureau of Transportation Statistics:

https://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/publications/transportation_economics_trends_and_measurement_concepts/chapter_5/index.html

http://www.ectri.org/Documents/2017-03-13_SETRIS_Truly%20integrated%20Final%20Edition%20(WEB).pdf

http://www.ectri.org/Documents/2017-03-13_SETRIS_Truly%20integrated%20Final%20Edition%20(WEB).pdf



Page | 42


Version 5.0, 28/07/2017

productivity will be demonstrated through the Asset Management Category, in WP6 case, the asset optimization use will impact in the workers operations allocation, improving the time used for the maintenance activities, as well as, impacting in the travel time reduction, fuel consumption, and CO2 emissions.

However, elements of efficiency that are in line with the definition presented above are included in studies (e.g., World Bank, 2012) and statistical resources of logistics performance at national level. Such elements may include:

Vehicle utilization (utilization of available capacity of vehicles in terms of load factors, empty runs etc.);

Cargo handling efficiency (efficiency of use of cargo handling equipment at storage facilities and terminals / transshipment points);

Infrastructure utilization (efficiency of use of space and infrastructure of storage facilities and terminals / transshipment points);

Frequency of on-time shipments arrival and dispatch (World Bank, 2012).

Capacity utilization (warehouses and other cargo handling facilities, vehicles);

Logistics process efficiency (proportion of time in which the cargo inventory is active/moving in logistics activities, average order cycle time);

Stocking efficiency (inventory turns);

Workforce efficiency (throughput and productivity of workforce).

As may be noticed, there a wide range of indirect impacts (such as Environmental areas) occasioned by the Economic Outcomes. At the time that the project advance in later stages, the deployment of the big data solutions will come up in parallel with a set of indirect benefits arisen from the economic areas of TT pilot domains.

Table 7: Definition of TT Economic outcome

Outcome

category Technical Definition

Sub

-ca

tego

ry


algorithms adoption

In accordance with the DoA, TT is looking to unlock the untapped potential of big data technology: “Doubling the use of Big Data technology in the mobility and logistics sector from the currently 19% to at least 38%”;

Page | 43


Version 5.0, 28/07/2017

TransformingTransport will provide the stakeholders with a set of baseline solutions that evolves the big data technologies in Transport and puts them into a privileged position using high-quality technology. The interest of applying Big Data technologies in the daily operations of the transport market is increasing each year, as well that now it is not largely applied according to the Industry partners of the project, it is expected that the project will achieve the ambition of doubling the use of big data technology in the mobility/logistics industry. In particular, it will be used data sources across all different transport modes and perform vertical and horizontal data integration, thereby contributing to break silos between “sectors” and to cross-fertilise the usage of Big Data technology and applications. Together with the significant improvements in cost structures of existing processes and the opportunity for new services and business models.

Big data techniques and algorithms adoption: each of the pilot have configured in their pilot design deliverable (M3) which is the technical architecture to deploy for the S2-S3 stages, which includes a large-scale experimentation and demonstration, for this purpose, the pilots will have running dedicated large-scale data processing infrastructure for experimental purposes. The adoption of this kind of big data large scale platforms will boost the adoption and replicability of them into the concrete transport domains.

Data management and processing: in this category, the objective is to enhance the current processes for managing large transport data sets collected during the technology deployment. The large-scale demonstrations of the TransformingTransport pilots will work at the scale of data volume, velocity and variety expected by the end of the project (i.e., by the year 2020)

Finally, the main advances in the state of the art beyond the technical outcomes of interest is that, TransformingTransport will stress test European Big Data technologies and establish best practices of employing Big Data at the technical level through the effective deployment of near-market ICT market technologies within large-scale transport pilots. The project will establish transferrable and reproducible best practices for Big Data that can be re-used in other data-intensive.

Data volume management

and processing

Table 8: Definition of TT Technical outcome

Outcome Environmental Definition

Page | 44


Version 5.0, 28/07/2017

category

Sub

-cat

ego

ry

Emission to air

TT is also looking for producing a set of environmental and energy impacts to the transport domain. Those impacts are directly connected with the performance of pilot domains in the categories and KPIs identified for it. According to the Transversal Priority Topics definition in D2.2, the outcomes that will be monitored in this category are the following one; Emission to air:

Reduction of pollutant emissions: In order to assess transport pollutant emissions, Transforming Transport some pilots will estimate the contents of nitrogen oxides (NOx) and particulate matter (PM) generated during their activity.

Reduction of GHG Emissions: Emissions caused by transportation systems are a main contributor of global warming, which threatens to modify many natural processes in unpredictable ways. Greenhouse Gas (GHG) emissions, in particular carbon dioxide (CO2) emissions.

Energy savings:

Which consists in improving the energy efficiency during the operation of the vehicles and therefore, the energy requirements of the fleet

Nosie:

Noise pollution: is harmful for the environment and for humans particularly, as it may originate psychological disturbance (stress, sleep disorders), physical troubles (difficulty to communicate, loss of audial capacity, cardiovascular disorders), social impacts (aggressive behaviour, scholar lags) and economic losses (increase in health costs, lower productivity, accidents, or even damage property value). Transport is responsible in a high percent of this kind of pollution, so the pilot domains will ideally try to reduce the noise of the transport modes used.

Energy savings

Noise

Table 9: Definition of TT Environmental outcome

Outcome

category

Social Definition

Sub

-ca

tego

ry

Safety and

security

Accidentally rates are relevant indicators to evaluate safety levels on transportation systems, since security is a priority matter when developing or modifying transport infrastructures. Fatalities and injuries caused by traffic accidents are one of the most important social costs associated with

Page | 45


Version 5.0, 28/07/2017

transport systems, so in order to retrieve this situation, the analytics of current accidents will bring insights on the main variables that can anticipate and prevent these situations. The definition of situations where a specific message should be addressed to road users it will allow to prevent certain unsafety events. TT will measure which is the performance in the current pilot locations by providing to end-users ad-hoc information of the highways status or unsafety events.

Table 10: Definition of TT Social outcome

Therefore, based on this information, a detailed analysis of which is the relation between the

outcomes of interest of TT project within the impacts resulting from pilot execution, and versus

those detailed in the DoA, may now be accomplished. Below, the relation it’s described

presented as follow:

Page | 46


Version 5.0, 28/07/2017

TT level: Outcomes of Interest Categories

Sub-category

TT pilot level: Transversal

Priority Topic in Pilot domain

TT expected market impacts Pilot

Assessment category

Pilot domain

Economic

Transport operation costs

Reduction of operational costs

Competitiveness of logistics and transport enterprises/institutions/end-users and service providers

Increase of the operating revenues

Increase of profitability

Logistics cost for shipment activities

CAPEX variance

OPEX Variance

Operational Efficiency (OE)

Transversal (ALL)

Reduction of travel time

Better service reliability

Travel time and fuel consumption savings

Decrease of the pollutant emissions to air

Efficiency of vehicle use (idle time, empty runs, route planning efficiency)

Reduction of congestion level

Decrese of the end-to-end transportation time

Reduction of overall turnaround times (airports)

Increase delivery fleet efficiency and punctuality in last-mileage operations (dynamic supply networks)


Transversal (ALL)

Productivity/ Efficiency

Maintenance optimisation

The asset optimization-use will reduce the total maintenance costs

Effective forecasting on demand

Improve maintenance work scheduling

Asset Management

(AM)

Transversal (SH, CV, RI,

PLH)

Page | 47


Version 5.0, 28/07/2017

and planning

Travel time and fuel consumption savings of the maintenance vehicle fleet

Improve occupation rates in freight delivery parking areas

Asset use optimisation

Reduction of the operator expenses allocated to maintenance activities

Integration of predictive maintenance activities

Performance of the HMI (Human Machine Interface) systems to improve the maintenance operations of the assets.

Asset Management

(AM)


PLH)

Technical


algorithm adoption

Transferability

Openness and accuracy of data flows between logistics stakeholders

Integration/interoperability of datasets among different transport modes

Integration between different types of transport (passenger/freight)

Increase of the total number of data sources

Bigger amount of data collected in the transport domains

Enhancing and contributing to the Open data ecosystem in the European transport domain


Transversal (ALL)

Data management

and processing

Environmental and Energy

Emissions to air

Reduction of pollutant emissions

Better air quality

Reduction of pollutant emissions to air Environmental

Quality (EQ)


IU, SN)

Reduction of GHG Emission

Decrease of CO2 emissions

Decrease of NOx emissions

Energy Consumption

(EC)


IU, SN)

Energy savings Energy savings Decrease of the energy consumption in Energy

Consumption Transversal (SH, CV, RI,

Page | 48


Version 5.0, 28/07/2017

vehicle fleets

Improve driver's compliance with green driving

(EC) PLH IU, SN)

Noise Reduction of noise level

Noise reduction Environmental

Quality (EQ)


IU, SN)

Social Safety and

security

Reduction of the % of accidentally rate

Effective bi-directional communication with end-users in order to avoid unsafety traffic events

Enhancing worker safety Safety (SF)

Transversal (SH, CV)

Emergency response time

Reduction of the time required to activate and perform emergency assistance activities

Transversal (SH, CV

Table 11: Expected impacts at TT project level

Page | 49


Version 5.0, 28/07/2017

The objective is to present in a conjunction of tables which are the current impacts identified

within the pilot domains that will contribute to reach the expected impact stablished by TT;

The overall TT objectives established in the DoW, and consist of:

TT Expected impact

Related Outcome of

Interest category

Sub-category Assessment Project KPI

Category

Project level-KPI

codes

“Demonstrating an increase of

productivity in the Transport sector of at least 15%”

Economic

Transport operation costs


EG-PR-1 EG-MS-1

Productivity/efficiency Boosting the business

performance of the operations in pilots

BPO-IO-1

“Doubling the use of Big Data

technology in the mobility and

logistics sector from the currently

19% to at least 38%”

Technical

Big data techniques and algorithm adoption

Boosting the business

performance of the

operations in pilots

Enhance the

transport big data research

and policy ecosystem

BPO-PS-1

RP-RTB-1

Data management and processing

“Leveraging additional target

sector investments of more than 6

times the EC investment”


EG-IL-1

“Involving at least 120 organizations

participating actively in Big

Data demonstrations”

Boosting the business performance of

operations in pilots

BPO-JC-1

Page | 50


Version 5.0, 28/07/2017

5 Conclusions and next steps

The completion of this deliverable implies a deep understanding of whole KPI Framework based on

the development and implementation of the Strategic Assessment Level, coming from the Four

Level of TT Assessment Framework. The outcomes achieved provide the necessary tools and

processes to assess which performance has been reached during project execution from a pilot and

a project dimension. It will enhance the development of cross-fertilisation and replicability actions

for other interested parties along the EU territory. Additionally, this document provides the EC with

an evaluation method that could be re-used for a better understanding of the value of Big Data

integration within the transport and logistic sectors.

The holistic KPI framework is also promoting the adoption and exploitation of Big Data solutions by

showing which performance and positive results has been achieved during the piloting actions. It

will serve for future research and innovation projects as guidelines for measuring the advance of Big

Data in different areas of the categories groups under study.

Furthermore, we have identified the following next steps required to be performed in order to

advance in the assessment activities as well as, for putting the basics for the measurement

evaluation of the performance targets reports to be delivered in M18. The key next steps are:

For Strategic Assessment Level implementation, a consultancy process will be implemented

to assign weights to the various TT impacts. The magnitude of such values will depend on

the inputs and feedback obtained from the internal prospective internal and in case it’s

required we will extend the consultation process arrangement with selected external

stakeholders.

The performance target values at project level based on the results obtained in the

dedicated pilot deliverables referenced as “D*.2 Specific Performance Assessment Plan”,

due in M9. The indicative timeframe when the performance targets at project level should

delivered is between M10-M12.

Development of the KPI reporting document based on “Table 1: Measurement Evaluation

Results template” for collecting the status of performance targets setting by the pilot

domains, as well as, the aggregation of the pilot domains results in order to measure which

has been the performance in the Strategic Assessment level.

Finally, in the Market Impact section, an upgrade and updated including an impact

identification revision within the scope of TT pilot actions will be delivered. On top of that, a

preliminary list of potential benefit observations at a project level will be depicted to raise

the awareness of TT within the transportation and mobility environment.

Page | 51


Version 5.0, 28/07/2017

6. References

Tariq van Rooijen & Nina Nesterova; Don Guikink (2013) CIVITAS WIKI-– Coordination,

evaluation and dissemination of CIVITAS PLUS II.

Silvia De Francisci, Christophe Dromacque, Thomas Mikkelsen Friedrich Georg

Schwarzlaender(2016), D3.1 – Definition of project KPIs, Flexiciency project.

Final report (2011), Assessing the macroeconomic impact of SESAR.

European Institute of Innovation and Technology (EIT) (2015). EIT Monitoring Strategy

Maiju Vuolle, Antii Lönnqvist, Giovanni Schiuma (2014), Developing of Key Performance

Indicators and impact assessment for SHOKs

Page | 52


Version 5.0, 28/07/2017

ANNEX 1- Strategic Assessment KPI level

First of all, it’s worth to mention which is the KPI terminology abbreviation legend used for each

of the KPI table, we have summarised in the following table:

Supporting Economic Growth EG

/Overall transport and logistics productivity PR

/Investments leveraged IL

/Market share MS

Boosting the business performance of operations in pilots

BPO

/Improvement of Transport/Mobility operations IO

/Perform the current product/service portfolio PS

/External Joint-collaborations within Third parties JC

Improved environmental and energy consumption

EE

Pollutant emissions PE

Energy savings ES

Enhance the transport big data research and policy enviroment

RP

Knowledge transfer from research to business

RTB

Page | 53


Version 5.0, 28/07/2017

1. Supporting economic growth

Project KPI Assesment Global Target EG Economic Growth

Topic PR Overall transport and logistics productivity

Project KPI code

Definition

Proposed formula

Unit

Measurement method(Data sources , Frequency and reporting)

Liaison with TT objectives

Time savings: minutes/vehicle-km, minutes to users, minutes/tonnes

Increase productivity: %

EG-MS-1: Aggregated productivity increase in the TT pilot domains

EG-PR-1

In order to assess the productivity within the TT pilot domains, the factor which has been selected

(among multiple productivity transport KPIs), it's is the time savings % of reduction due to the fact

that it's a cross-domain KPI that will be measured inside all the pilot domains as well as based on

the consideration that time savings is a key factor drive to enhance the productivity within their

operational ecosystem. So, the aggregated results will allow us to determinate the total %

aggregated increase of productivity in the project.

Productivity: Σ(P1, P2, P3, P4)

Data sources: the data/figures need to compute the proposed formula will come from the concrete

inputs requested to pilots leaders in tje Transversal KPIs, concretely in Operational Efficiency (OE)

assessment category. Eg; a KPI which monitorize the performance within time savings could be SH-

OE-2.

Frequency of measuring: the first reporting assessment of the aggregated results coming from pilot

domains will be available for M18. The second KPI assessment performance will be availiable at

the end of the project in M30.

Objective O1 "Piloting": Execute effective large-scale piloting and targeted demonstrations

Objective O2 "Value": Operational efficiency of processes, improved customer experience and new business models.

Objective O3: " Reusability": Replicated Big Data solutions in cross-doamins

Objective O4: "Scalability": Ensure that the Big Data solutions developed in TransformingTransport will work at the scale

of the anticipated mobility and logistics processes

Objective O7: "Maket impact": Strengthen both the position of EU Big Data providers as well as sectorial companies to

engage in bids and business opportunities. Fostering an increase of Market Share.

Objective O9: "Mobilisation": Mobilise verifiable commitment to additional sector investments in data assets and big

data technologies

Objective O6: "Transfer": Doubling the use of Big Data solutions in the mobility and logistics sector

Page | 54


Version 5.0, 28/07/2017

Project KPI Assesment Global TargetEG Economic Growth

Topic PR Overal transport and logistics productivity

Project KPI code

Definition

Proposed formula

Unit

Measurement method(Data sources , Frequency and reporting,)


Data sources: the data/figures need to compute the proposed formula will come from the concrete inputs requested

to pilots leaders in "Table 1: Measure Evaluation Results Template". Each of the WP leaders should report which

Market Share % of increase has obtained.

Frequency of measuring: the first reporting assessment of the aggregated results coming from pilot domains will be

available for M18. The second KPI assessment performance will be availiable at the end of the project in M30.

Company and sales revenue: €

Market share: %

EG-PR-1

EG-PR-1: Total porcentage of market share obtained by TT Industry Members

Market share refers to a company's portion of sales within the entire market in which it operates. This metric

indicates a company's size within its market.

Market Share = (Particular Company's Sales Revenue in Time Period X) / (Relevant Market's Total Sales Revenue in

Time Period X)




Objective O4: "Scalability": Ensure that the Big Data solutions developed in TransformingTransport will work at the scale of the

anticipated mobility and logistics processes

Objective O7: "Maket impact": Strengthen both the position of EU Big Data providers as well as sectorial companies to engage in bids and

business opportunities. Fostering an increase of Market Share.

Objective O9: "Mobilisation": Mobilise verifiable commitment to additional sector investments in data assets and big data

technologies


Page | 55


Version 5.0, 28/07/2017

Page | 56


Version 5.0, 28/07/2017

2. Boosting the business performance of operations in pilots

Project KPI Assesment Global Target BPO Business perfromance operations

Topic IO Improvement of Transport/Mobility operations

Project KPI code

Definition

Proposed formula

Unit

Measurement method (Data sources , Frequency and reporting,)


BPO-IO-1: Total accumulated operations costs of existing proceses and services reduced by TT pilot deployment

BPO-IO-1

The aggreated % of operation costs reduced by TT will be expressed in conjunction within this KPI. The aim is to

understand how much big data could impact within the operational costs structure of the pilot domains from the

different transport modes inside TT.

Operational costs reduction: Σ( OP1, OP2, OP3, OP4, etc.)

€/(tonne x year)

€/vehicle-km ; €/km

€/(passenger x year)

Data sources: the data/figures need to assess the performance on this KPI has to be fed by the concrete indicators

that will measure the % of reduction inside the pilot domain. In this case, Operational Efficiency (OE), is the

category defined for TT to assess the performance among pilots (eg; SH-OE-1).

Frequency of measuring: the first reporting assessment of the aggregated results coming from pilot domains will

be available for M18. The second KPI assessment performance will be availiable at the end of the project in M30.






Objective O7: "Maket impact": Strengthen both the position of EU Big Data providers as well as sectorial companies to engage in

bids and business opportunities. Fostering an increase of Market Share.


technologies


Page | 57


Version 5.0, 28/07/2017


Topic PS Perform the current product/service portfolio

Project KPI code

Definition

Proposed formula

Unit



BPO-PS-1: Total number of new products/services/processes with big data features launched into the market

BPO-PS-1

The pilot design D*.1 has reveal which technical architecture as well as the concrete Business Cases will be under

the deploy and assessment within TT pilot domains. The technical and business findings achieved up to M6 has

revealed that in most of the pilot domains will come up with the development of new product/services/process in

later project stages ath the time that the results are consolidating within the pilot domain. The design and

implementation of the technical architecture is currently a big advance in the business art of different pilot

leaders. So, we need to asses which is the potential of the big data to produce the kind of services and products

expected.

Business performance: Σ(New Business Models X,New Products X, New Services X, New Processes X, etc.)

Total number (asolute units)

Data sources: the data/figures need to assess the performance on this KPI has to be fed by two different channels.

One of them, it will directly taken from the exisitng (as well as future) exploitation assests definition, and the

other one will be directly fed by the inputs from pilot leaders in case we haven't identify it within TT exploitable

assets. This two-side analysis will serve to higlight and check the main results obtained in the business

performance products/services that include big data features are in line with the exploitation assets identification.











technologies


Page | 58


Version 5.0, 28/07/2017



Project KPI code

Definition

Proposed formula

Unit



BPO-PS-2: Total number of existing improved products/services/processes with big data features launched into the market

BPO-PS-1

The pilot design D*.1 has reveal which technical architecture as well as the concrete Business Cases will be under

the deploy and assessment within TT pilot domains. The technical and business findings achieved up to M6 has

revealed that in most of the pilot domains will come up with the performance of the existing

product/services/process in later project stages ath the time that the results are consolidating within the pilot

domain. The design and implementation of already technical big data architectures and components will enhance

the development of the product/Service owner by adding big data features.

Business performance: Σ(Business Models X,Products X, Services X, Processes X, etc.)

Total number (asolute units)

Data sources: the data/figures need to assess the performance on this KPI has to be fed by two different channels.

One of them, it will directly taken from the current and exisitng exploitation assests definition, and the other one

will be directly fed by the inputs from pilot leaders in case we haven't identify it within TT exploitable assets. This

two-side analysis will serve to higlight and check the main results obtained in the business performance

products/services that include big data features are in line with the exploitation assets identification.











technologies


Page | 59


Version 5.0, 28/07/2017



Project KPI code

Definition

Proposed formula

Unit



BPO-JC-1: Total number of Involved organizations which are participating actively in Big Data demonstrations

BPO-PS-1

In order to asses which it's the performance of TT of the multiplier effect on project successful results by

implementing a two-sided innovation and incubation strategy called ‘from local-to-project-to-global’, we have set

up a KPI that reflects the total accumulated number of third party organizations that will joint TT to contribute to

expand and exploit the results obtained.

External joint collaborations: Σ(Third parties)

Total number (absolute units)

Data sources: the data/figures need to assess the performance on this KPI has to be fed by the inputs coming from

T3.6 – Post-project replication, Incubation & Innovation management











technologies


Page | 60


Version 5.0, 28/07/2017

3. Improved environmental and energy consumption

Project KPI Assesment Global Target EE Energy efficiency

Topic PE Pollutant emissions

Project KPI code

Definition

Proposed formula

Unit



EE-PE-1: Total direct CO2 emission reduction obtained

EE-PE-1

The GHG emissions caused by the transportation systems are the main contributor to global warning, in particular

TT will assess which carbon dioxide (CO2) emissions reduction will be evaluated as an indicator of energy

consumption, as the emission of this substance is proportional to the vehicle’s energy consumption.

% CO2 emissions reduction: Σ(PE1, PE2, PE3, etc.)

CO2: g CO2/km


pilot domains, concretely to the Transversal KPIs included in the Energy Consumption (EC) Assessment Category.

From example, CV-EC1











technologies


Page | 61


Version 5.0, 28/07/2017

Project KPI Assesment Global Target EE Energy efficiency

Topic PE Energy Savings

Project KPI code

Definition

Proposed formula

Unit



EE-ES-1: Total direct energy savings obtained

EE-ES-1

This KPI assess the energy efficiency improvement during the operation of the vehicles and therefore, the energy

requirements of the fleet. One important aspect is the way to demonstrate that Big data solutions have several

direct and indirect impacts, for instance, a key challengue for EC is to decrease the energy demand for Public

Transport operators as well as from the end-user perspective. The indirect impact is directly related with economic

savings produced by the reduction of the energy consumption. So, the objective of TT is to compute which final %

reduction has been achieved within TT pilot domains.

% Energy Savings: Σ(ES1, ES2, ES3, etc.)

Fuel consumption: kWh/100 km


pilot domains, concretely to the Transversal KPIs included in the Energy Consumption (EC) Assessment Category.

For example, we can extract an pilot KPI related with this topic which is IU-EC-2 "Vehicle energy consumption per

100 km. "











technologies


Page | 62


Version 5.0, 28/07/2017

4. Enhance the transport big data research and policy ecosystem

Project KPI Assesment Global Target RP Enhance the transport big data research and policy ecosystem

Topic RTB Knowledge transfer from research to business

Project KPI code

Definition

Proposed formula

Unit



RP-RTE-1: Total number of big data components integrated into pilot domain platforms

RP-RTB-1

TT is looking to boost the adoption of big data products and services within the pilot domains, so for this purpose,

we have identified a KPI that will measure which is the performance degree for integration big data features

within their current technical architectures. We will measure the total number of big data components integrated

that will enhance the current position of the pilot in the big data transport ecoystem.

Big data components: Σ(BC1, BC2, BC3, etc.)

Total components (absolute numbers)

Data sources: the data/figures need to compute the proposed formula will come from the concrete inputs

requested to pilots leaders in "Table 1: Measure Evaluation Results Template". Each of the WP leaders should

report which Market Share % of increase has obtained.











technologies


KPI Assessment Framework - Transforming...

Documents

Transcript of KPI Assessment Framework - Transforming...