Viajeo Validation Plan D5 1 Final Version · E-mail: [email protected] Abstract The deliverable...

D5.1 Validation plan

28/02/2011 of 63 Version 2.0

VVIIAAJJEEOO


Author(s) Philipp Böhnke (DLR), Dr. Matthias Hetscher (DLR), Annie Kortsari

(CERTH), Mai Sloth (KeyResearch)

Project VIAJEO – International Demonstrations of Platform for Transport

Planning and Travel Information

Date Contractual: 30 Aug 2010 Actual: 01 Nov 2010

Project Coordinator

Yanying Li ERTICO – ITS Europe Tel: +32 2 400 07 37

E-mail: [email protected]

Abstract The deliverable D5.1 defines indicators and procedures for the validation

and the impact assessment of the Viajeo project.

Keyword list Validation, Impact assessment, Indicators, Demo sites

Nature of deliverable

Report

Dissemination Public1

Project financially supported by

European Commission DG Research

Project number 233745 FP7- SST.2008.3.1.6

1 This is either: Public, restricted to other programme participants, restricted to a group specified by the

consortium, confidential


28/02/2011 of 63 Version 2.0

Control sheet

Version history

Version number Date Main author Summary of changes

1.0 14.06.2010 Philipp Böhnke (DLR) Draft Version

2.0 25.08.2010 Philipp Böhnke (DLR) Final draft Version

3.0 15.10.2010 Philipp Böhnke (DLR) Final Version

Approval

Name Date

Prepared Philipp Böhnke (DLR) 15 Oct 2010

Reviewed All demo site leaders 01 Nov 2010

Authorized Yanying Li (ERTICO) 01 Nov 2010

Circulation

Recipient Date of submission

European Commission 05 Nov 2010


28/02/2011 of 63 Version 2.0

Table of content

1. Introduction ......................................................................... 5

1.1. About the Viajeo project ............................................................ 5

1.2. Target audience ....................................................................... 5

1.3. Role of this report in Viajeo ........................................................ 6

1.4. Structure of the document .......................................................... 6

2. Basis and definitions of the validation plan ................................... 7

2.1. Used documents ....................................................................... 7

2.2. MAESTRO Evaluation Guidelines ................................................... 7

2.3. Open platform requirements ....................................................... 7

2.4. Definitions for WP5 ................................................................... 8

2.5. Demo sites and planned extensions ............................................... 8

3. Approach of the overall validation plan ...................................... 11

3.1. Validation (technical) .............................................................. 11

3.2. Impact assessment .................................................................. 11

3.3. Perspectives for European ITS technologies in China and Brazil ......... 11

3.4. Relation between components and testing phases........................... 12

4. Validation ........................................................................... 13

4.1. Objectives – (technical) validation .............................................. 13

4.2. Approach and methodology – (technical) validation ......................... 13

4.3. Validation - Testing phase I (Laboratory test) ............................... 14

4.4. Validation – Testing Phase II (Field test) ...................................... 17

4.4.1. Open platform - requirement ......................................................... 17

4.4.2. Validation of the new developed/expanded and implemented modules ... 19

4.4.3. Validation of selected and implemented European standards (adaptability and practicability) .................................................................................... 44

5. Impact assessment ................................................................ 45

5.1. Objectives ............................................................................ 45

5.2. Target groups ........................................................................ 45

5.3. Approach and methodology – impact assessment ............................ 45

5.3.1. Impact assessment indicators – Athens ............................................. 47

5.3.2. Impact assessment indicators - Sao Paulo ......................................... 53

5.3.3. Impact assessment indicators – Beijing ............................................. 55

5.3.4. Impact assessment indicators – Shanghai .......................................... 59

6. Perspectives for European ITS technologies in China and Brazil ......... 61

6.1. Objectives ............................................................................ 61


28/02/2011 of 63 Version 2.0

6.2. Approach and Methodology ....................................................... 61

7. Data (collection, analysis, validation, tables and reports) ................ 62

7.1. Data - collection .................................................................... 62

7.2. Data - analysis ....................................................................... 62

7.3. Data - validation .................................................................... 62

7.4. Data – charts of responsibility ................................................... 62

7.5. Reporting system for WP5 ........................................................ 63


28/02/2011 of 63 Version 2.0

1. Introduction

1.1. About the Viajeo project

The Viajeo project designs, demonstrates and validates an open platform which facilitates data sharing and exchange from different sources and also provides data processing and management to support a variety of services. The project integrates the open platform with local components and demonstrates its applications in four cities: Athens, Beijing, Shanghai and São Paulo. Viajeo (September 2009 – August 2012) is co- funded by the EC DG Research for Specific International Cooperation Actions (SIGA). Cities face ever increasing demands on their transportation systems, especially in developing regions with growing car ownership and rapid urban migration. Even more than heavy infrastructure investment, strategic mobility management is becoming the most important tool for meeting this demand. Mobility management is the domain of activities that combine long term planning and implementation activities with real time and historical information. To be able to deliver effective transport planning and implement the plans, huge amounts of information about usage of the transportation network are needed. Currently, traditional methods (such as roadside units to collect traffic data) and new methods (probe vehicles) are being used to provide traffic data. However, integration of such data is essential in order to support innovative long-term planning and short-term proactive and reactive management of the transport network. The open platform will benefit travelers, transport planners and transport system operators and managers. The open platform will be able to support cross-modal journey planning and seamless traveler information service. Data exchange between operators will lead to a more optimized operation of all systems and individual operation strategies can be harmonized by the data exchanges and information sharing. The platform will enable all data and information used for real-time operation to be used for transport planning. Transport planners will have access to traffic data and information on traffic management strategies. Such information will lead more detailed and comprehensive historical database for transport modeling development and long term policy evaluation The objectives of the Viajeo project are to:

• Design of an open platform with interfaces to a wide range of mobility services.

• Implement of the open platform in Europe, and in the emerging economies, i.e. China and Brazil.

• Validation of the open platform.

• Assess of social and transport impacts of the implementation and demonstration of the open platform.

1.2. Target audience

Target audiences of the deliverable D5.1 “Validation plan” are mainly all partners of the Viajeo project as well as all involved participants/working parties in the demo sites in Athens, Beijing, Shanghai and Sao Paulo. Beside this the deliverable D5.1 is also part of the official documentation and reporting of the Viajeo project to the European Commission. Deliverable D5.1 “Validation plan” is primarily part of the working level. Final results will be given in the deliverables D5.2 “Assessment and validation results” and D5.3 “Best practice guidelines for business models” which will be based on D5.1 “Validation plan” during the further progress of the Viajeo project.


28/02/2011 of 63 Version 2.0

1.3. Role of this report in Viajeo

The report D5.1 provides the validation and impact assessment plan of the Viajeo project. The focus of the report D5.1 is the Viajeo objectives “Validation of the open platform” and “Assessment of social and transport impacts of the implementation and demonstration of the open platform”. The structure of the overall validation plan, especially e.g. chosen indicators, questions and requirements will help all project partners to focus their research activities within the project. This is of particular importance because of the geographical positions of the demo sites within Europe, China and South America. In each demo site the Viajeo project faces new boundary conditions (e.g. social, cultural and technical). It is important to show that the developed open platform system is highly adaptable but it is also necessary to set up a universally valid validation plan which is strict enough to make the results of the different demo sites comparable but which also allows to go into the specifics. The validation plan is cross-linked with the work packages 2, 3, 4, 6 and 7 of the Viajeo project to ensure that the overall project objectives given in the description of work (DoW) will be reached at the end of the project. At the end the validation plan is an instrument which will help to evaluate / validate the project results of the Viajeo project.

1.4. Structure of the document

Following a brief overview about the structure of the document is given. The report starts with a short summary of the used documents, guidelines, definitions and assumptions in chapter 2. Chapter 3 explains the approach and methodology of the overall validation plan. Chapters 4, 5 and 6 deal each with one of the main parts of the overall validation parts (Validation, Impact assessment, Perspectives for European ITS technologies in China and Brazil) whereas chapter 7 gives short information about data collection, data validation and data analysis.


28/02/2011 of 63 Version 2.0

2. Basis and definitions of the validation plan

To guarantee a good quality of results in the further progress of the Viajeo project and to avoid misunderstandings based on different wordings the present chapter gives a brief overview of used documents, the MAESTRO Evaluation Guidelines and definitions as well as a short description of the planned services within the demo sites (inside the Viajeo project).

2.1. Used documents

Beside the information (which was) generated specially for deliverable D5.1” by the WP5 partners five important documents form mainly the basis for the validation plan:

• Description of Work (DoW)

• D2.1 Investigation report

• D6.1.1 Investigation final (Athens)

• D6.2.1 Investigation draft (Sao Paulo)

• D6.3.1 Investigation final (Beijing)

• D6.4.1 Investigation final (Shanghai)

2.2. MAESTRO Evaluation Guidelines

MAESTRO (Monitoring Assessment and Evaluation of Transport Policy Options in Europe) was a project funded by DG TREN, the Transport and Energy Directorate of the European Commission. The MAESTRO project brought together a team of leading evaluation experts from 11 EU member states. The purpose of the project was to develop a common framework and methodology for the selection, design and evaluation of pilot and demonstration projects within the specific research program on transport. Although these Guidelines focus on EU-level research projects, they may be applied to P/D projects at all levels, even down to the most localized level. The MAESTRO Guidelines follow a comprehensive approach which means that they cover all specific stages and processes within a P/D project. In the field of evaluation they provide – beside guidance about how to define new / specific impacts and indicators, including necessary specifications / details of an indicator – a list of potential project impacts and associated indicators.

All indicators of the following Viajeo validation plan are defined according to the MAESTRO Evaluation Guidelines. This ensures a comprehensive definition of each indicator, a broad scale of important impacts as well as a general compatibility with the requirements of P/D projects funded by the EU. All aspects are very important to archive a found basis for the following phases of the validation process and also to archive best results for the whole Viajeo project.

2.3. Open platform requirements

Open platform” is neither an official standard nor does a universal valid definition exist. Therefore the partners of the Viajeo project worked out an own definition. This definition forms the basis of the overall validation plan as well as of the following validation and impact assessment:

• A well defined set of core processes accessible via published open external interfaces which are accessible by external parties.

• Standardized, openly described interfaces for data exchange between processes in the realm of the open platform. This set of interfaces is to be used by any new functionality implemented within the open platform concept.

• An “Open Platform” does not mean it is “Open Source”.


28/02/2011 of 63 Version 2.0

• Related to Open Standards, an open platform means the platform is publicly available and has various rights to use associated with it.

2.4. Definitions for WP5

Overall validation plan: In the following document “D5.1 Validation plan” is also called “overall validation plan”. It consists of different components. One component is called “Validation” and is just a part of the overall validation plan. The other two components are called “Impact assessment” and “Perspectives for European ITS technologies in China and Brazil”.

Validation: Validation is defined as the technical evaluation of the system which will be developed within the Viajeo project. Validation deals with e.g. the system architecture, specifications, open platform requirements, interfaces and all kind of technical parts.

Impact assessment: Impact assessment is defined as the evaluation of impacts to users (end users, transport planners and operators), stakeholder and decision makers. For the impact assessment an already implemented systems is obligatory.

2.5. Demo sites and planned extensions

Athens

Athens (Municipality of Athens), which is the Greek capital, has a population of 745,514 inhabitants (according to the census of 2001) within its administrative limits and a land area of 39 km2. The urban area of Athens, however, extends beyond the administrative city limits with a population of 3,130,841 inhabitants (2001 census) and a land area of 412 km2. According to Eurostat, the Athens Larger Urban Zone (LUZ) is the 7th most populous LUZ in the European Union (the 5th most populous capital city of the EU) with a population of 4,013,368 (in 2004). Athens is the central to economic, financial, industrial, political and cultural life in Greece and is rapidly becoming a leading business centre in the European Union. Planned extensions within the Viajeo project:

The three services that are planned to be implemented for Athens test site are the following:

• Taxi fleet management and traffic information

• End user multi-modal trip planning and traffic information

• Observatory for Public Authorities and Traffic Planners

All three services comprise of both existing and new modules and interfaces. Most of the existing elements are to be developed further in the project.

Beijing

Beijing, as the capital city and a municipality of the People's Republic of China (PRC), is a very large city, quickly becoming one of the main business centers in the world. Its statistics are quite impressive too: a 735 km2 urban area, more than 16 million permanent resident people, 3.9 million vehicles, more than 20.000 public transport buses, and more than 200 kilometers of underground line. Traffic in the city centre is often gridlocked, with rush hour lasting 11 hours a day as of recent years, and smooth traffic only available at night. The target area of the demonstration project covers the northwest corner of the area between the 3rd and 4th ring roads, thus comprising both bus and metro lines. The approximate size of the area is 140 square kilometers. Planned extensions within the Viajeo project:

The Beijing test site is going to include services in the following fields:


28/02/2011 of 63 Version 2.0

• Within the Viajeo project a new source of floating car data will be introduced (by T-systems) and thus improving the available information for traffic management and planning. Besides the basic GPS information (position, speed, course), data on fuel consumption will also be registered.

• The existing public transport websites are to be extended with a co-modal journey planner. Improved bus timetables and information at bus stops are also part of the Viajeo project.

Shanghai

Shanghai is the largest city in China, and one of the largest metropolitan areas in the world. Located on China's central eastern coast just at the mouth of the Yangtze River, the city has a population of 19,213,200 inhabitants within its administrative limits and a land area of 6,340 km2. However, the urban area of Shanghai extends to 5,299 km2. After 1990, the economic reforms introduced by Deng Xiaoping resulted in intense re-development and financing in Shanghai, and in 2005 Shanghai became the world's largest cargo port. Shanghai will hold the World Expo 2010, the largest event in China since the 2008 Olympics. Planned extension within the Viajeo project:

The Shanghai test site is going to include services in the following fields:

• Within the Viajeo project, a new source of environmental data will be introduced and thus to give a real time vision on air quality over the test site. These data with traffic data will be contributed to calibrate and validate a traffic related environmental model which can simulate environmental impacts of road traffic. The traffic environmental model will be used for the appraisal of transport policy and planning. During the demonstration, a number of environmental sensors with GPS and GPRS units will be installed on buses and other cars depending on the actual situation to measure air pollution along the bus route.

• Besides this, the Shanghai demonstration will improve the bus information at bus stops such as forecasting bus arrival times. Existing interfaces will be also improved in accordance with open standards. The bus arrival times are also going to be used to form dynamic timetables which will improve bus operation.

São Paulo

São Paulo is the largest city in Brazil and is the world’s 7th largest metropolitan area. With a population of 11,105,249 residents within an area of 1,523 km2, the city is the largest city in the Southern hemisphere in terms of population. The Greater São Paulo, on the other hand, consists of 39 municipalities, including the capital, São Paulo, with around 27,640,577 population spread over 23,062 km2.

Greater São Paulo has over 6.2 million cars, 4 million of them in São Paulo city. São Paulo is a heavily congested city with a record of 270 km traffic queue in 2010. Demand management has been applied, i.e. 20% of cars are not allowed to drive in the extended centre area between 7am and 10am and between 5pm and 8pm on weekdays. São Paulo has a comprehensive public transport service consisting of metro, train and buses. However, 45% of daily journeys use private cars. Traffic Engineering Company (CET) manages and maintains the traffic system in São Paulo city. The network consists of 5500 traffic lights of which 1000 are connected through ATCs in 8 zones. As expected, CET has been having difficulties controlling the network due to the equipment’s age (around 20 years old) and lack of a more thorough maintenance.

The Sao Paulo test site is going to include services in the following fields:


28/02/2011 of 63 Version 2.0

• Proof-of-Concept of integration of probe vehicles and urban traffic control centre;

• Integrating urban traffic control centre data with motorway traffic data; The open platform will connect data from urban traffic control centre and data from motorway tolling system in order to estimate real-time traffic condition in urban and motorway networks.

• Use of urban traffic control centre for real-time traffic information. The real-time data will be stored into a historical database. The real-time information and historical database will be used to support a web-based journey planner.

• Real-time traffic information service delivered to on-board units.


28/02/2011 of 63 Version 2.0

3. Approach of the overall validation plan

To fulfill the multilayer objectives of WP5 given in the description of work the overall validation plan is subdivided into three different components:

1. Validation

2. Impact assessment

3. Perspectives for European ITS technologies in China and Brazil

Each component can be assessed more or less individually but all components are needed to evaluate the complex structure of the whole Viajeo project. Below a short overview of each of the different components is given. A detailed description of the proceeding is given in the corresponding chapters.

3.1. Validation (technical)

Following the definition within the Viajeo project, “Validation” is defined as the technical evaluation of the system which will be developed within the Viajeo project. Validation deals with, among other things, the system architecture, specifications, open platform requirements, interfaces and other technical parts.

Technical systems, especially complex software systems, need to be tested during their development to guarantee that the whole system is running. Normally, like in Viajeo, they consist of various single modules and interfaces. Each module and interface as well as the interaction between them has to be tested. Within the Viajeo there are two defined testing phases:

1. Testing phase I (laboratory test) In the laboratory tests all new designed parts of the system will be tested as well as the whole core system. All laboratory tests are related to the non implemented system.

2. Testing phase II (field test) In the field tests the planned and already implemented system will be tested at each demo site. The field tests are integration tests which will help to confirm whether data sources and services are integrated into the developed open platform.

3.2. Impact assessment

Following the definition within the Viajeo project, “Impact assessment” is defined as the evaluation of impacts to users (professionals as transport and traffic planners as well as transport operators, non-professionals, stakeholders and decision makers). In order to perform the impact assessment, an already implemented systems is obligatory. This leads to the fact that the impact assessment can also be assigned to the testing phase II (field test).

3.3. Perspectives for European ITS technologies in China and Brazil

This component is little detached from the other components. It assesses the potential for further expansion of the demonstrated technologies to other parts of China and Brazil and looks at the potential in general for the use of European ITS technologies in the two countries.

An existing open platform system is not at all mandatory although the experiences and input of the Viajeo project are valuable and necessary for the preparation of the best practice guidelines.

This component cannot be assigned to one of the testing phases.


28/02/2011 of 63 Version 2.0

3.4. Relation between components and testing phases

Figure 1 illustrates the relation between components and testing phases. Each component of the overall validation plan is clearly represented. The three separate columns are partly overlapped by two rows which represent the different testing phases. The overlapping area of the columns and rows show that there is a relation between the components and the testing phases. As testing phase I deals only with the non-implemented systems (laboratory test), while impact assessment needs an already implemented system, this overlapping area is crossed out. The three overlapping areas, which build the main parts of the overall validation plan, as well as the last column “Perspectives for the European ITS technologies in China and Brazil” will be described in detail in the following chapters.

Perspectives forEuropean ITS

technologies in China and Brazil

Impact assessment

Labora

tory

te

st

Testi

ng

ph

ase

I

Fie

ldte

st

Tes

tin

gp

has

eII

Validation(technical)

xModule test

Integration test

System test

Demonstrator tests

• Open platform

• Processing chain

• Multi-/Intermodal

routing

Impact assessmentplan

• Athens

• Beijing

• Shanghai

• Sao Paulo



Impact assessment

Labora

tory

te

st

Testi

ng

ph

ase

I

Fie

ldte

st

Tes

tin

gp

has

eII


xModule test

Integration test

System test

Demonstrator tests

• Open platform



routing


• Athens

• Beijing

• Shanghai

• Sao Paulo



Impact assessment

Labora

tory

te

st

Testi

ng

ph

ase

I

Fie

ldte

st

Tes

tin

gp

has

eII


xModule test

Integration test

System test

Demonstrator tests

• Open platform



routing


• Athens

• Beijing

• Shanghai

• Sao Paulo

Figure 1: Relation between components and testing phases


28/02/2011 of 63 Version 2.0

4. Validation

Following the technical validation, which comprise of a laboratory and a field test phases, is described. Each phase can be assessed more or less individually but both are needed for the validation of the whole project.

4.1. Objectives – (technical) validation

The goal of the validation is to evaluate if the developed open platform works correctly and if it conforms to the technical requirements of the description of work. The technical validation of the whole system will be done through the validation of the modules and interfaces integrated into the open platform system combined with a validation of the implemented service within each demo site.

4.2. Approach and methodology – (technical) validation

A brief overview of the chosen tests and testing proceedings for the open platform system is given below. The chosen tests consist of four stages:

I. Module test

II. Integration test

III. System test

IV. Test during the final demonstration in each demo site (Acceptance test)

I. Module test Goal of this test is to test the software functionality at the level of the single module itself. It provides the developer with information about the validity of algorithms, functions and routines. This test is planned and done by the module developer himself. To specify the test cases the module interface specifications are used. II. Integration test Goal of this test is to test the interaction between several single modules. These modules are put together/composed to create subsystems to test a bigger part of the whole software system. It ensures the validity of the I/O routines, interfaces and the consistency of the software body. This test is also planned and carried out by the software developer in all versions. III. System test Goal of this test is to compare the whole developed software system with the technical specification (DoW). This test is planned between the different parties providing system modules. It is focused on the implementation of the proposed functionalities and basic features. This test pays attention on:

• Performance

• Reliability

• Handling

• Functionality

• Documentation

and provide both parties with measurable indicators. IV. Test during the final demonstration (acceptance tests) Goal of this final test is to demonstrate that the delivered system fulfils the originally and currently defined requirements. It is performed at the demo sites after the integration of the developed open platform system into the existing infrastructure. This test is planned by


28/02/2011 of 63 Version 2.0

WP5 and the developer and will be executed and evaluated by the software developer and the demo site operators. Within the Viajeo project the module, integration and system tests are dedicated to the “Validation – Testing Phase I (Laboratory test)” while the tests during the final demonstrations (acceptance tests) are dedicated to the “Validation – Testing Phase II (Field test)”. The module and integration tests will be done only by the software developer which means that they can chose a testing procedure. Test results and reports will be handed over to WP5. The system and field tests (acceptance test) are developed according to the Viajeo requirements.

4.3. Validation - Testing phase I (Laboratory test)

The module and integration tests – in contrast to the system test – will be developed and done only by the software developers themselves. As already mention there will be no specifications for these tests defined by WP5. This approach makes sense because number, complexity and indicators of those tests depend highly on the realization of the final software platform as well as on the newly developed modules. The necessary degree of knowledge and information about the modules and the essential tests will (only) be combined by the developers at the end of the software work. Results and reports of these tests will be transferred by the developers to the WP5 leader who will take them into the WP5 reports.

However, this approach does not mean that there will be no independent validation of the modules and/or interaction at all (because they are developed and tested only by the developers). Within the system test the functionality of the whole software platform will be tested which means that therefore also the single modules as well as the interactions between them are indirectly tested.

Goal of the system test is to compare the developed software modules with the technical requirements and specifications defined in Viajeo. These tests will be done in the laboratories of the developing companies (Mizar/Ptv) corresponding to the specifications defined in the following. All results and reports will be transferred to the WP5 leader who will take them into the WP5 report. Main focuses of these tests are:

• Performance

• Reliability

• Handling

• Functionality

• Documentation quality as acceptance criterion

The whole system including the peripheral components will be tested in the final test for the Viajeo project which is the acceptance test during the demonstrations within the demo sites (field test). Following evaluation methods and indicators for the system test (laboratory test) of the core system are defined, related to the MAESTRO evaluation guidelines of the EU:

Test Indicator Description

Data attributes

Sources of data Unit

Type of data

acquisition

Method of data

acquisition

Performance test

Time response

Mean time between raw data input and data output

Seconds

Quantitative Measurement Special reference data set / test data


28/02/2011 of 63 Version 2.0

set

Actuality Mean actuality of output data – time difference between oldest used data set and output data

Seconds Quantitative Measurement Time stamp of used data sets and output data

Correctness Percentage of accuracy of data output (data output structure)

% Quantitative Measurement / Survey

Special reference data set / test data set

Reliability Percentage of error-free run-time

% Quantitative Measurement /Survey


Goal: Validation of the software platform performance. Does the measured performance fit the requirements (dynamic online services possible, real-time capabilities)


Data attributes Sources of

data

Unit Type of data acquisition

Method of data

acquisition

Reliability test

Validation time (Maturity)

Mean time of data validation (Maturity of results)

Seconds

Quantitative Measurement Time stamp of used data sets and output data

Fault tolerance

Percentage of temporal data gaps (no input at all) with which the system still is running

% Quantitative Survey / Measurement

Special reference data set / test data set with temporal data gaps

robustness Percentage of incomplete input data (incomplete data set / e.g. data of one sensor is missing) with which the system still is running


Special reference data set / incomplete test data set

Recoverability

Mean time for a system recovery after a system crash

Seconds Quantitative Measurement

Conformity Percentage of erroneous data (obvious wrong/impossible data) with


Special reference data set / erroneous test data


28/02/2011 of 63 Version 2.0

which the system still is running

set

Goal: Verification of the enhancement of run time stability


Data attributes


Type of data

acquisition

Method of data

acquisition

Handling test

Comprehensibility Percentage of people who understand the system

%

Qualitative Survey System itself

Operability Percentage of people who are able to operate the system

%


Conformity Conformity of the two different systems (Ptv/Mizar)

% Qualitative Survey Systems itself

Goal: Demonstration of an easy handling


Data attributes

Sources of data

Unit

Type of data

acquisition

Method of data

acquisition

Functionality test

Suitability / correctness

Percentage of suitable and correct output data

%

Quantitative

Measurement


Security Rating given to security of system (How easy is a manipulation of the system)

% Qualitative Survey System itself

Interoperability

Maximum/minimum no. of different input data sources and services with which the system (traffic engine) is able to run

No. Quantitative

Measurement /Survey

System itself / input data sources / services

Goal: Demonstration of output correctness and operation security of the software

Documentation Indicator Description

Data attributes


Type of data

acquisition

Method of data

acquisition

Documentation quality as acceptance criterion

Documentation Percentage to which all important methods and

%

Qualitative Survey Documentation


28/02/2011 of 63 Version 2.0

software components of the system (following the before defined open platform standard) are sufficient described

Goal: To guarantee that the defined open platform standard is kept

4.4. Validation – Testing Phase II (Field test)

The tests during the final demonstration (acceptance tests) belong to the “Validation – Testing Phase II (Field test)”. The goal of these tests is to evaluate / validate if the implemented demonstrator systems in the test sites are running and if they fit the technical requirements of the Viajeo project. Therefore a (tripartite) validation and testing procedure has been developed by WP5 (based on the description of work and the defined services respectively the use cases defined for the demo sites) which focus on the following requirements:

I. Validation of the “open platform” requirement

II. Validation of the new developed/expanded and implemented modules

III. Validation of selected and implemented European standards (adaptability and practicability)

The first requirement “open platform” verifies that it is (easily – only with the published information) possible to integrate new data sources or services into the platform which means that the platform is indeed an open platform. The second requirement is to validate the new developed or expanded and implemented modules in the demo sites. Modules which already exist and modules which will be developed outside the Viajeo project will not be validated within the WP5. Finally the third requirement focuses on the adaptability and practicability of the selected and implemented European standards to existing local components and services within the demo sites in Europe, China and Brazil.

4.4.1. Open platform - requirement

One of the main requirements for the platform used in the Viajeo project is that it should be an open platform. To evaluate if the requirement is fulfilled, an additional fictitious data source (raw data set) will be integrated into the already implemented and running platforms within the sites. This will be done according to the official documentation and information. The generated information of the corresponding service (e.g. Traffic information) will be checked and evaluated.

To keep the additional amount of work within an economically justifiable effort, the additional fictitious data source (dummy data) should be defined according to the interface specifications like an already implemented data source (in favor analog FCD). Therefore no (possibly) complex preprocessing or adjustments of the additional data set will be necessary. The data/content within the additional data set will be created / manipulated in a certain way so that the results - after processing and integrating it into the platform – change in an expected manner. As a result of the manipulation it will be possible to verify if the additional data source is fully integrated into the platform and if the functionalities operate according to the specification. The kind of additional data set as well as the testing area (even if our partners will try hard to realize large-scale data coverage within the whole demo site it is sufficient to check the open platform requirement within a small


28/02/2011 of 63 Version 2.0

area) will be chosen by the demo sites in cooperation with the platform developer (MIZAR and PTV). Intended goals are:

• Evaluation of the documentation

• Evaluation of the open platform accessibility via interfaces

• Evaluation of the core process functionalities (e.g. data fusion) of the open platform as specified by WP3

Open issues to be defined by the demo sites and the platform developer during the Viajeo project:

• Kind of additional data set

• Testing area

Example sheet:

The example sheet has to be filled in by the demo site leaders (in cooperation with the platform developer) during the further progress of the Viajeo project.

Validation “Open platform”

(demo site - name)

Selected data source (additional):

Filled in at a later date by the demo sites in cooperation with the platform developer!

Testing area: Filled in at a later date by the demo sites in cooperation with the platform developer!

Data set: • Reference data set

• Additional and manipulated data set

Evaluation method / indicator:

• Comparison of results of the original versus the enhanced implemented system (added by selected data source).

• Simulation of e.g. congestion or free traffic flow by manipulation of the raw data set � Verification: Traffic information services should display the expected traffic situation

Goal: • Evaluation of the documentation

• Evaluation of the open platform accessibility – core system adjustment via interfaces

• Evaluation of the integration possibilities like e.g. data fusion of additional and already existing data


28/02/2011 of 63 Version 2.0

4.4.2. Validation of the new developed/expanded and implemented modules

All modules which will be developed or expanded within the Viajeo project are already identified by WP2 “Investigation”. Based on the investigation reports prepared by the demo sites themselves (D6.1.1, D6.1.3 and D6.1.4) the overall investigation report of WP2 (Version: v26) contains service diagrams of all planned services. The service diagram consists of modules ordered into five layers (data collection, processing, applications, services and devices) with data transfer interfaces between the layers. Within these service diagrams all modules are marked as already existing modules, as modules which have to be new developed or as modules which have to be expanded within the Viajeo project. All new or expanded modules which are likely (depending on data availability) to be implemented in the sites and for this reason need to be validated are marked in the following with a pink

star ( ). Modules which are currently not likely to be implemented are not listed below.

Figure 2: The five layers of the service diagram. Squares indicate modules and circles indicate interfaces

Due to the fact that the data collection itself is not part of the Viajeo project the validation focuses only on the layers processing, application, services and devices. Each layer has been related to several of the following focuses which are also used in the testing phase I (laboratory test) added by the focus of economy:

• Performance - Does the measured performance fit the requirements (dynamic online services possible, real-time capabilities)

• Reliability – validate stability and robustness of the system

• Functionality / user perception – safe operation and output correctness

• Handling – easy to use

• Economy – cost benefit ratio

Table 3 gives an overview of the relation between layers and dedicated validation focuses:

Layer Focus Processing • Performance & Reliability

• Functionality

Application • Performance & Reliability

• Functionality

Service • User perception

• Handling

• Economics

Device • User perception

• Handling

• Economics Table 3: Relation between layers and focuses


28/02/2011 of 63 Version 2.0

According to the MAESTRO Evaluation Guidelines indicators for the different validation focuses are defined, including the name of the indicator, description, data attributes and source of data.

Following service diagrams and indicator lists for each demo site related to corresponding service and module are listed.

4.4.2.1. Athens – Indicator list (field test)

Athens - Taxi fleet management and traffic information (Service A):

1.2

FCD collected by

GPS (current

position)

Stakeholder: Taxi

company

2.1

Taxi data

processing

Stakeholder:

ANCO

3.1

Calculate

Travel times

Stakeholder:

Infotrip

4.1

Professional

service

gateway

(Communicatio

n gateway)

Stakeholder:

ANCO

5.1

Taxi navigation

unit

Stakeholder:

Taxi driver

1.1

Proprietary

XML

SERVICE A:

Taxi Fleet Management and Traffic Information

1.4

Real time traffic

data collected by

road side

equipment

Stakeholder:

Ministry of

Infrastructures

1.1

Taxi assignment

and destination

Stakeholder: Taxi

company

(dispatch center)

1.2

Proprietary

XML

1.4

Proprietary

XML

2.2

Real time

traffic data

processing and

model

Stakeholder:

Infotrip

2.1.2

Proprietary

XML

2.2.1

Proprietary

XML

3.2

Taxi route

engine & Route

monitoring

Stakeholder:

Infotrip

3.2

Proprietary

XML

4.1

Proprietary

XML

1.3

Static data (e.g.

road network)

Stakeholder:

Infotrip

1.3

Proprietary

XML

2.1.1

Proprietary

XML

3.3

Traffic engine

(traffic alerts)

Stakeholder:

Infotrip

2.2.2

Proprietary

XML

3.1

Proprietary

XML

3.3

Proprietary

XML

Figure 4: Diagram of the service chains in Service A (Athens)

Module 2.1: Taxi data processing

Indicator Description

Data attributes

Source of data Comments / Remarks Unit

Type of data

acquisition

Method of data

acquisition

Performance and Reliability test

Status Mean no. of measuring devices working well related to number of active devices per mode (taxi assignment / FCD)


Measured data set

Actuality Mean time between data updates (taxis assignment / FCD system)

Seconds

Quantitative Survey / Measurement

Measured data set

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices/equipment may not work)


Measured data set

User Perception

Correctness / quality

Percentage of accuracy of data output (taxi processing)

% Qualitative Measurement / Survey

Measured data set as well as data of existing system


28/02/2011 of 63 Version 2.0

Module 2.2: Real time traffic data processing and model


Data attributes Source of data

Comments / Remarks Unit

Type of data

acquisition

Method of data

acquisition


Status Mean no. of measuring devices working well related to number of active devices per mode (statistic data / real time data)


Measured data set


Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (real time traffic data)


Measured data set as well as data of a reference measuring system

Module 3.1 Calculate travel time




Type of data

acquisition

Method of data

acquisition


Status Percentage of availability of input data (all possible input data are available)


Measured data set

Actuality Mean time between data updates (travel time)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (travel time)



Module 3.2 Taxi route engine & route monitoring




Type of data

acquisition

Method of data

acquisition




Measured data set


28/02/2011 of 63 Version 2.0

Actuality Mean time between data updates (taxi route / route monitoring)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (taxi route / route engine)



Module 3.3 Traffic engine (traffic alerts)




Type of data

acquisition

Method of data

acquisition




Measured data set

Actuality Mean time between data updates (traffic alerts)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (no. total alerts related to no. of correct alerts)



Module 4.1 Professional service gateway (Communication gateway)




Type of data

acquisition

Method of data

acquisition

Functionality test

Utilizability Mean percentage of error free data communication time between system and gateway


Data stream

Conformity Mean percentage of information available after conversion of transmission protocols


Data stream

Handling test

Operability Percentage of people who are able to operate the gateway (settings)

%



28/02/2011 of 63 Version 2.0

Module 5.1 Taxi navigation unit




Type of data

acquisition

Method of data

acquisition

User perception

Actuality Mean time between data updates (traffic alerts)

Seconds

Quantitative /Qualitative

Survey / Measurement

Measured data set


% Quantitative /Qualitative

Measurement /Survey

Measured data set

Utilizability Mean percentage of error free data communication time between gateway and taxi navigation units

% / unit Quantitative /Qualitative

Measurement / Survey

Measured data set




Measured data set

Handling test

Operability Percentage of people who are able to operate the system (settings)

%


Comprehensibility Percentage of people who understand the given output data of the system

%


Satisfaction Percentage of user who are satisfied with the system

%



28/02/2011 of 63 Version 2.0

Athens – End-user Multi-modal Trip Planning and Trafic Information (Service B)

SERVICE B:

End-user Multi-modal Trip Planning and Traffic Information

3.1

Traffic engine

(traffic

information)

Stakeholder:

Infotrip

4.1

Consumer

service

gateway

Stakeholder:

Infotrip

5.1

Mobile, LBS

app

Stakeholder:

End-user

1.5

Static data

(public transport)

Stakeholder:

Public transport

company,

CERTH/HIT,

Infotrip

1.5

Proprietary

XML

2.2

Real time

traffic data

processing and

model

Stakeholder:

Infotrip

2.2.1

Proprietary

XML

3.1

Proprietary

XML

4.1.1

Proprietary

XML

1.3

Static data (e.g.

road network)

Stakeholder:

Infotrip

1.3

Proprietary

XML

1.2

Real time traffic

data collected by

road side

equipment

Stakeholder:

Ministry of

Infrastructures

1.2

Proprietary

XML

5.2

Web

Stakeholder:

End-user3.3

Multi-modal

route planner

(multi-modal

routing)

Stakeholder:

Infotrip

2.2.2

Proprietary

XML

3.3

Proprietary

XML

4.1.2

Proprietary

XML

1.1

FCD collected by

GPS (current

position)

Stakeholder: Taxi

company

1.4

Proprietary

XML

1.4

Historical data

(Portal)

Stakeholder:

CERTH/HIT

1.1

Proprietary

XML

3.2

Calculate Travel

times

Stakeholder:

Infotrip

3.2

Proprietary

XML

Figure 5: Diagram of the service chains in Service B (Athens)

Module 2.1 Real time traffic data processing and model




Type of data

acquisition

Method of data

acquisition




Measured data set


Seconds


Measured data set



Measured data set

Functionality test






28/02/2011 of 63 Version 2.0

Module 3.1 Traffic engine (traffic information)




Type of data

acquisition

Method of data

acquisition




Measured data set

Actuality Mean time between data updates (traffic information)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (quality of traffic information)



Module 3.2 Calculate travel times




Type of data

acquisition

Method of data

acquisition




Measured data set

Actuality Mean time between data updates (travel time)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (travel time)



Module 3.3 Multi-modal route planner (multi-modal routing)




Type of data

acquisition

Method of data

acquisition




Measured data set


28/02/2011 of 63 Version 2.0

Actuality Mean time between data updates (multi-modal routing)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (multi-modal routing)



Module 4.1 Consumer service gateway




Type of data

acquisition

Method of data

acquisition

User perception




Data stream




Data stream

Handling test


%


Module 5.1 Mobile, LBS, app




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and mobile, LBS, app



Measured data set

Handling test


%



%



%



28/02/2011 of 63 Version 2.0

Module 5.2 Web




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and web



Measured data set

Handling test


%



%



%



28/02/2011 of 63 Version 2.0

Athens – Observatory for Public Authorities and Traffic Planners (Service C)

SERVICE C:

Observatory for Public Authorities and Traffic Planners

3.1

Observatory

(Portal)

Stakeholder:

CERTH/HIT

4.1

Public

authorities

gateway

(Portal)

Stakeholder:

CERTH/HIT

5.1

Public

authority

1.3

Transport and

traffic indicators

(Portal)

Stakeholder:

CERTH/HIT

2.1

Data

aggregation

and processing

Stakeholder:

CERTH/HIT4.1.2

Proprietary

XML

1.2

Historical data

(Portal)

Stakeholder:

CERTH/HIT

1.1

Real time traffic

data

Stakeholder:

Infotrip

1.1

Proprietary

XML

5.2

Traffic planner

4.1.1

Proprietary

XML

Figure 6: Diagram of the service chains in Service C (Athens)

Module 2.1 Data aggregation and processing




Type of data acquisition

Method of data

acquisition


Actuality Mean time between data updates (real time / traffic indicators)

Seconds


Measured data set

No of accurate DB updates

Ratio of total devices vs correct devices (statistical analysis)


No of inconsistent DB matchings

No Quantitative Survey / Measurement

Functionality test


Percentage of accuracy of data output (data aggregation / data fusion)




28/02/2011 of 63 Version 2.0

Module 3.1 Observatory (Portal)




Type of data

acquisition

Method of data

acquisition


No of inaccurate calculationa

No of inadequate calculation vs total no.


Measured data set

More to be defined

Functionality test


Percentage of accuracy of data output (traffic indicators)



Module 4.1 Public authorities gateway (Portal)




Type of data

acquisition

Method of data

acquisition

User perception




Data stream




Data stream

Handling test


%


Module 5.1 Public authority




Type of data

acquisition

Method of data

acquisition

User perception

No of false downloads of data

Mean percentage of error free data communication



Measured data set




Measured data set

Handling test


%



%



28/02/2011 of 63 Version 2.0

Module 5.2 Traffic planner




Type of data

acquisition

Method of data

acquisition

User perception

No of false downloads of data

Mean percentage of error free data communication



Measured data set




Measured data set

Handling test


%



%



28/02/2011 of 63 Version 2.0

Data Type

4.4.2.2. Sao Paulo – Indicator list (field test)

Viajeo São Paulo site

Service 1: Real time traffic data processing with floating vehicle data

Module 2.1 Real time traffic data processing and model




Type of data

acquisition

Method of data

acquisition




Measured data set

Actuality Mean time between data updates ( FCD)

Seconds


Measured data set



Measured data set

Data Source Users

Services Data processing

Viajeo Platform

Historical traffic

database

Floating Car Data

(FCD

Fleet Management

Operators

Real time traffic

data

GPS equipped cars

Real time

information

Web based

journey planner

Dynamic traffic

information

systems

End-user on-board

unit & website

Mizar, IPT

Magnetti Marelli Magnetti Marelli

Traffic Engine

(traffic info &

alerts)

Traffic data

processing & model

Mizar

Mizar

Magnetti Marelli, Mizar

Magnetti Marelli


28/02/2011 of 63 Version 2.0

Functionality test





Service 2: Traffic Engine (traffic information) Module 3.1 Traffic engine (traffic information)




Type of data

acquisition

Method of data

acquisition




Measured data set

Actuality Mean time between data updates (traffic information)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (quality of traffic information)



Service 3: Traffic information for end users (web-based and mobile applications)

Module 5.1 Mobile application




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and mobile applications



Measured data set

Handling test


%



%



%



28/02/2011 of 63 Version 2.0

Module 5.2 Web




Type of data

acquisition

Method of data

acquisition

User perception




Measured data set

Handling test


%



%



%



28/02/2011 of 63 Version 2.0

4.4.2.3. Beijing – Indicator list (field test)

Beijing – Floating car data (T-System+BTRC)

1.2 FCD T-

systems

1.2 Proprie-

tary

kml/xml

4.2 Consumer

Service Gateway

T-systems

3.1 Traffic

Engine and model

T-systems

2.2.1 Mobile Real-

Time Data

Processing

2.2.2 DB processing

T-systems

3.1 Proprie-

tary

kml/xml

2.2 Proprie-

tary

kml/xml

4.1 Public

Authority

Gateway

T-systems

4.2

Proprie-

tarykml/xml

4.1

Proprie-

taryKml/xml

5.3 Information

in the Internet

T-systems

5.2 Information

in the Internet

T-systems

Figure 7: Service diagram - floating car data from private vehicles (Beijing)

Module 2.2.1 Mobile real time processing




Type of data

acquisition

Method of data

acquisition


Setup No. of measuring devices / equipment (FCD)

No. Quantitative Survey / Measurement

Measured data set

Status Mean no. of measuring devices working well related to number of active FCD devices


Measured data set

Actuality Mean time between data updates (Mobile real-time data processing)

Seconds


Measured data set

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices/FCD may not work)


Measured data set

Functionality test


Percentage of accuracy of data output (Mobile real-time data processing)



Module 2.2.2 DB processing




Type of data

acquisition

Method of data

acquisition


Setup No. of measuring devices / equipment (FCD)


Measured data set

Status Mean no. of measuring devices working well related to number of active FCD devices


Measured data set

Actuality Mean time between data updates (DB processing)

Seconds


Measured data set


28/02/2011 of 63 Version 2.0

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices/FCD may not work)


Measured data set

Functionality test


Percentage of accuracy of data output (DB processing)



Module 3.1 Traffic Engine and model




Type of data

acquisition

Method of data

acquisition


Setup No. of input data/sources No. Quantitative Survey / Measurement

Measured data set



Measured data set

Actuality Mean time between data updates (Traffic data and model)

Seconds


Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (Traffic data and model)



Module 4.1 Public Authority Gateway




Type of data

acquisition

Method of data

acquisition

User perception




Data stream




Data stream

Handling test


%


Economic test

Cost of operation Mean costs of operation and maintenance costs

€/month Quantitative Survey


28/02/2011 of 63 Version 2.0

Module 4.2 Consumer Service Gateway




Type of data

acquisition

Method of data

acquisition

User perception




Data stream




Data stream

Handling test


%


Economic test



Module 5.2 Information in the internet (public authorities & Operators)




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and internet



Measured data set

Handling test


%



%



%


Economic test



Module 5.3 Information in the internet (Public)




Type of data

acquisition

Method of data

acquisition

User perception




Measured data set

Handling test


%



28/02/2011 of 63 Version 2.0


%



%


Economic test




28/02/2011 of 63 Version 2.0

Beijing – Data from buses equipped with Automatic Vehicle Monitoring system (AVM)

Figure 8: Service diagram, data from buses equipped with AVM (Beijing)

Module 2.5 Realtime Bus Data processing BPT


Data attributes

Source of data Comments / Remarks Unit

Type of data

acquisition

Method of data

acquisition


Setup No. of measuring devices / equipment (AVM)


Measured data set

Status Mean no. of AVM devices working well related to number of active AVM


Measured data set

Actuality Mean time between data updates (Real time bus data processing)

Seconds


Measured data set

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices / bus data may not work)


Measured data set

Functionality test


Percentage of accuracy of data output (Real time bus data processing)



Module 5.1 Bus stop




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and bus stop information

% Quantitative Measurement / Survey Measured data set

Handling test


%



28/02/2011 of 63 Version 2.0


%



%


Economic test



Module 5.2 Onboard




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and onboard information



Measured data set

Handling test


%



%



%


Economic test




28/02/2011 of 63 Version 2.0

4.4.2.4. Shanghai – Indicator list (field test)

Shanghai – data from bus equipped with GPS

Figure 9: Service diagram, data from buses equipped with GPS

Module 2.1 Bus data processing




Type of data

acquisition

Method of data

acquisition


Setup No. of measuring devices / equipment (Bus data)

No.

Quantitative


Measured data set

Status Mean no. of measuring devices working well related to number of active Bus data devices


Measured data set

Actuality Mean time between data updates (Real time data processing)

Seconds


Measured data set

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices / bus data may not work)


Measured data set

Functionality test


Percentage of accuracy of data output (Bus data processing)



Module 3.1 Bus arrival time




Type of data

acquisition

Method of data

acquisition


Setup No. of input data/sources No. Quantitative Survey / Measurement

Measured data set



Measured data set



Measured data set

Functionality test


Percentage of accuracy of data output (public transport journey planner)




28/02/2011 of 63 Version 2.0

Module 5.1 Display systems at bus stops




Type of data

acquisition

Method of data

acquisition

User perception

Utilizability Mean percentage of error free data communication time between gateway and onboard information



Measured data set

Handling test


%



%



%



28/02/2011 of 63 Version 2.0

Shanghai - Environmental data from mobile sensors

Figure 10: Service diagram, environmental data from mobile sensors

Module 2.2 Road Environment data processing




Type of data

acquisition

Method of data

acquisition


Setup No. of measuring devices / equipment (road environment detector)


Measured data set

Status Mean no. of measuring devices working well related to number of active road environment detectors


Measured data set

Actuality Mean time between data updates

Seconds



Measured data set

Reliability Percentage of error-free run-time related to total runtime (whole system, single devices / road environmental equipment may not work)


Measurement /Survey

Measured data set

Module 3.2 Realtime monitoring




Type of data

acquisition

Method of data

acquisition

Functionality test


Percentage of accuracy of data output (environmental model)



Module 4.2 Web based information for user




Type of data

acquisition

Method of data

acquisition

User perception


Days



Measured data set


28/02/2011 of 63 Version 2.0



Measurement /Survey

Measured data set




Measured data set

Module 4.3 Environment model




Type of data

acquisition

Method of data

acquisition

User perception


Minutes



Measured data set



Measurement /Survey

Measured data set




Measured data set


28/02/2011 of 63 Version 2.0

4.4.3. Validation of selected and implemented European standards (adaptability and practicability)

The open platform itself as well as the integrated local components and services consist of modules and their dedicated interfaces. One of the Viajeo requirements is that – as far as possible – all interfaces correspond to European standards. Due to the fact that Viajeo is a R&D project with demo sites in Europe, China and Brazil it is likely that their might be difficulties regarding the implementation of European standards. Difficulties can be caused e.g. by a lack of suitable European standards (because of new technologies which are still in the research phase) or by incompatibility of the European standards and the local components and services which can not be solved within the Viajeo project. However the table below gives an overview about the selected standards which are likely to be integrated into the implemented systems in the demo sites. Beside columns for the interface name, designation, typically transmitted data and standard to be used there are also columns for remarks, demo sites and comments. The remarks are related mainly to general difficulties (non implemented system) and are made by WP3 while the comments are related to specific difficulties during the implementation of the systems in the sites. Comments will be given during the implementation (at later stage of the project) by those partners who are responsible for the implementation.

Interface Name Desig-nation

Data Typically Transmitted Standard to be used

Remark Demo sites Comments

FCD I1 Vehicle ID Time Stamp Position Speed

SIMONE SIMONE does not count as an European Standard (see chapter 4.3 for explanation, D3.1) but is used in the market

Athens, Brazil, Beijing, Shanghai

Roadside Traffic Count Data

I2 Detector ID Lane N° Direction Traffic Count (passenger cars) Traffic Count (lorries) Time Gap (between vehicles)

DATEX 2 Road side units (especially in case of older systems) may feature low computing power and low bandwidth connections. DATEX 2 is designed for centre to centre communication and rather heavy, hence a level 3 compliant interface is used.

Athens, Brazil

Roadside Environmental Data

I3 Detector ID Direction Measurement CO2 Measurement NOx

DATEX 2 Shanghai

Centre Road Traffic Data Interface

I4 Traffic Events Traffic Load per Segment

DATEX 2 Classic DATEX 2 application Athens, Brazil

Static Public Transport Information

I5 Timetables Stop Positions

VDV 452 SIRI does not cover static data, the German VDV 452 standard is based on the same principles and widely used in Europe

Beijing, Shanghai

Dynamic Public Transport Information

I6 Vehicle ID Time Stamp Delays (line,vehicle)

SIRI Beijing, Shanghai

Traffic Information over TPEG

I7 Traffic Messages Traffic Load

TPEG RTM/TEC

TPEG RTM defined as widely used standard, TPEG TEC is widely used in the European automotive sector.

Athens, Brazil, Beijing

Comodal Routes I8 Waypoints Travel Time on Links Traffic Mode

IN TIME / eMotion

No public standard available, the European Project IN TIME defined a well received approach (see chapter 4.7 for explanation).

Athens, Brazil, Beijing

Table 1: Interface – standard – demo site relation (Source: PTV)


28/02/2011 of 63 Version 2.0

5. Impact assessment

The present section deals with the “Impact assessment – Testing Phase II (Field test)” that refers to the evaluation of expected impacts of the Viajeo project. For the impact assessment an already implemented systems is obligatory.

5.1. Objectives

Goal of the impact assessment is to evaluate the expected impacts of the Viajeo project to users (professionals, non-professionals), stakeholder and decision makers in the demo sites. A broad overview about the impacts in the different sites is important to evaluate the whole Viajeo project and to validate if the project requirements are met.

5.2. Target groups

The following three target groups have been identified to participate in the impact assessment of the Viajeo Platform:

• End Users: These are the daily users that are expected to use most of the services of the Viajeo platform once they are finalized. Members of this group have already been involved in the project during the identification of user needs in the framework of WP2 (residents, visitors, taxi drivers)

• Transport and traffic planners (incl. stakeholder and decision maker): These are planners or decision makers responsible for the transport and traffic planning not only of the involved cities but of other cities as well. They can either be high level civil servants working for relevant ministries, municipalities and authorities and responsible for the decision making, but also experts as well as researchers.

• Transport operators: This group involves people from transport companies providing transport services to citizens.

5.3. Approach and methodology – impact assessment

A brief overview of the chosen approach and methodology is given in the following. Basis for the impact assessment are the finally implemented use cases in the demo sites. These use cases are developed in WP 2 based on the user needs identified during the investigation in the demo sites. Related to the use cases services/extensions are developed and planned by the demo sites themselves.

Not all use cases defined in WP 2 are going to be finally implemented in the demo sites. The following table gives an overview about the use cases which will be implemented in the Viajeo project related to the corresponding demo sites.

For each implemented use case a set of indicators has been developed according to the MAESTRO Evaluation Guideline which means that each indicator is related to an evaluation category, a sub category and an impact. Furthermore there is a short description of the indicator as well as information about (expected) data attributes and the intended target groups.


28/02/2011 of 63 Version 2.0

Implemented use cases Demo site for the use cases

Number Description Athens Sao Paulo

Beijing Shanghai

1 Real time bus arrival information on displays at bus stops

X X

2 Web-based journey planner

Multi-model journey planer (pre-trip) X X

Car only journey planer (pre-trip) X

Multi-modal journey planer on buses (on-trip) X

3 Dynamic assignment of taxies X

4 Traffic alerts for taxi drivers (realtime) X

5 Traffic information to end users (realtime)

Mobile application / traffic information / route planer

X

Mobile application X

Traffic information X

6 Processed traffic information and indicators to transport and traffic planners

With specific users (transport and traffic planner) X

Interface without specific users (Sao Paulo) X

7 Real time bus operation (to operators) X

8 Environmental data monitoring

Real time monitoring X

Historic data base and modelling X

5 3 5 3

Table 2: Relation between implemented use cases and demo sites


28/02/2011 of 63 Version 2.0

5.3.1. Impact assessment indicators – Athens

Use case 2: Web-based multi modal journey planner

Number Evaluation

Sub-Category Impact Indicator Description

Data attributes

Target group Unit


Method of data

acquisition

Acceptance

2.1.1 General attitude (operator)

Operator attitude

Operator acceptance

Increase of frequency of use of advanced technologies in urban mobility

Index Qualitative Survey Transport and traffic planer

2.1.2 Degree to which the transport operators can adapt to the changes caused by the new services


2.1.3 General attitude (user)

New technology acceptance

User Acceptance

Increase of satisfaction of the users when travelling in the urban network due to offered services

Index

Qualitative Survey End user

Accessibility

2.2.1 Accessibility Service integration

Interchange points

Possibility of change of route of users due to the provided information

Index Qualitative Measurement End user

2.2.2 Service integration

Achievement of better coordination between different transport modes

Index Qualitative Survey End user

Transport patterns

2.3.1 Route change Route change Average route change trips



2.3.2 Route guidance

Improved journey time

Reduction of travel times Index Quantitative Qualitative

Measurement or modelled

End user


28/02/2011 of 63 Version 2.0

Quality of service

2.4.1 Information Availability of information

Information accessibility

Percentage of people who know to access information

% Index

Quantitative Qualitative

Survey End user

2.4.2 Ease of use Ease of use Clarity of information

Increase of overall satisfaction / understanding of information of special categories of users (elderly people, mobility impaired, etc) by the provided services of public transport modes


Use case 3: Dynamic assignment of taxies

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


Operator attitude

Operator acceptance

Increase of the use of new technologies by the taxi companies

Index Qualitative Survey Transport operators

3.1.2 Satisfaction of taxi companies by the available information for the route planning

Index Qualitative Survey Transport operators


User acceptance of demonstrated service

Acceptance rating

Degree to which the use of more up to date devices by taxi drivers will increase as a result of the services offered through them.

Index


3.1.4 Satisfaction of taxi drivers due to the more efficient distribution of their routes

Index



28/02/2011 of 63 Version 2.0

Accessibility

3.2.1 Accessibility Accessibility System access distance

Increase in quality of provided services in terms of quick response to a call for a taxi due to the decrease of traffic congestion

Minutes Index


Measurement Survey

End user / Transport operators

Capacity

3.3.1 Network efficiency

Network efficiency

Passenger load factor

No. occupied taxis in relation to available taxis

% / hr Index


Collected / Measurement Survey


3.3.2 More optimized transport network due to the more efficient planning of taxi routes

Index Qualitative Collected / Survey


Quality of service



Percentage of taxi driver who can access information



Rating given to ease of understanding information


Use case 4: Traffic alerts for taxi drivers

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance



Acceptance rating


Index

Qualitative Survey End user/ Taxi Driver


28/02/2011 of 63 Version 2.0

4.1.2 Satisfaction Increase of satisfaction of the users when travelling in the urban network due to offered services

Index


Transport patterns

4.2.1 Route change Route change Average route change trips

Increase of taxi drivers with changed route

% Index


Measurement or Survey

Taxi driver

4.2.2 Route guidance

Improved journey time

Journey time saved due to revised route choice

Index Quantitative Qualitative

modelled / Survey

Taxi driver

Quality of service



Index Qualitative Survey Taxi driver

Use case 5: Traffic information to end users

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance



Acceptance Increase of frequency of use of advanced technologies in urban mobility

Index



Index


Transport patterns

5.2.1 Reduction of traffic

Reduction of traffic

Route changes

Possibility of change of route of users due to the provided information

Index Qualitative Survey End user/ Taxi Driver


28/02/2011 of 63 Version 2.0

Quality of service



people who know to access information


5.3.2 Success of advertising

people who know about service


Use case 6: Processes traffic information and indicators to transport and traffic planners

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


Operator attitude

Operator Acceptance

Increase of satis-faction of planners from kind of available information, traffic models and data accuracy




User Acceptance

Increase in the number of users


Accessibility

6.2.1 Accessibility Better transport systems accessible for all

Increase of special user groups i.e. seniors

Increase of number of users and satisfaction by transport and planning


Transport Pattern

6.3.1 Optimisation More Optimised transport systems

Reduction of travel times

Reduction of travel time due to the better coordination of transport modes



28/02/2011 of 63 Version 2.0

6.3.2 Congestion More Optimised transport systems

Decrease of Congestions

Decrease in the traffic congestion due to the better coordination of transport modes



28/02/2011 of 63 Version 2.0

5.3.2. Impact assessment indicators - Sao Paulo

Use case 2: Web-based journey planner

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


User acceptance of demonstrated data

Acceptance rating

Attitude survey of current and potential utility

Index


Transport patterns

2.3.2 Route guidance Improved journey time

Reduction of travel times Index Qualitative Measurement or modelled

End user

Capacity

2.2.1 Information Quality index

Composition rating of pre defined criteria / set of information

Index Qualitative Collected / Measurement or Survey

End user


Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance




Index



28/02/2011 of 63 Version 2.0


Index


Quality of service




Index/% Qualitative/ Quantitative

Survey End user/ Taxi Driver



Index/% Qualitative Quantitative

Survey End user/ Taxi Driver

Use case 6: Processes traffic information and indicators to transport and traffic planners

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


Operator attitude

Operator Acceptance

Increase of satis-faction of planners from kind of available information, traffic models and data accuracy




User Acceptance

Increase in the number of users


Accessibility

6.2.1 Accessibility Better transport systems accessible for all

Increase of special user groups i.e. seniors

Increase of number of users and satisfaction by transport and planning



28/02/2011 of 63 Version 2.0

5.3.3. Impact assessment indicators – Beijing

Use case 1: Real time bus arrival information on displays at bus stops

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


User acceptance of demonstrated Data

Acceptance rating


Index


1.1.2 Accessibility Access Service access Information readability and information content


Quality of service



Percentage of people who know how to access information


1.2.2 Information of advertising

Percentage of people who know or have recognized the new service


Use case 2: Web-based journey planner (pre trip)

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance


Operator attitude

Operator acceptance

Acceptance confidence rating given by operator to demonstration parameters



28/02/2011 of 63 Version 2.0



Acceptance rating


Index


2.1.4 Ease of modal transfer

Level of conflict between modes

Conflict rating

Public survey to measure parameters of conflict between modes


Capacity

2.2.1 Information Quality index

Composition rating of pre defined criteria / set of information

Index Qualitative Collected / Measurement or Survey

End user

Quality of service






Percentage of people who know about service


Use case 2: Web-based journey planner (on trip)

Number Evaluation


Data attributes Target group Unit


Method of data acquisition

Acceptance



Acceptance rating


Index


2.1.4 Access for mobility-impaired PAX

System access

Survey about the possibility of use by mobility-impaired PAX (e.g. location inside/ free space in front / height of installation)



28/02/2011 of 63 Version 2.0

Quality of service






Percentage of people who know about service



Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance




Index



Index


Quality of service








Use case 7: Real time bus operation

Number Evaluation




Method of data

acquisition

Acceptance

7.1.1 General Operator Operator Acceptance confidence Index Qualitative Survey Transport


28/02/2011 of 63 Version 2.0

attitude (operator)

attitude acceptance rating given by operator to demonstration parameters

operator

Quality of service





28/02/2011 of 63 Version 2.0

5.3.4. Impact assessment indicators – Shanghai

Use case 1: Real time bus arrival information on displays at bus stops

Number Evaluation




Method of data acquisition

Acceptance


User acceptance of demonstrated Data

Acceptance rating


Index


1.1.2 Accessibility Access Service access Information readability and information content


Quality of service



Percentage of people who know how to access information


1.2.2 Information of advertising

Percentage of people who know or have recognized the new service


1.2.3 Service frequency

Service frequency

Average update time

Average time between information updates

Min. Quantitative Measurement End user

1.2.4 Reliability reliability Accuracy of timekeeping

Percentage of services arriving within given arrival time






28/02/2011 of 63 Version 2.0

Use case 8.1: real time environmental data monitoring

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance

8.1.1 General attitude

General attitude

Acceptance Acceptance confidence rating to demonstration parameters


Quality of service

8.1.2 Accuracy Accuracy of information

Information accuracy

Comparison mobile to fixed stations

Index Qualitative Survey End user and Operator

Use case 8.2: environmental data and modelling

Number Evaluation


Data attributes

Target group Unit


Method of data

acquisition

Acceptance

8.2.1 General attitude

Planer attitude

Acceptance Acceptance confidence rating given by operator to demonstration parameters


Quality of service

8.2.2 Accuracy Accuracy of information

Information accuracy

Comparison measured and modelled dataset



28/02/2011 of 63 Version 2.0

6. Perspectives for European ITS technologies in China and Brazil

Following a brief overview of the proceeding used for the assessment of the perspectives for European ITS technologies in China and Brazil are given.

6.1. Objectives The partners in the Viajeo project aim to ensure that the project will have long lasting effects on the cooperation between European companies dealing in ITS technologies and users at the test sites and other markets. To help achieve this goal, task 5.5 will assess the future potential for export of European technology to the Chinese and Brazilian markets and analyse how entrance into these markets will be able to have a successful outcome.

6.2. Approach and Methodology

The analysis will be conducted on two parallel tracks.

• Firstly, existing models normally used by European firms when considering entering the two markets in question will be discussed. Focus will be placed on both the strategic reflections a company must go through before expansion and practical behaviour necessary to obtain success when first established on the market. Even though China and Brazil have historically been popular for outsourcing, this analysis will only examine the countries in their capacity of markets and ability to purchase European ITS technology. Business models for exporting to China and Brazil will not necessarily be alike for different product groups because buyer segments vary and competition may be fierce in some industries and lax in others. Thus the analysis will as far as possible be centered around export of technologies and ITS technology more precisely.

• Secondly, concrete business models considered by the partners at the test sites will be surveyed. These surveys will be conducted through interviewing the partners who are already on the Chinese and Brazilian markets. The interviews will seek to establish how the partners practically handle the culturally and legislatively different markets and how they plan to expand their operations, if indeed they have such plans. Naturally the interviews will focus on export rather than outsourcing and on the issues specifically faced when offering ITS technology.

The two analyses will result in a guideline for the successful commercial utilization of European ITS technologies in Sao Paulo, Shanghai and Beijing. The aim is that the guideline will be used for opening other Chinese and Brazilian cities for European companies, so European technology will gain a more solid market share in these countries and boost European export.


28/02/2011 of 63 Version 2.0

7. Data (collection, analysis, validation, tables and reports)

Below a short description about the collection, validation and analysis of the data used in WP5 is given.

7.1. Data - collection Goal of the Viajeo project is to implement an open platform system in different demo sites. This will be done close to the end of the project duration during the demonstration of the demo site. It’s important to note that the data collection will take place during a limited period. Comprehensive measurements like long term measurements are not part of WP5.

The data collection will be carried out at the different test sites, for each of the use cases considered in the validation and impact assessment. The collection itself will be done in the demonstration WP.

The finally choice of the method of data collection is within the responsibility of the corresponding partner in the demo site. This makes sense because these partners know the locally situation. However for each indicator a method of data collection is proposed by WP5 which should (if possible) be used.

7.2. Data - analysis A data analysis can only be as good as the used data basement. Since the data basement within the Viajeo project is not that comprehensive (compare: data-collection) the generated results need to be interpreted carefully. All partners in the demo sites are requested to collect the data as carefully as possible. WP5 will put the results into the context which allows a more general interpretation.

7.3. Data - validation Due to the fact that Viajeo is a research project all collected validation and impact assessment data have to be used carefully related to the circumstances of their collection. A universal validity of the collected data and/or the generated results is therefore not possible. Based on the independent results of the four different demo sites it is very well possible to get a competent knowledge of difficulties and possibilities.

7.4. Data – charts of responsibility The charts of responsibility give information (description, related to, format, responsibility, delivery date) about needed / required data sets which are used in the validation and impact assessment process of the Viajeo project (WP5). There will be one chart for each demo site. The charts will be filled during the process of implementation of functionality and services and will be updated on a regular basis, corresponding to available and/or required data. First contacts within the sites are the site representatives who have two possibilities: (1) to deliver the data or (2) to appoint a finally responsible contact person who deliver the data.

All data sets will be given to the WP5 leader (DLR) who collects and (if necessary) provides the data. This guarantees that all necessary data sets (including possible updates) are available in one place.

The structure of the charts of responsibility is exemplary shown in Table 2.

Athens Data set Related to Data

format Responsibility Delivery date

Description (Name, short description)

Module (Number)

Indicator (Name)

Format (e.g. Excel,

Word, data base)

Organisation (Name)

Person (Name, email, phone-nr.)

Intended (date)

Real (date)

FCD single car Service A all Data base Infotrip Mr. / Ms.

FCD accumulated Service B all Data base Infotrip Mr. / Ms.

… … … … … … Table 3: Chart of responsibility - Example


28/02/2011 of 63 Version 2.0

7.5. Reporting system for WP5 Within the Viajeo project the open platform system will be implemented in four cities which belong to three different continents. Data collection and at least a first data analysis and validation will be organized by the demo sites. The results and reports will be part of the “Assessment and validation results (D5.2)”. To harmonize/standardize the results and reports of the four demo sites a kind of guideline (e.g. table of content) will be prepared by WP5. All WP5 partners are required to follow the guidelines. This guarantees a constant quality of the reports and makes it easier to integrate the results into the final deliverable of WP5 (D5.2).

Viajeo Validation Plan D5 1 Final Version · E-mail: [email protected] Abstract The deliverable...

Documents

Transcript of Viajeo Validation Plan D5 1 Final Version · E-mail: [email protected] Abstract The deliverable...