Deliverable 10.4 Project Final Report€¦ · ADAS&ME (688900) D10.4 – Final project report...

Adaptive ADAS to support incapacitated drivers Mitigate Effectively risks through tailor

made HMI under automation

This project has received funding from the European Union’s Horizon 2020

research and innovation programme under grant agreement No 688900

Deliverable 10.4 – Project Final Report

Deliverable Identity

Work Package No. WP10

Work Package Title Management

Activity No. A10.1

Activity Title Administrative and Overall Management

Dissemination level PU = Public

Main Author(s)

Anna Anund (VTI)

Lena Nilsson (VTI)

File Name ADASANDME_Deliverable_xx.x_dd-mm_y.doc

Online resource http://www.adasandme.com

Ref. Ares(2020)1270577 - 28/02/2020

http://www.adasandme.com/link-to-deliverable

ADAS&ME (688900) D10.4 – Final project report

February 2020 of 68 Version 2.1

Version History

Date Version Comments

2020-02-16 1.0 First draft made by VTI with input from WP-leaders

and use case leaders

2020-02-19 1.1 Reviewed by Quality manager

2020-02-23 2.0 Revised by VTI

2020-02-26 2.1 Revised by CERTH

Authors (full list)

Anna Anund, VTI

Lena Nilsson, VTI

Andreas Absnër, Scania

Frederik Diedrich, Fraunhofer

Marc Figuls, RACC

James Jackson, IDIADA

Christoph Allig, Denso

Marcel Mathissen, Ford

Eleonora Meta, CTL

Stella Nikolaou, CERTH

Kevin Nguyen, Valeo

Davide Sette, Ducati

Sri Venkata Naga Phanindra Akula, TUC

Marc Wilbrink, DLR

Project Coordinator

Dr. Anna Anund

Research Director / Associate Professor

VTI - Olaus Magnus väg 35 / S-581 95 Linköping / Sweden

Tel: +46-13-20 40 00 / Direct: +46-13-204327 / Mobile: +46-709 218287

E-mail: [email protected]

Legal Disclaimer

The information in this document is provided “as is”, and no guarantee or warranty is given that the information

is fit for any particular purpose. The above referenced authors shall have no liability for damages of any kind

including without limitation direct, special, indirect, or consequential damages that may result from the use of

these materials subject to any liability which is mandatory due to applicable law.

The present document is a draft. The sole responsibility for the content of this publication lies with the authors. It

does not necessarily reflect the opinion of the European Union. Neither the INEA nor the European Commission

is responsible for any use that may be made of the information contained therein.

© 2016 by ADAS&ME Consortium

mailto:[email protected]

ADAS&ME (688900) Deliverabel 10.4


Table of Contents

1 THE SCOPE OF ADAS&ME ...................................................................................................................... 11

1.1 OBJECTIVES ................................................................................................................................................ 13 1.1.1 Objectives of ADAS&ME ................................................................................................................ 13 1.1.2 Objective of this report .................................................................................................................. 13

2 USE CASES AND PRIORITY SCENARIOS – SETTING THE SCENE ................................................................ 14

3 ADAS&ME WORK PROCESS ................................................................................................................... 17

3.1 SYSTEM ARCHITECTURE AND SPECIFICATIONS ..................................................................................................... 17 3.2 ENVIRONMENTAL SENSING............................................................................................................................. 19 3.3 DRIVER/RIDER STATE MONITORING.................................................................................................................. 21

3.3.1 Physical Fatigue ............................................................................................................................. 22 3.3.2 Rest ................................................................................................................................................ 22 3.3.3 Sleepiness ...................................................................................................................................... 23 3.3.4 Stress ............................................................................................................................................. 23 3.3.5 Distraction ..................................................................................................................................... 23 3.3.6 Emotion ......................................................................................................................................... 23

3.4 HMI ACTIONS AND TRANSITIONS.................................................................................................................... 25 3.4.1 Iterative HMI development and testing ........................................................................................ 25 3.4.2 HMI Framework............................................................................................................................. 26 3.4.3 HMI Elements and Modalities ....................................................................................................... 27 3.4.4 Automation ................................................................................................................................... 29 3.4.5 HMI personalisation ...................................................................................................................... 30 3.4.6 Decision and Support System ........................................................................................................ 31 3.4.7 Main Innovations........................................................................................................................... 32 3.4.8 Outlook .......................................................................................................................................... 32

3.5 INTEGRATION IN DEMONSTRATORS ................................................................................................................. 33 3.6 EVALUATIONS ............................................................................................................................................. 35

3.6.1 Evaluation framework ................................................................................................................... 35 3.6.2 Pre-pilot data collections ............................................................................................................... 36 3.6.3 Final evaluations ........................................................................................................................... 37 3.6.4 UC A - Truck ................................................................................................................................... 37 3.6.5 UC B – Electrical passenger car ..................................................................................................... 38 3.6.6 UC C/D - Conventional passenger car ............................................................................................ 39 3.6.7 UC E/F - Motorbike ........................................................................................................................ 40 3.6.8 UC G – Automated docking at bus stop ......................................................................................... 41 3.6.9 Demanding issues during the final evaluation .............................................................................. 42 3.6.10 Achievements ........................................................................................................................... 42 3.6.11 Recommendations .................................................................................................................... 43

4 USE CASE APPROACH AND ACHIEVEMENTS .......................................................................................... 44

4.1 USE CASE A – TRUCK ................................................................................................................................... 44 4.1.1 Aim ................................................................................................................................................ 44 4.1.2 Approach ....................................................................................................................................... 44 4.1.3 Achievements ................................................................................................................................ 46 4.1.4 Innovations .................................................................................................................................... 48 4.1.5 Limitations ..................................................................................................................................... 48

4.2 USE CASE B - ELECTRICAL VEHICLE .................................................................................................................. 49 4.2.1 Aim ................................................................................................................................................ 49 4.2.2 Approach ....................................................................................................................................... 49 4.2.3 Achievements ................................................................................................................................ 50

4.3 USE CASE C/D - CONVENTIONAL PASSENGER CAR .............................................................................................. 51 4.3.1 Aim ................................................................................................................................................ 51 4.3.2 Approach ....................................................................................................................................... 51 4.3.3 Achievements ................................................................................................................................ 52 4.3.4 Innovations .................................................................................................................................... 53



4.3.5 Limitations ..................................................................................................................................... 53 4.4 USE CASE E/F - MOTORBIKE AND PROTECTIVE GEAR........................................................................................... 53

4.4.1 Aim ................................................................................................................................................ 54 4.4.2 Approach ....................................................................................................................................... 54 4.4.3 Achievements ................................................................................................................................ 54 4.4.4 Limitations ..................................................................................................................................... 55

4.5 USE CASE G - CITY BUS ................................................................................................................................. 55 4.5.1 Aim ................................................................................................................................................ 55 4.5.2 Approach ....................................................................................................................................... 56 4.5.3 Achievements ................................................................................................................................ 56 4.5.4 Innovations .................................................................................................................................... 57 4.5.5 Limitations ..................................................................................................................................... 57

5 ASSESSMENT OF THE IMPACT ............................................................................................................... 58

5.1 SAFETY, MOBILITY AND ENVIRONMENT ............................................................................................................. 58 5.2 ECONOMIC AND SOCIAL IMPACT ...................................................................................................................... 59 5.3 LEGAL AND REGULATORY IMPACT ASSESSMENT .................................................................................................. 60

6 EXPLOITATION OF THE RESULTS ............................................................................................................ 61

7 DISSEMINATION .................................................................................................................................... 63

8 CONCLUSIONS ....................................................................................................................................... 66

9 CONTACTS ............................................................................................................................................. 67

9.1 COORDINATION TEAM................................................................................................................................... 67 9.2 DISSEMINATION MANAGER ............................................................................................................................ 67 9.3 WP LEADERS .............................................................................................................................................. 67 9.4 USE CASE LEADERS ....................................................................................................................................... 67

10 REFERENCES .......................................................................................................................................... 68



Index of Figures FIGURE 1 THE DECISION AND STRATEGY LOOP FOR ADAS&ME .......................................................................................... 11 FIGURE 2 ADAS&ME PROJECT OUTLINE. ....................................................................................................................... 12 FIGURE 3 OVERALL METHODOLOGY FOR DEFINITION AND SELECTION OF FINAL USE CASES OF ADAS&ME. .................................. 14 FIGURE 4 ADAS&ME WORKSHOP FOCUSING ON USE CASES AND SCENARIOS PRIORITISATION. ................................................. 15 FIGURE 5 SELECTED USE CASES...................................................................................................................................... 16 FIGURE 6 DRIVER/RIDER MODEL. .................................................................................................................................. 16 FIGURE 7 FRAMEWORK FOR ADAS&ME SYSTEM SPECIFICATIONS. ..................................................................................... 17 FIGURE 8 ADAS&ME GENERAL SYSTEM ARCHITECTURE. .................................................................................................. 18 FIGURE 9 DATA FLOW BETWEEN MODULES INCLUDED FOR ENVIRONMENTAL SENSING.............................................................. 20 FIGURE 10 VARIOUS LAYERS OF LOCAL DYNAMIC MAP. .................................................................................................... 21 FIGURE 11 OVERVIEW OF ALL DRIVER STATES AND SENSORS APPLIED FOR EACH USE CASE. ...................................................... 22 FIGURE 12 PERFORMANCE RESULTS OF DRIVER STATES ALGORITHMS. ................................................................................... 24 FIGURE 13 PHYSIOLOGICAL SENSORS DEVELOPED IN ADAS&ME TO COLLECT HEART-RELATED DRIVER DATA UNOBTRUSIVELY IN THE

CAR. LEFT: SENSOR SEAT, RIGHT: RADAR STEERING WHEEL. ...................................................................................... 24 FIGURE 14 ITERATIVE HMI TESTING OVERVIEW FOR ALL USE CASES. ..................................................................................... 25 FIGURE 15 IMPRESSIONS FROM THE ITERATIVE AND USER-CENTRED TESTING OF ADAS&ME HMI. ........................................... 26 FIGURE 16 ADAS&ME “STATECONS” REPRESENTING THE DRIVER STATES. ........................................................................... 26 FIGURE 17 HMI ELEMENTS ALLOCATED TO THE DEMONSTRATOR VEHICLES AND DRIVER STATES. ................................................ 27 FIGURE 18 UC A – HMI ELEMENTS FOR THE TRUCK. ........................................................................................................ 28 FIGURE 19 UC B – HMI ELEMENTS FOR THE ELECTRIC CAR. ............................................................................................... 28 FIGURE 20 UC C/D HMI ELEMENTS FOR THE COMBUSTION ENGINE CAR. ............................................................................. 28 FIGURE 21 UC E/F - HMI ELEMENTS FOR THE PTW. ....................................................................................................... 29 FIGURE 22 UC G - HMI ELEMENTS FOR THE CITY BUS. ...................................................................................................... 29 FIGURE 23 HIGH-LEVEL ARCHITECTURE OF THE PERSONALIZED HMI. .................................................................................... 31 FIGURE 24 DECISION SUPPORT SYSTEM IN THE ARCHITECTURE OF THE COMBUSTION ENGINE CAR. ............................................. 31 FIGURE 25 DECISION SUPPORT SYSTEM IMPLEMENTED INTO THE COMBUSTION ENGINE CAR. .................................................... 32 FIGURE 26: PHASES OF ADAS&ME TECHNICAL VERIFICATION ........................................................................................... 33 FIGURE 27: SCREENSHOTS FROM INTEGRATION PLUGFESTS. ............................................................................................... 34 FIGURE 28: ADAS&ME UC C&D AND UC E&F HMI DEMONSTRATIONS DURING THE 2ND EUCAD CONFERENCE ...................... 34 FIGURE 29: ADAS&ME DEMONSTRATORS DURING THE PROJECT FINAL EVENT. ................................................................... 35 FIGURE 30 OVERVIEW OF 3-PHASE TESTING PROCESS. ....................................................................................................... 36 FIGURE 31 INTERIOR OF SCANIA TEST TRUCK (LEFT) STATIONARY DEMONSTRATION SYSTEM (RIGHT). .......................................... 38 FIGURE 32 ADAS TEST TRACK AT IDIADA ...................................................................................................................... 38 FIGURE 33 OPEN ROAD TEST ROUTE USED FOR UC B. ....................................................................................................... 39 FIGURE 34 PARTICIPANT DRIVING UC B DEMONSTRATOR VEHICLE ON THE TEST ROUTE WITH HMI ACTIVE. ................................. 39 FIGURE 35 USE CASE C DEMONSTRATOR VEHICLE WITH INTEGRATED ADAS&ME COMPONENTS. ............................................. 40 FIGURE 36 GENERAL ROAD TEST ROUTE USED FOR UC E PHYSICAL FATIGUE INDUCEMENT. ....................................................... 40 FIGURE 37 PARTICIPANT WITH DUCATI MULTISTRADA WITH INTEGRATED ADAS&ME COMPONENTS. ....................................... 41 FIGURE 38 VTI BUS SIMULATOR USED FOR THE USE CASE G EVALUATION. ............................................................................ 41 FIGURE 39 PARTICIPANT DRIVING THE VTI BUS SIMULATOR IN USE CASE G. .......................................................................... 42 FIGURE 40 SUMMARY OF UC A ARCHITECTURE................................................................................................................ 45 FIGURE 41 PHOTO OF STAGE I (LEFT) AND STAGE II (RIGHT) DATA COLLECTION. ..................................................................... 45 FIGURE 42 HMI STRATEGY OVERVIEW TABLE. ................................................................................................................. 46 FIGURE 43 GRAPHICAL REPRESENTATION OF HMI ELEMENTS (LEFT PANEL) AND PHOTO FROM SIMULATOR TESTING (RIGHT). .......... 46 FIGURE 44 THE UC C/D CAR. ...................................................................................................................................... 52 FIGURE 45 RIDER PROTECTIVE GEAR WITH INTEGRATED SENSORS. ........................................................................................ 54 FIGURE 46 USE CASE G SCENARIO WITH HMI FOR AN AUTOMATED DOCKING FUNCTIONALITY INTEGRATED ................................. 56 FIGURE 47 IMPACT ASSESSMENT AND ITS RELATIONS TO OTHER PARTS OF THE WORK IN ADAS&ME ......................................... 58 FIGURE 48 SAFETY IMPACT ASSESSMENT PROCEDURE ........................................................................................................ 59 FIGURE 49 ADAS&ME PROJECT YOUTUBE CHANNEL. ...................................................................................................... 63 FIGURE 50 PARTICIPANTS AT THE ADAS&ME FINAL EVENT DECEMBER 2019. ..................................................................... 64



Index of Tables TABLE 1. THE 11 OBJECTIVES OF ADAS&ME.................................................................................................................. 13 TABLE 2 DIFFERENT TYPES OF RISKS IDENTIFIED IN ADAS&ME ........................................................................................... 19 TABLE 3 AUTOMATED FUNCTIONS IN THE HMI STRATEGY PER USE CASE................................................................................ 30 TABLE 4 EXPLOITABLE RESULTS OF ADAS&ME ............................................................................................................... 61



Glossary

AD AUTOMATED/ AUTONOMOUS DRIVING

AV AUTOMATED/ AUTONOMOUS VEHICLE

ADAS ADVANCED DRIVER ASSISTANCE SYSTEM

API APPLICATION PROGRAM INTERFACE

ARAS ADVANCED RIDER ASSISTANCE SYSTEM

CPM COOPERATIVE PERCEPTION MESSAGE

DSS DECISION SUPPORT SYSTEM

DOA DESCRIPTION OF ACTION

EEG ELECTROENCEPHALOGRAM

FQMR FINANCIAL QUARTERLY MONITORING REPORT

HD HIGH DEFINITION

CAD CONNECTED & AUTOMATED DRIVING

DIM DRIVER IDENTIFICATION MODULE

ECG ELECTROCARDIOGRAM

FMEA FAILURE MODE AND EFFECTS ANALYSIS

GDPR GENERAL DATA PROTECTION REGULATION

GSA GENERAL SYSTEM ARCHITECTURE

GSR GALVANIC SKIN RESPONSE

GUI GRAPHICAL USER INTERFACE

HD MAP HIGH DEFINITION MAP

HMI HUMAN MACHINE INTERACTION

HRV HEART RATE VARIABILITY

IPR INTELLECTUAL PROPERTY RIGHTS

ITS INTELLIGENT TRANSPORTATION SYSTEM

KPI KEY PERFORMANCE INDICATOR

KSS KAROLINSKA SLEEPINESS SCALE

MQTT MQ TELEMETRY TRANSPORT (A MACHINE-TO-MACHINE (M2M)/"INTERNET OF

THINGS" CONNECTIVITY PROTOCOL)

NDA NON-DISCLOSURE AGREEMENT

PTW POWERED TWO-WHEELER

QMR QUARTERLY MONITORING REPORT

UC USE CASE

VR VIRTUAL REALITY

WP WORK PACKAGE

LED LIGHT-EMITTING DIODE



MCA MULTI-CRITERIA ANALYSIS

PHP HYPERTEXT PRE-PROCESSOR

PPE PERSONAL PROTECTIVE EQUIPMENT

PS PERSONALISATION SYSTEM

PPS PRE-PILOT STUDIES

REST REPRESENTATIONAL STATE TRANSFER

SOA STATE OF THE ART

SQL STRUCTURED QUERY LANGUAGE

XML EXTENSIBLE MARK-UP LANGUAGE



Executive Summary

ADAS&ME (“Adaptive ADAS to support incapacitated drivers & Mitigate Effectively risks

through tailor made HMI under automation”) aimed to develop Advanced Driver Assistance

Systems that incorporate driver/rider state, situational/environmental context, and adaptive

interaction to automatically transfer control between vehicle and driver/rider and thus ensure

safer and more efficient road usage for all vehicle types (conventional and electric car, truck,

bus, motorcycle). To achieve this, a holistic approach has been applied which considers

automated driving along with information on driver/rider state and the environment.

This report is the Final report of ADAS&ME with the aim to provide the reader with an

overview of the ADAS&ME project scope, the outline of the work, how it was performed, and

an overview of the results achieved.

Chapter 1 describe the overview of the scope of ADAS&ME and its objectives, chapter 2 focus

on the use cases and the scenarios that based on indicative use cases were updated and for which

the prioritisation of scenarios was made. The use cases were then matched with demonstrators

in which all systems develop had to be integrated. The demonstrators were also used for the

final evaluation. Chapter 3 cover the work process of ADAS&ME including the steps system

architecture and specification, environmental sensing, driver/rider state assessment, HMI action

and transition, integration and final evaluations. Chapter 4 then gives an overview of the

approach and achievements for each use case. Chapter 5 describe the impact assessment.

Chapter 6 describe the exploitation and chapter 7 gives a brief overview of the disseminations

of ADAS&ME. Chapter 8 is about the conclusions in relation to the objectives of ADAS&ME

and finally chapter 9 provide a list of contact persons per work package and use cases.

ADAS&ME included 11 objectives with the following overall achievements: ▪ The development of robust detection/prediction algorithms for driver/rider state enabling

personalisation of individual driver’s physiology and driving behaviour. This has been

achieved with good accuracy for mostly all driver state detections.

▪ The development of multi-modal, user oriented and adaptive information, warning, actuation

and handover strategies, based on current and predicted driver/rider state, criticality of

scenario and its environmental context. This has been achieved for all seven use cases.

▪ Integration of the developed algorithms, sensing technologies, supportive technologies

(automation, V2X) and HMI algorithms/components into driver/rider state monitoring

systems. This has been achieved in all five demonstrators.

▪ The development of personalised driver/rider behaviour profiles, considering inter-individual

differences. This has been achieved in the majority of the use cases.

▪ A design of HMI concepts, prototypes and guidelines for automated functions that take

driver/rider state into account, for possible implementation in future driver/rider state-adapted

automated systems. This has been achieved for all seven use cases.

▪ Instrumentation of evaluation/demonstration tools (simulators and vehicles) for evaluation of

the developed systems in different environments, as specified by the ADAS&ME use cases

and its adaptive architecture. This has been achieved for all seven use cases expect for those

that due to high safety risk with driver state inducement had to be simulated to evaluate the

HMI components.

▪ Adaptation of existing EuroNCAP test protocols from non-automated to automated driving

modes. The experience in the project has been used as input for future test protocols.



▪ Perform targeted tests for the selection of HMI elements that optimally support each Use Case.

This has been done for all use cases as a part of an agile development including iterative

testing.

▪ Ethical and legal considerations regarding the experiments carried out on driver/rider state

monitoring, the evaluation of the developed systems, as well as when and how automated

systems should interact with the driver/rider. This has been achieved and all test with humans

have been ethical approved and in line with the legalisation.

▪ An evaluation of the developed systems and use cases with a wide pool of drivers/riders under

simulated, controlled and real road conditions and for different driver/rider states and

automation use cases/levels. This has been achieved with almost 200 drivers/riders involved in

data collection for algorithm and evaluations.

▪ A holistic impact assessment of automation opportunities to enhance safety by supporting the

impaired driver/rider, as well as of handover transitions optimisation, taking the driver/ rider

estimated state into account. This has been achieved for all seven use cases.



1 The scope of ADAS&ME

ADAS&ME (“Adaptive ADAS to support incapacitated drivers & Mitigate Effectively risks

through tailor made HMI under automation”) aimed to develop Advanced Driver Assistance

Systems that incorporate driver/rider state, situational/environmental context, and adaptive

interaction to automatically transfer control between vehicle and driver/rider and thus ensure

safer and more efficient road usage for all vehicle types (conventional and electric car, truck,

bus, motorcycle). To achieve this, a holistic approach has been applied which considers

automated driving along with information on driver/rider state and the environment.

The holistic approach of ADAS&ME considers automated driving/riding along with

information on driver/rider state, to develop optimized HMI and support strategies, where

automated and partly automated driving/riding is seen as both an influencing factor and a tool

to affect driver/rider state, see Figure 1.

In the ADAS&ME decision and strategy loop the driver state is influenced by different factors,

including individual factors (e.g. sleepiness, inattention, stress, anxiety, physical fatigue etc.),

external factors (e.g. weather conditions, traffic density/interactions with others) and the

driver/rider vehicle interaction mode (automated, semi-automated or manual). Sometimes

driving/riding performance, like speed, lane keeping, or headway will be influenced by the

driver/rider state, as well as by the driver/rider vehicle interaction mode and by external factors.

Based on the output from driver/rider state monitoring and current driving performance, a

decision about the need of mitigation or countermeasures is made and used as input to the

strategy for intervention or driver vehicle interaction change. The strategy uses information,

warning, actuation and handover strategies, depending on the need, as well as different HMI

modalities (haptic/tactile, auditory and visual).

Figure 1 The decision and strategy loop for ADAS&ME



The work in ADAS&ME is built around 10 Work Packages (WP), see Figure 2. In addition,

use case teams have been working in parallel. The starting point for all work is a generic

approach that humans with their pros and cons benefit and suffer from the same things

regardless of what type of vehicle they are driving. This means that generic work on system

architecture, specifications and verifications are used, but also that driver/rider state detection

and HMI strategies are based on the same principles, but of course adjusted per use case and its

demonstrator.

Figure 2 ADAS&ME project outline.



1.1 Objectives

1.1.1 Objectives of ADAS&ME

ADAS&ME encompasses 11 key objectives (all achieved), see Table 1.

Table 1. The 11 objectives of ADAS&ME

Objectives

Development of robust detection/prediction algorithms for driver/rider state monitoring

of fatigue/drowsiness, stress, inattention/distraction and impairing emotions, employing

14 existing and novel sensing and speech technologies, thereby enabling ADAS&ME to

be personalised for individual driver’s physiology and driving behaviour.

Development of multi-modal, user oriented and adaptive information, warning, actuation

and handover strategies, based on current and predicted driver/rider state, criticality of

scenario and its environmental context.

Integration of the developed algorithms, sensing technologies, supportive technologies

(automation, V2X) and HMI algorithms/components into driver/rider state monitoring

systems.

Development of personalised driver/rider behaviour profiles, considering inter-

individual differences.

Design of HMI concepts, prototypes and guidelines for automated functions that take

driver/rider state into account, for possible implementation in future driver/rider state-

adapted automated systems.

Instrumentation of evaluation/demonstration tools (simulators and vehicles) for

evaluation of the developed systems in different environments, as specified by the

ADAS&ME use cases and its adaptive architecture.

Adaptation of existing EuroNCAP test protocols from non-automated to automated

driving modes.

Performance of targeted tests for the selection of HMI elements that optimally support

each Use Case.

Ethical and legal considerations regarding the experiments carried out on driver/rider

state monitoring, the evaluation of the developed systems, as well as when and how

automated systems should interact with the driver/rider.

Evaluation of the developed systems and use cases with a wide pool of drivers/riders

under simulated, controlled and real road conditions and for different driver/rider states

and automation use cases/levels.

Holistic impact assessment of automation opportunities to enhance safety by supporting

the impaired driver/rider, as well as of handover transitions optimisation, taking the

driver/ rider estimated state into account.

1.1.2 Objective of this report

This report is the Final report of ADAS&ME with the aim to provide the reader with an

overview of the ADAS&ME project scope, the outline of the work, how it was performed, and

an overview of the results achieved.



2 Use cases and priority scenarios – setting the scene

ADAS&ME is built around use cases and scenarios. Already from the beginning of the project

seven indicative use cases was defined. At the beginning of the project, those were further

analysed, and a set of relevant scenarios were developed for each of them. Each use case was

then matched to a specific demonstrator in which all components dealing with environmental

sensing, detection of driver/rider state and HMI develop were integrated. The demonstrators

were also used for the final evaluation phase.

The selection of use cases and the prioritisation of scenarios aimed to set the scene. This

required a work to identify the most important issues and knowledge gaps related to driver state

and automated driving, to understand the users’ needs (driver/riders), see Figure 3. The work

started with an SoA and benchmarking activity (Touliou, Maglavera, & Britsas, 2017).

Templates were prepared and shared with partners to gather information about available

solutions for driver/rider monitoring. A template was also prepared and filled out to align

actions to specific priorities.

Figure 3 Overall methodology for definition and selection of final use cases of

ADAS&ME.

End users’ perspectives and wishes were gathered through an open web-survey, and an analysis

was done based on more than 1000 EU respondents. The survey was built around short stories

for each use case that together with illustrations were used in the web survey to visualize the

concept. The web survey used the SoSci Survey tool (https://www.soscisurvey.de/en/about

retrieved: 2020-02-21). A wide range of stakeholders, identified with the help of input from all

partners, were invited. To complement the survey five focus groups were organised. The aim

was to get a deeper understanding of the users’ views and wishes. In addition, a workshop with

https://www.soscisurvey.de/en/about



participation of more than 25 stakeholder representatives was held with the aim to provide input

to the final selection of the use cases and prioritisation of scenarios per use case, see Figure 4.

Figure 4 ADAS&ME workshop focusing on Use Cases and scenarios prioritisation.

The prioritisations from the different quantitative methodologies were consolidated with help

of a Multi-Criteria Analysis (MCA) based on all stakeholders input and a final prioritisation of

scenarios for each use case was achieved. This work also included an analysis of the potential

impact of the selected use cases and scenarios. In total seven use cases were selected with two

priorities per identified use case (Dukic Willstrand et al., 2017). In the end, seven ADAS&ME

use cases were selected for implementation together with two prioritised scenarios, see Figure

5.

The prioritised scenarios for each use case were:

Use case A: Attentive long-haul trucking (truck) ▪ 1st Priority: Safe Stop

▪ 2nd Priority: Traffic Jam negotiation

Use case B: Electric Vehicle (EV) range anxiety (e-car) ▪ 1st Priority: EV range problem appears due to traffic jam

▪ 2nd Priority: Driver trusts the system and follows indications to power charging station

Use case C: Drive state based smooth and safe automation transitions (conventional car) ▪ 1st Priority: Unsuccessful handover due to non-reacting driver and safe stop

▪ 2nd Priority: Controlled (automation initiated) handover transitions taking the driver state into

account for the interaction design into

Use case D: Non-reacting driver emergency manoeuvre (conventional car) ▪ 1st Priority: Unsuccessful handover and takeover transitions

▪ 2nd Priority: Controlled (automation initiated) takeover transitions based on driver state

Use case E: Long range attentive touring with motorbike (motorbike) ▪ 1st Priority: Assistance during long range touring in case of tiredness

▪ 2nd Priority: Assistance during long range touring in case of inattention

Use case F: Rider Faint (motorbike) ▪ 1st Priority: Activation of active systems if the rider is fainting

▪ 2nd Priority: Activation of active systems if the rider is going to faint and is ignoring

assistance



Use case G: Passenger pick-up/drop-off automation for buses (city bus) ▪ 1st Priority: System initiated takeover

▪ 2nd Priority: Driver initiated takeover

Figure 5 Selected use cases.

Note: from left to right: UCA Long haul trucking, UCB electrical vehicle range anxiety, UCC

and UCD Conventional car: “driver state based safe and smooth automation transition” and

“none reacting driver emergency manoeuvre”, UCE and UCF “long range attentive touring with

motorbike” and “rider faint”, UCG Passenger pick up/drop off automation for buses.

A theoretical model for driver/rider state was also defined (Dukic Willstrand et al., 2017).

General definitions of the driver/rider states, investigated in ADAS&ME, were elaborated in

collaboration with the developers of the driver/rider state detection algorithms, Figure 6.

Suitable reference/ground truth measures and performance evaluation methods for the

individual states were investigated and research on ground truth generation methods and

suitable experiments for “stress/workload”, “inattention/distraction” and “physiological

impairment” were conducted.

Figure 6 Driver/rider model.

The work with “setting the scene” was performed mainly in Work Package 1 of ADAS&ME.



3 ADAS&ME work process

3.1 System architecture and specifications

Work package 2 focused on ADAS&ME System architecture and specifications. The goal of

this work was to design a flexible architecture to facilitate smooth integration of various

software and hardware components designed during the project across the identified use cases.

This work consisted of three major activities:

1. System Architecture

2. Technical specifications

3. Risk Analysis

Figure 7 shows the framework followed for achieving system specifications. Once the scope of

the use cases was finalized, the work in WP2 started by collecting the requirements for the use

cases.

System Design

ADAS&ME Project Proposal Idea

Survey

State of the Art List of Users User wishes Stakeholder wishes

Needs and Limitations

User needs Stakeholder needs Planned Innovation Limitations Use Cases

System Requirements

Tasks Communication Timing Priorities

Functional Blocks

ADAS&ME System Specifications

Input Data Output Data

Error Handling

Figure 7 Framework for ADAS&ME System Specifications.

Based on the collected requirements, the ADAS&ME General System Architecture was

designed, see Figure 8. This led to the visualization of key elements (e.g. subsystems, modules

and components) that are required for a successful execution of the use cases in the final

demonstrators. These general key elements were modified to suit the needs of each

demonstrator. System architecture was specified for each use case at different levels e.g.

functional architecture, communication architecture and the physical architecture. In order to

facilitate a flexible communication across various heterogeneous elements, ADAS&ME GSA

presented an innovative way by unifying the message data used for communication across

various components.



The technical specification of each key element was identified using a template. The template

described the key elements in a tabular form and consisted of 4 sections, namely:

1. Introduction – containing the information about the manufacturer, input and output

details.

2. Performance section – containing information about the key performance aspects of the

elements e.g. accuracy of measurement, frequency etc. of a sensor.

3. Physical aspects section – containing the information about the physical weight and

dimension of the elements.

4. Environmental aspects section – containing the information about the tolerable

environmental conditions e.g. temperature for the respective element to function

correctly.

Environmental Situation Awareness Subsystem (ESAS)

Algorithms for Environmental Situation Determination

ADAS&ME Core (ADAS&ME C)

Personalization Module (PM)Decision Support Module (DSM)

HMI Controller Module (HMI CM)

Vehicle Automation Subsystem (VAS)

Vehicle Automation Module (VAM)

Soft Sensor data

Driver monitoring Data

Environmental/Vehicle Data

Digital Infrastructure

DataHMI Data

Interface Module (IM)

Sensor Subsystem (SS)

Driver State Monitoring Subsystem (DSMS)

Algorithms for Driver State Determination

Figure 8 ADAS&ME General System Architecture.

A risk assessment was carried out using Failure Mode and Effects Analysis

(FMEA)methodology to identify the design flaws and any issues that could occur during the

integration and execution of the use cases. Risks were identified in 4 categories, i.e. technical,

behavioural, legal and organizational. Table 2 quantitatively describes the various risks

identified during the project.



Table 2 Different types of risks identified in ADAS&ME

3.2 Environmental sensing

Within the scope of ADAS&ME, Advanced Driver Assistance Systems (ADAS) were

developed in order to ensure safer and more efficient road usage. These ADAS incorporate

driver/rider state, situational/environmental context, and adaptive interaction to automatically

transfer control between vehicle and driver/rider. The goal with the situational/environmental

was to derive an indicator about the stressfulness or dangerousness of the surrounding. The

indicator is supposed to serve a following decision-making module in deciding whether the

current environmental situation is advantageous for manual or autonomous driving and whether

a transfer of control would be beneficial.

Figure 9 shows all modules that have been developed. The Sensor Data Fusion Module

processes sensor data to derive the own vehicle state. The Communication Module consists of

three sub-modules: The Digital Infrastructure looks further along the road ahead to understand

what to expect. V2X Data Sharing provides means to communicate with the close vicinity via

V2X communication. The information received via V2X is fused by the V2X Data Integration.

The Environmental State Determination Module combines in a first step the own vehicle state

and environmental context information in order to form a Local Dynamic Map (LDM). Step 2

estimates the behaviour of objects present in the LDM over a specified time and classifies the

current environment situation as Normal, Critical or Dangerous.



Figure 9 Data flow between modules included for environmental sensing.

The Sensor Data Fusion Module determines the vehicle’s current location in a reliable and

probabilistic way. The innovation compared to state-of-the-art is that the confidence of the

location estimation itself is estimated. For next-generation automated driving functionality it is

important to not only know the position itself, but also the confidence.

The Digital Infrastructure determines the driver task intensity based on static and transient

dynamic data. More precisely is the driving task intensity derived from traffic flow data,

location-based weather data and High Definition (HD) map data such as number of lanes, lane

transitions and lane geometry (e.g. curvature). The driving task intensity quantifies the

stressfulness and dangerousness of the road ahead ignoring the highly dynamic data.

The V2X Data Sharing covers all (vehicle external) communication aspects. It enables direct

vehicle to vehicle and vehicle to infrastructure communication according to ETSI standards

(https://www.etsi.org/ Retrieved:2020-02-21). As part of the ADAS&ME activities, new

messages for collaborative perception and cooperative manoeuvres were developed and

discussed in ETSI.

The V2X Data Integration fuses information that is received by the V2X Data Sharing and the

measurements from the on-board sensors. This includes a new proposal for the temporal and

spatial alignment, innovative CPM dissemination strategies and concepts for more robust

perception.

The task of the Environmental State Determination Module is to assess the current

environmental situation around the own vehicle. Data describing the situational/environmental

context is provided by the aforementioned modules. Distinct sources such as maps, landmarks,

weather information, and vehicles or infrastructure that are equipped with exteroceptive and

proprioceptive sensors have been considered. In a first step the data is gathered by the Local

Dynamic Map (LDM), which stores the information depending on the persistency of the

information. As shown in Figure 10 the LDM had four layers with different types of data.

https://www.etsi.org/



Figure 10 Various Layers of Local Dynamic Map.

The LDM enables awareness of the current situation around the automated vehicle From this

point, and in order to assess the situation, a Decision Network (DN) was designed. It considers

estimated values from the sensors along with their confidence values and computes the utility

value for each decision alternative. The principle of maximum expected utility is applied i.e.,

the decision alternatives of the environment with highest utility will be chosen as output.

The work on “environmental sensing” were performed mainly in Work Package 3 of

ADAS&ME.

3.3 Driver/rider state monitoring

The objective of the driver/rider state monitoring was to develop and demonstrate algorithms

and sensors for driver/rider state detection. Within the context of ADAS&ME these solutions

should be able to detect and measure undesirable or unusual driver conditions to enhance safety

of drivers/riders. The drivers were supported during handover transitions thanks to the

developed adaptive HMI systems considering the actual driver state.

After receiving the input from the stakeholder and end user surveys, which resulted in an update

of the use case scenarios, an updated list of driver states per Use Case was generated. Within

this work, indicators for each of the specified driver states were identified. Furthermore, a

thoroughly defined theoretical background for each driver state was established which is

documented in deliverable 4.2 (Hennes & Mathissen, 2020) (confidential). The work was

supported by the driver/rider state model (Dukic Willstrand et al., 2017). Additional

contributions were coming from expert background knowledge, literature review and additional

acquisition of data from testing both in lab and in the field. The driver state indicators were then

used for the development and evaluation of the actual driver state detection algorithms. To

provide input data to the driver state algorithms specific sensors were required. A state-of-the-

art for sensors were developed and an identification of high-quality data for the driver state



algorithms were made. During the project, different sensors were tested and either rejected or

verified to be applicable in the specific use cases which resulted in a final set of sensors for

each use case. The sensors and algorithms were later integrated into the demonstrators and

evaluated with all other ADAS&ME components as a complete system.

An overview of all selected driver/rider state sensors and algorithms is given in Figure 11.

Figure 11 Overview of all Driver states and Sensors applied for each Use Case.

A total of 7 different sensor systems provide data to algorithms detecting 6 different driver

states: Distraction, Emotions, Rest, Physical Fatigue, Sleepiness and Stress. These were

integrated in the five demonstrators covering all seven use cases. Main innovations and

limitations for each of the driver/rider state algorithms are several.

3.3.1 Physical Fatigue

For the first time a model for predicting thermal fatigue of riders was developed. This was

enabled by a unique set of data collected during the simulation tests on rider thermal stress.

Since there is almost no background research on rider state detection a lot of innovative work

was done within the rider use case (UC E/F) in ADAS&ME.

3.3.2 Rest

For the first time a driver state algorithm estimating drivers’ rest was developed. A main

challenge was the missing ground truth of resting which posed a challenge for the developers.

A sophisticated three-process model of alertness was used to estimate how much driving time

has been gained by resting.



3.3.3 Sleepiness

This driver state has been very well investigated in the last decades. Within the project, a new

representation of input features was introduced and specifically designed to improve

performance when using neural networks.

3.3.4 Stress

Three very different stress inducement strategies were hypothesized and finally tested to select

the best one for final evaluation. It was shown that personalization of the algorithm shows

improvement over the generalized algorithmic approach (ca. +10% F1-Score).

3.3.5 Distraction

The same algorithmic approach was used for different UCs which was further enhanced by

tailoring to the specific features of the demonstrator vehicles. For the truck use case (UC A) the

importance of mirrors was acknowledged while for the bus use case (UC G) a novel situation-

dependent multi-buffer approach was developed. For the rider use case (UC E/F), the first ever

distraction algorithm for riders was developed.

3.3.6 Emotion

Three different emotion algorithms were developed (audio, video and physiological based). The

combination of different modalities allowed continuous tracking of the emotional state. The

algorithms allowed classification of low-expressive emotions of high naturalness.

Generally, different driver/rider states share certain commonalities, as:

• It was often difficult to collect relevant data due to safety or ethical reasons.

• In many cases there was a high inter-individual variability which would require large

data sets to be addressed properly.

• For most driver states, the problem of finding a proper ground truth which is less

intrusive than questionnaires during driving persists.

An overview of the performance metrics of each algorithm is shown in Figure 11.



Figure 12 Performance results of driver states algorithms.

Note: Data points refer to mean values and error bars to standard deviation while coloured

background indicates the range. The number of prediction levels (classes) per algorithm is

indicated in parenthesis.

There were two different classes of sensors implemented in the ADAS&ME project.

First, off-the-shelf sensors were chosen based on initial requirements and evaluated for their

actual performance by lab testing. Secondly, new contactless sensors for heart-related

measurements were developed and designed to be directly integrated into the accessories of

the car, either in the driver’s seat or in the steering wheel of the vehicle as shown in Figure 13.

Figure 13 Physiological sensors developed in ADAS&ME to collect heart-related driver

data unobtrusively in the car. Left: Sensor seat, Right: Radar steering wheel.

Furthermore, a novel wearable sensor system designed for riders was developed and directly

integrated into rider’s equipment. While the contactless sensors were working reasonably well

under laboratory conditions, their performance in a moving vehicle was significantly degraded.

Overall, unobtrusive sensors for capturing physiological data remain a major challenge for

driver state detection and need further improvement. Within ADAS&ME, reference sensors

were used as a fall-back solution for UCs that planned to rely on unobtrusive sensors.

In addition to algorithms and sensors, a Personalisation system (PS) was developed consisting



of two distinctive modules. The “Driver Identification Module (DIM)” allows identification of

drivers/riders and the “Personalization module (PM)” gathers, stores and provides relevant

information. While the PM follows the same approach across all UCs the DIM is dependent on

the UC, e.g. the driver/rider may be identified by his/her face (via the SEP camera) or personal

item (rider suit, mobile phone, etc.). The PS was designed as a cross-WP system, e.g. the

personalization of HMI modalities is based on this system. All algorithms, sensors and the

personalization system communicate via MQTT which was defined as a message protocol.

Other work packages adapted this protocol afterwards which makes it the common exchange

platform for messages. Furthermore, all algorithms and sensors follow the same message format

across all platforms.

The work on “Driver/Riders state detection” was performed mainly in work package 4 of

ADAS&ME.

3.4 HMI Actions and Transitions

The aim was to develop HMI elements and strategies for adaptive transitions, a work done in

WP5. HMI elements, adaptive HMI actions and adaptive transitions were developed, evaluated

and demonstrated in all envisioned demonstrators, namely truck, electric car, combustion

engine car, PTW and city bus.

3.4.1 Iterative HMI development and testing

Due to the diversity of the demonstrator vehicles and use cases the approach was to develop

different HMI elements, modules and strategies, following a common HMI framework and

design recommendations. All developments also followed a common user-centred development

process with several development iterations and intermittent user tests. Figure 14 shows the

iterations carried out in the HMI development for each use case and the equipment used in the

laboratories of the responsible partners.

Figure 14 Iterative HMI testing overview for all use cases.



Figure 15 shows impressions from the iterative testing of the HMI developed or adapted in

ADAS&ME. In total, feedback and user experiences of 237 potential users of the technology

have been included in the development process.

Figure 15 Impressions from the iterative and user-centred testing of ADAS&ME HMI.

The achievements are different HMI elements, adapted to the project needs or specifically

developed within the project, tailored HMI strategies for each use case and multiple driver

states, the user-centred iterative development, and the final evaluation and demonstration in the

demonstrator vehicles. Common HMI elements have been used where meaningful, especially

related to the driver states and the levels of automation which were common among the use

cases. As a prominent example, “statecons” were developed and used in all use cases in order

to represent the driver states, see Figure 16.

Figure 16 ADAS&ME “Statecons” representing the driver states.

3.4.2 HMI Framework



The different driver states required very different HMI strategies. For example, a moderately

sleepy driver requires interaction and involvement in the driving task, while a very sleepy driver

requires a rest and if possible, Level 4 automation (https://www.sae.org/). A stressed driver

requires little disturbance and a nudging reminder to use automation, while a distracted driver

requires an earlier prompt to take control when using automation. The HMI strategies are

represented in a generic framework possible to use to be adapted to define the strategies for all

target vehicles. The framework enables defining the vehicle–system–user

interactions, considering the driver/rider state, personalization, the environmental context and

all HMI elements (Diederichs et al., 2018).

Besides the different HMI strategies also the different HMI elements can be used for different

driver states. Figure 17 shows the allocation of HMI elements to the use cases and its

demonstrator, considering also the driver/rider states.

Figure 17 HMI elements allocated to the demonstrator vehicles and driver states.

3.4.3 HMI Elements and Modalities

The HMI elements and modalities for each demonstrator are represented in Figure 18 - Figure

22.



Figure 18 UC A – HMI elements for the truck.

Figure 19 UC B – HMI elements for the electric car.

Figure 20 UC C/D HMI elements for the combustion engine car.



Figure 21 UC E/F - HMI elements for the PTW.

Figure 22 UC G - HMI elements for the city bus.

3.4.4 Automation

In ADAS&ME the automated functions were understood as an important part of the HMI

strategy as they provide important kinaesthetic information to the drivers and passengers. Thus,

the actuation of automation needs to be well explained and seamlessly integrated by the HMI

elements and strategies. Table 3 displays the automated functions that were realized for the final

demonstration.



Table 3 Automated functions in the HMI strategy per use case.

Use Case Automated functions in the HMI strategy:

Driver state related automation behaviour

Use Case A

(Demonstrator: Truck)

Safe Stop

Use Case B

(Demonstrator: Electric

vehicle)

Transition to Automation at critical battery level,

Automated driving to next car sharing, Automated

parking

Use Case C

(Demonstrator:

Passenger Vehicle)

Adaptation regarding timing and information strategy in

transitions of control (SAE3-SAE0;

https://www.sae.org/)

Use Case D

(Demonstrator:

Passenger Vehicle)

Timing of Minimum Risk manoeuvre (earlier in critical

driver states)

Use Case E/F

(Demonstrator:

Motorbike)

Adaptation regarding HMI warning with progressive

severity, including torque limitation and capsize control

Use Case G

(Demonstrator: Bus

simulator)

Automated approaching of bus stop, automated stop,

automated departure from bus stop, Safe Stop

3.4.5 HMI personalisation

The adaptation of the HMI also included personalisation of it. For the realization of a

personalized HMI a rule engine with personalisation and adaptation rules was developed. It

enables working in distributed environments with a personalised HMI Controller acting as a

server and HMI modalities and GUI applications acting as clients. Figure 23 shows the high-

level architecture of the personalized HMI.



Figure 23 High-level architecture of the personalized HMI.

3.4.6 Decision and Support System

A Decision Support System (DSS) was developed for the decision of which interaction strategy

should be selected in case multiple driver states appear at the same time. Individual software

architectures of the different demonstrator vehicles required individualized solutions, adapted

to existing architectures and OEM specific procedures in the vehicles. Figure 24 show the DSS

implementation in the combustion engine car of UC C/D.

Figure 24 Decision Support System in the architecture of the combustion engine car.

Figure 25 show the implementation of the DSS into the combustion engine car in an example.



Figure 25 Decision Support System implemented into the combustion engine car.

3.4.7 Main Innovations

Main innovations of WP5 are related to the development of the first application of new HMI

elements, particularly:

▪ Invention of “Statecons”.

▪ Establishment of LED strips for automation and transition feedback.

▪ Inclusion of automated functions and transitions into HMI strategy.

▪ Personalization framework and adaptation based on driver states.

▪ Consistent HMI strategies for transitions into and out of automation in highly

demanding situations with compromised drivers.

▪ PTW capsize stabilization, rider HMI integrated in gloves and helmet and information

to other road users that impaired rider state is detected.

• First time application of automated functions and driver state adaptive transitions for

Truck, Car, PTW and Bus.

3.4.8 Outlook

The ADAS&ME HMI achievements are highly relevant for further research and series

production vehicles. From a research point of view a driver state adaptive HMI should be

applied also in future projects covering more type use cases, since it will go to series production

timely as from 2022 the EU safety regulation requires it. A main result for future research and

Decision Tree – UC C HandoverH

MI S

trat

egy

Fru

stra

tion

Str

ess

Sle

epin

ess

Dis

trac

tion

1 : Alert

2: No 1 :Yes

3 : Very Drowsy

CoE_S < CoE_D

2 : Drowsy

Sleepiness State

1 : Alert

Sleepiness State

3 : Very Drowsy

Sleepiness State

2 : Drowsy

0:Normal 1:Elevated

3 : Very Drowsy

CoE_S ≥ CoE_D

3 : Very Drowsy

0:Normal 1:Elevated 1:Elevated 0:Normal 0:Normal 1:Elevated 1:Elevated 0:Normal 1:Elevated 0:Normal 0:Normal 1:Elevated

YesNo YesNo No Yes No Yes No Yes YesNo No Yes YesNo

HMI Strategy for distraction HMI Strategy for sleepiness

No Yes No Yes No Yes YesNo NoYes No Yes

HMI Strategy for Stress

* **

* **

HMI Strategy for FrustrationDefault HMI Strategy (No State)

** : CoE_Str<CoE_Frust* : CoE_Str≥CoE_Frust



development is that different driver states and different vehicles require development of

different HMI strategies and HMI elements. For the transition automation to driver more user

tests, legal considerations and ethical debates are needed to discuss if and how drivers shall be

allowed to get control back when their driver state is compromised. For the transition driver to

automation the application of driver state adaptive HMI, supporting the transition from L0 to

L4, is very innovative with a high potential for enhanced safety. It More effort in development

and research as well as in public discussions is required to form an understanding of the need

from the public society.

The relevant work of “HMI action and transitions” was performed mainly in Work package 5

of ADAS&ME.

3.5 Integration in Demonstrators

The architectural framework was used as the starting point to achieve a successful integration

of the developed system into all demonstrators. The framework covers all individual

ADAS&ME components, encompassing the HMI, sensors, algorithms and environmental

monitoring functions. The starting point for the integration was the work done on system

architecture and specifications. The integration work included the task of defining protocols to

be used during the technical verifications of sensors, algorithms, modules and use cases towards

final readiness testing before the launch of the evaluation work performed in the evaluation

work package, see Figure 26. To support the integration framework a platform was developed

on which all subsystem providers (mainly hardware) and those responsible for a demonstrator

added the corresponding information. The tool was used to support the integration and

evaluation phases and to better organise the transfer of components between integration and

evaluation sites as well as to get information on spared components in case of a broken or

problematic component.

Figure 26: Phases of ADAS&ME Technical Verification

Within the framework of the use cases integration a series of ”Plug-fests” (technical integration

meetings) was introduced in order to better monitor but also in the best way facilitate the

cooperation between the use case integration teams, see Figure 27. The final integration at the

evaluation sites took place in order to facilitate the requirements of each use case evaluation.

Phase 1 : Design of ADAS&ME Technical

Verification Plans

Phase 2 : Implementation of ADAS&ME

Technical Verification Plans

Phase 3 : ADAS&ME Technical Verification Results and Analysis



Figure 27: Screenshots from Integration Plugfests.

Furthermore the work included the preparation of Use Case demonstrations in dedicated

external events, such as the UCA demonstration at the 2018 2018 EUCAR Reception and

Conference which took place on 6-7 November in Brussels, using the SCANIA truck VR

simulator and the UC C&D and UC E&F adaptive HMI demonstrations during the 2nd EUCAD

Conference held on April 2-3, 2019 in Brussels, see Figure 28.

Figure 28: ADAS&ME UC C&D and UC E&F HMI demonstrations during the 2nd

EUCAD Conference

In addition the preparation of the demonstrations for the Project Final Event that was held on

December 3rd, 2019 at IDIADA premises was also realised for all the Use Cases of the project,

see Figure 29.



Figure 29: ADAS&ME Demonstrators during the Project Final Event.

As an output of this work, a list of recommendations for the future was devised as follows:

• Revisit physical and communication architecture before integration starts.

• Continuous risk assessment at all key phases of the project and at all levels;

technical, operational, behavioural and legal.

• Perform incremental integration; components to systems -> technical verification;

systems to systems -> technical verification and so on to avoid solving an error at a

late and complex process level.

• Stepwise and cross-use case integration approach (monitoring tool and plugfests) is

a pre-requisite for a complex integration such as the ADAS&ME project required.

• Close collaboration/communication between integration teams at Use Case level.

• Enough resources at each partner involved in the integration.

• Proactive reserve of funds (limited) for any support to purchase integration

consumables or software if required.

The work on “Integration” was performed mainly in Work package 6 of ADAS&ME.

3.6 Evaluations

Work package 7 concerned the data collections and evaluation work done for all the seven use

cases. The work involved the creation of the evaluation framework, rre-pilot data collections,

the final evaluations of all use cases carried out. and the consolidation of the results of the use

case evaluations.

3.6.1 Evaluation framework

The Evaluation framework developed in ADAS&ME defines a methodology for evaluating the

system functions based on two principle questions concerning (1) whether the system does what



it is supposed to do within its defined specification and (2) how the users experience the system

regarding their responses and behaviour observed in conjunction with its use (Cocron et al.,

2019). The activity was based upon the development of appropriate data collection protocols

and tools which are applicable to all the use cases with the intention to make it possible to

consolidate the findings from all use cases. The work included detailed test protocol, with

information needed to conduct a successful evaluation. Amongst this were the specific data

collection methods and tools for objective and subjective measures. These included items such

as questionnaires and objective data points from physiological sensors and vehicle performance

measures. Also defined was the methodology and process for driver state inducements required

for the testing of use case demonstrators. Subjective measures of driver state were also specified

to quantify the success / failure of the inducement. By extension fall-back strategies were also

investigated, with these included in the evaluation procedure to account for eventualities where

the inducement procedure. These were then integrated into the general framework defined, but

with individual evaluation strategies adapted to fulfil the requirements of each individual use

case.

3.6.2 Pre-pilot data collections

The pre-pilot data collections were a series of tests with the intention to collect empirical data

regarding the components of each use case required to support the developers of driver/rider

detection systems, but also those working on environmental issues. The testing methodology

comprised a 3-phase process, see Figure 30 which involved test planning as part of the initial

phase. The work was aimed to support the data collection phase 2 and subsequent sharing

between the relevant work package partners.

Figure 30 Overview of 3-phase testing process.

The pre-pilot data collections were conducted to support the algorithm developers, the sensor

developers and the use case teams. All use case teams performed at least one pre-pilot data

collection, but most of them at least two different tests (Anund et al., 2020). The data collections

were carried out across two iterative stages. The first iterative testing stage involved a series of



experiments conducted across the different use cases. The testing focussed on driver states and

their effects. The second iteration of testing was carried out regarding some early

implementation of ADAS&ME system components for all use cases. The data collection and

the results during the pre-pilot are confidential. Data was however always made available to

partners through the sharing platform.

3.6.3 Final evaluations

The final evaluations of the use cases and the related scenarios covered a series of technology

demonstrator platforms with various sensors, algorithms, HMI, and automated control functions

developed as part of ADAS&ME. The evaluations involved the scoping and planning,

development off the experimental procedure, including technical functionality control of the

ADAS&ME systems as well as their effects on real users focusing on driver/rider state,

behaviour and perception. The realisation of the final evaluations and the consolidated results

are possible to read more about in Deliverable 7.2 (Anund et al., 2020).

To ensure that the developed systems can accurately detect when the driver/rider is not

capacitated to drive, and consequently mitigate these states and avoid dangerous situations

using adapted HMI and automation modes, the overall ADAS&ME final evaluation aims were

to:

1. Evaluate the effectiveness of the systems to recognize the driver’s/rider’s state.

2. Assess the capacity of the HMI to display clear and unambiguous information.

3. Evaluate driver/rider behaviour following a system warning/suggestion.

4. Collect the driver’s/rider’s opinion on the system’s usability.

5. Assess the driver’s/rider’s trust and acceptance levels regarding the ADAS&ME

functions.

In total 198 participants (48 of them were female) have been involved in data collections related

to creation of datasets for the development of algorithms and during the final evaluations.

3.6.4 UC A - Truck

The evaluation of use case A was carried out in a Scania truck fitted with the ADAS&ME

system concept shown in Figure 31. The evaluation procedure was modified due to technical

limitations with the demonstrator vehicle (no automated system, truck not able to be driven

with the system active). This led to objective data being collected primarily for manual driving

with the induced driver state, with the intention being to demonstrate the experience of driving

in such a scenario. The evaluation took place on the ADAS test track at the IDIADA proving

ground, see Figure 32. A stationary ADAS&ME system demonstration was then conducted

with a simulated automation system used to present the system functions. In total 7 truck drivers

participated.



Figure 31 Interior of Scania test truck (left) stationary demonstration system (right).

Figure 32 ADAS test track at IDIADA

3.6.5 UC B – Electrical passenger car

An open road test route, see Figure 33 - Figure 34 was used to induce anxiety, providing a

highly representative real-world test scenario. The vehicle entered the test track areas at

IDIADA for activation of the automation system. There were issues with the anxiety

inducement due to the test conditions. However, the fall-back strategies were effectively used

to trigger the system functions. A total of 14 participants were tested and the system concept

was generally well received, although there were some apparent issues with comprehension of

the HMI. A positive impact on the acceptance of battery-electric vehicles was shown.



Figure 33 Open road test route used for UC B.

Figure 34 Participant driving UC B demonstrator vehicle on the test route with HMI

active.

3.6.6 UC C/D - Conventional passenger car

The evaluation of Use Case C/D was carried out in DLRs test vehicle fitted with the

ADAS&ME system concept, see Figure 35. In total 8 German speaking participants were tested

using a within-subjects design across 2 sessions.



Figure 35 Use Case C demonstrator vehicle with integrated ADAS&ME components.

The results show that the system functioned well in some cases, although there were some

scenarios where the ADAS&ME components were not active as planned. The participants

experienced the system positively overall.

3.6.7 UC E/F - Motorbike

A session of extended riding on the ‘General Road’ at IDIADAs proving ground, see Figure

36, was used to induce physical fatigue aiming to closely replicate the physical demands of long

distance riding journeys

Figure 36 General road test route used for UC E physical fatigue inducement.

The distraction component was unfortunately abandoned due to a non-functioning algorithm.

Thus, there was no data collected for visual distraction. A total of 14 participants were tested at

IDIADA, see Figure 37. The results show that the fatigue detection concept was well received,

as was the HMI. However, the performance limitation function was generally seen as negative.



Figure 37 Participant with Ducati Multistrada with integrated ADAS&ME components.

The evaluation of use case F was moved to the Ducati test track facilities due to the

developmental nature of the prototype motorcycle used to demonstrate the stabilisation

function. The evaluation was carried out with a small sample of 3 expert riders evaluating the

system performance. For safety reasons it was not possible to make rider faint, that is actually

induce one of the critical states. Hence this state was manually triggered in order to evaluate

the developed stabilisation function.

3.6.8 UC G – Automated docking at bus stop

The evaluation of Use Case G was conducted in a bus simulator at the VTI facilities, see Figure

38. The evaluation was carried out with 16 participants, see Figure 39.

Figure 38 VTI bus simulator used for the Use Case G evaluation.



Figure 39 Participant driving the VTI bus simulator in Use Case G.

All data collected have been analyses and a consolidation of results is presented in Deliverable

7.2 (Anund et al., 2020). The data and the results are also used as an input to the impact

assessment in Deliverable 8.1 (Meta, Shingo Usami, Azarko, Jackson, & Verschuur, 2020b)

and also in Deliverable 9.4 covering the exploitation results of ADAS&ME (Verschuur, 2020)

(confidential).

3.6.9 Demanding issues during the final evaluation

The work with data collections and final evaluations had to respond to various events occurring

throughout the project prior to the implementation of the evaluation procedure in the final

evaluations. These events primarily stemmed from issues associated with technological

maturity regarding individual components, algorithms, and the final demonstrator platforms

themselves. The technological maturity issues became especially apparent during realisation of

the final evaluations. Above all, in the evaluation’s changes had to be made from the already

defined experimental procedure in order to respond to integration issues. In some cases,

evaluations had to be postponed and there were issues with missing data due to inconsistently

functioning data loggers.

Unreliability of driver state inducements was also an issue, with these not proving effective in

many cases. This prevented the full assessment of the system functions and resulted in the use

of fallback solutions to activate the ADAS&ME HMI and automation functions.

Test track availability also resulted in some issues and following changes to original proposals

defined in the evaluation framework. This came about for two reasons (1) a delay in the overall

project schedule moving the planned testing outside of the ‘low season’ testing window at the

IDIADA proving ground and causing scheduling conflicts leading to changes and (2) risk

assessment of the evaluation procedure that required changes from the original plan to ensure

the safety of experimenters and participants.

3.6.10 Achievements

Overall the outcomes of the evaluation work package led to the development of a framework

for evaluation of the ADAS&ME system. The framework was successful in providing a generic

template of comparable methods which could be applied across the diverse test vehicles and

locations used for the evaluations. The evaluation framework provided an output that was both

detailed and comparable between use cases. Also, as part of the work package the evaluation

framework was successfully implemented to assess the components of the ADAS&ME system,



whilst being able to respond to required changes.

3.6.11 Recommendations

For future evaluations it is recommended that a process of demonstrator technical integration

is fully completed well in time before the commencement of evaluations is possible.

Recommended time before evaluations are at least more than 2 weeks before to ensure that any

late problems can be responded to and would give a better opportunity to consciously react and

adapt the evaluation protocol accordingly. Perhaps most significantly this would minimise any

impact on the schedule for final preparations.

The work on “Evaluation” was performed mainly in work package 7 of ADAS&ME.



4 Use Case Approach and Achievements

In the following section a brief description of each use case in terms of aim, approach,

achievements, innovations and limitations is presented. For more details of the results please

read ADAS&ME Deliverable 7.2 (Anund et al., 2020).

4.1 Use Case A – Truck

As higher automated vehicles become available, the opportunity exists to use automation to

compensate for compromised driver states. However, as the movement of control between

driver and vehicle becomes possible, there is a risk for confusion regarding who (driver or

vehicle) is responsible for what aspects of control at different times. This was one area

addressed within the ADAS&ME project. Furthermore, the design of the system has the

potential to significantly influence trust (and distrust) in the system which can ultimately affect

system usage. Additionally, a failure (on behalf of the driver) to understand the system status

or mode can significantly compromise safety. Within UC A effort was made to define and create

a system that includes a driver monitoring system and an adaptive HMI that works with an

autonomous vehicle.

4.1.1 Aim

Based on the assumption that automated driving, in general but especially during periods of

compromised driver states, will support safer, more efficient and pleasurable road transport UC

A defined four primary goals: 1. Detect periods of compromised driver states.

2. Create an HMI that:

a. Motivates the driver to handover control to the automation.

b. Builds trust between driver and vehicle.

c. Facilitates effective and pleasurable handovers/takeovers.

3. Develop smart automation that reacts to driver state.

4. Detect when during automated driving the driver is resting.

4.1.2 Approach

Consistent with the ADAS&ME work package structure, the process by which the ADAS&ME

system was developed involved the steps described below. The key use cases were identified

based on a user and stakeholder perspective. Where possible, development of the ADAS&ME

system was consistent with this and at the same time considered the general development

direction of technology within Scania and the different goals and timescales that exist between

ADAS&ME and Research and Development at Scania.

With the scenarios defined it was necessary to create a vehicle architecture around which the

ADAS&ME system could be built. The architecture was created in consultation with other UCs

to ensure that as much work as possible was re-usable between use cases, see Figure 40.



Figure 40 Summary of UC A architecture.

Based on the defined architecture, efforts were undertaken to define and determine environment

and driver states. Initially, the intention within Use Case A was to make use of an automated

vehicle platform being developed internally within Scania. For the environmental issues, the

effort was oriented towards adapting the environmental monitoring platform of the Scania

platform to be consistent with the ADAS&ME vehicle architecture. This included developing

or adapting the IDIADA test track maps for use with the Scania automated vehicle. Due

primarily to safety constraints, an automated vehicle was not available for use within

ADAS&ME. Consequently, an automation simulator was developed to compensate for this,

and the remaining ADAS&ME system components were created ‘plug-and-play’ ready for an

autonomous truck.

The work on driver impairment detection was focused on specifying what driver states could

be valuable to long-haul truck drivers and how they could be measured. To support the

algorithm developers two separate data collections were done. The first was with 10

professional drivers who drove two 450 km drives while data was collected from a range of

physiology monitoring sensors. A constraint identified post-hoc with this data collection was

that only a few of the desired driver states were occurring naturally. Thus, in the second data

collection a simulated driving task was used where the relevant driver states were induced, see

Figure 41.

Figure 41 Photo of Stage I (left) and Stage II (right) data collection.



The data collected supported the algorithm development. Additionally, information collected

influenced the ongoing HMI development and a major contribution was the development of the

HMI strategy which guided the development of the Decision Support Module, see Figure 42.

The HMI Strategy defined the state machine of behaviour related to different driver states,

environmental events and automation modes.

Figure 42 HMI Strategy Overview table.

In addition to the overarching HMI strategy and all the graphical and auditory files (and

programming of their behaviour) for the included HMI elements were created and tested. The

six HMI elements used were Instrument Cluster, Secondary Display, Audio Display, Seat

Vibrators, Steering wheel LEDs and Cabin LEDs, see Figure 43.

Figure 43 Graphical representation of HMI elements (left panel) and photo from

simulator testing (right).

With the architecture defined, the driver monitoring method and algorithm developed and the

HMI elements and behaviour defined the integration into the truck took place, see Figure 31.

4.1.3 Achievements

The results show that the participants spontaneously handed over when automation became

available, both in a non-elevated driver state (normal state) and when distracted, frustrated or

sleepy (impaired driver state). It was only during frustration that two participants did not

handover directly. All drivers seem to understand how the handover and takeover procedures



should be performed. The handover is done simply by pushing a button on the steering wheel,

with no other requirements on the driver, and was done without problem by the drivers during

all driver states. The takeover is a longer process that requires more of the driver but seems to

be equally understandable. Even though all drivers needed at least two tries before successfully

finishing their first takeover, the system was described in positive terms by all drivers. The

drivers found the takeover procedure strict but necessary. Some negative comments by drivers

related to how strict the gaze detection was during takeover. In general, the drivers found the

HMI understandable and clear. They especially liked the use of coloured LEDs on the steering

wheel and the instrument panel and the seat vibration as an escalated warning. The system was

well accepted, and the drivers commented that it would make it safer to drive with an elevated

driver state. Confidence in use, ease of use and system integration were given high ratings on

the System Usability Scale. System acceptance and trust were also very highly rated.

The drivers seem approve of the HMI and warnings for sleepiness and distraction, whereas for

frustration, the comments from the drivers were more mixed. The reason for this could be

related to how clear the risk is perceived for each driver state. Strong warnings were accepted

for sleepiness and distraction, but not for frustration. One recurring comment was that for

sleepiness the seat vibration should be triggered for moderately sleepy as well, and not only for

critically sleepy. The rationale was that it is better to warn the driver in time than wait for the

driver to become critically sleepy. For critically sleepy, several drivers commented that the

system should forcefully take over control.

One common error by the participants was that they did not look forward and did not have both

hands on the steering wheel during the takeover procedure in their first tries to take over control

(when trying the procedure uninitiated during the discoverability test). In the subsequent trials,

all participants completed the takeover. The system was designed to make it easy for the driver

to do a handover when automation was available, and to be strict and check the driver’s fitness

to drive manually when doing a takeover. The results show that all participants completed the

handover at their first try, and all drivers were pushed to look forward and have both hands on

the steering wheel for five seconds to complete the takeover. Some of the negative comments

regarding the strict takeover procedure related to a system error, resulting in gazes directed to

the instrument cluster not always being detected. This speaks for the original concept which

was designed to accommodate for gazes towards the instrument cluster, where the takeover

instructions were shown. Some of the drivers’ comments indicated that gazes directed towards

the mirrors should also be allowed , which could suggest that a more complex distraction

algorithm is needed to allow for the regular gaze behaviour used during driving, and not only

for gazes on the road ahead. Another reason for drivers failing to do the takeover procedure was

that the system did not register that the driver had both hands on the steering wheel while

pressing the steering wheel button. This was likely caused by an overly strict hands-on detection

system that did not allow for the hand movement needed to reach the button.

Four participants removed their hands from the steering wheel directly after automation was

activated when the system prompted that the automation was to be engaged. After 15 seconds,

a confirmation showed that the system was now driving the vehicle and that it was safe to

remove the hands from the steering wheel. This was when the drivers were expected to release

the steering wheel, while their actual behaviour (acting on automation activation) was likely

caused by an unnecessarily long delay before showing the confirmation.



Some of the key achievements from UC A include:

• Creation and implementation of a new driver monitoring system and adaptive HMI

that considers and influences automated driving mode.

• Full data flow from sensor data, driver state detection, decision module, and HMI

triggering/Automation change.

• Creation and development of the demonstration truck with open architecture

accessible to all (relevant) partners.

4.1.4 Innovations

Some primary innovations developed in UC A include:

▪ Use of a single sensor (eye tracker) to support detection of four driver states

(sleepiness, distraction, frustration, and resting).

▪ HMI strategy and Adaptive HMI Logic.

▪ Video based emotion algorithm enables continuous tracking of emotional state and is

integrated into a Jetson hardware.

▪ Well accepted and understood Handover and Takeover procedure.

4.1.5 Limitations

There are two primary limitations within Use Case A. First, although the ADAS&ME system

was designed to be ready for use in an autonomous truck, due primarily to safety reasons the

solutions developed were not deployed or tested in an autonomous truck. Despite this, the

developed system listens to and sends output to automation mode even though there is no actual

autonomous system. Related to this is that the system could not be tested in the intended way,

that is, with full automation possible. An additional limitation for the final testing arose due to

safety constraints associated with certain hardware integration, full manual control with all HMI

elements working was not possible. A solution was developed involving full manual control

(limited HMI), simulated manual control (full HMI), and simulated autonomous control (full

HMI) of the ADAS&ME truck.



4.2 Use Case B - Electrical vehicle

Range anxiety (or range paradox) is a concept emerged at late 90s which is the concern of not

reaching to the destination or to the next charging spot while traveling in an Electrical vehicle.

This is a stressful experience of a present or anticipated range situation, where the range

resources and personal resources are in fact available to effectively manage the situation,

however they are perceived to be insufficient. Studies show that, electric vehicle drivers usually

need around 160 km of autonomy per charge. Nevertheless, they often prefer vehicles with

considerably higher available range (around 350 km). This demand (which seems to be

avertable) comes from the worry of experiencing such a situation in the future or present, worry

of what happen if such a situation emerges, worry of not being able to find a solution to the

situation and from the worry to being stranded in this uncomfortable situation. If the

manufacturers cannot lower the range anxiety to near zero, electric vehicles will not be able to

compete with gasoline and diesel cars. Acceptability of electrical usage is often linked to the

anxiety of running out of battery power. This UC demonstrates how a better management of

the driving range information, coupled with an anticipative and protective approach, can

decrease this anxiety.

4.2.1 Aim

The aim was to develop a system with an HMI that effectively support drivers to reduce their

level of anxiety and at the same time reach their destination without running out of battery

power.

4.2.2 Approach

The work around this use case followed the same structure as the others considering system

architecture, specifications, driver detection algorithms, HMI development, verifications,

integration and final evaluations.

Two data collections took place before the final evaluation. Both were aimed to collect

empirical data related to electric car drivers’ emotional states while driving. Data was used

mainly to design, develop and test algorithms toward an automatic emotion recognition based

on facial expressions and speech. The data collection was also used to understand how to induce

range Anxiety among the electric car drivers.

The environment, the driving task or even the HMI itself might contribute to anxiety creation.

Hence, it is mandatory to verify whether the perceived anxiety is due to the range. The

experience of range anxiety is assumed to be expressed on the following levels:

1. Cognitive level: (i.e., negative cognition associated with range like concerns about

running out of energy and not being able to reach the destination)

2. Emotional level: (i.e., changes in affect associated with a range situation like feeling

of nervousness or even fear)

3. Behavioural level: Decreasing immediate anxiety by increasing perceptions of safety

and control, (i.e., certain activities like tapping with fingers on the steering wheel,

changing driving style to save energy or frequent checking of relevant displays, e.g.,

range and navigation display or yelling, honking, aggressive gesturing)

4. Physiological level: Under parasympathetic control (i.e., blood pressure, heart rate,

heart rate variability, galvanic skin response (skin conductance), cortisol level, pupil



diameter) and under sympathetic and parasympathetic control (i.e., respiratory

frequency).

In the light of previous information, range anxiety detection is accomplished in two distinct

steps. The first step is detecting basic anxiety by combining the cognitive, emotional and

physiological levels of expression. Although these clues are fundamental to detect the anxiety,

they are not capable to define the actual source of it. Hence, the information from the

behavioural expression level is used to conclude if the detected anxiety is related to the range

or not. The key data was to define if the driver was frequently checking the range information

displayed on the HMI while he was anxious.

4.2.3 Achievements

The results from the final evaluation show that the participants did not fully accept the system.

They rated most items near the zero-reference point. No aspect was rated negatively, but most

aspects only slightly positive which shows that there is some space for improvements. The

participants rated the system as quite pleasant, good and effective. Furthermore, they indicated

that the system is able to raise alertness. In addition, participants found the system quite easy to

use and would imagine that most people would learn to use the system quite quickly. They

assessed their trust on a medium level which means that they were not fully confident with the

system, and they were not totally convinced that the system provides safety. They rated the

system as quite integer, dependable, reliable and trustworthy and felt quite familiar with the

system. This might be due to the design of the data collection were the test leader played a role

convincing the driver that it was true they were soon out of battery power.



4.3 Use Case C/D - Conventional passenger car

4.3.1 Aim

The aim was to develop a system that provide a driver state based smooth and safe automation

transition in situations when unsuccessful handover due to non-reacting driving is the case. The

aim is to develop such system both during controlled and emergency situations.

4.3.2 Approach

Use Case C focused on driver state based smooth and safe transitions between manual and

automated driving. Therefore, tailored HMI strategies for the use case specific driver states

were developed and evaluated. Four different driver states were addressed in Use Case C:

sleepiness, stress, frustration and distraction.

When in manual mode, the ADAS&ME system checks if the driver is in an adequate driver

state. If the driver state changes into an inadequate driver state, the HMI reflects this and gives

the driver feedback regarding the detected driver state. Additionally, the ADAS&ME system

offers the opportunity to change to an automated driving mode. If the driver agrees, the switch

to the automation mode is communicated and the vehicle automation is in control of the vehicle.

When the driver experiences automated driving, the automation will constantly check for

possible future situations which cannot be handled by the automation, i.e. environmental

conditions where automated driving is no longer possible. In these situations, the driver

functions as a fall back and needs to take over the vehicle control. If a takeover by the driver is

needed, the system will also consider the driver’s state. In case the driver is capacitated to drive,

the system will perform a standard transition. However, if the driver is in a degraded driver

state (sleepy, stressed, frustrated or distracted), the timing of relevant HMI messages is adapted

to the criticality of the specific driver state, supporting the driver in performing smoother and

safer transitions.

The function developed for use case D is an escalation of use case C. Similar to use case C,

once driving on a highway in SAE level 3, when approaching a system limit a driver state based

transition request (from automated to manual) is sent. However, the driver does not react to the

request and fails to take over control of the vehicle in time. Due to the non-reacting driver the

automation needs to intervene and starts a safe stop manoeuvre. The safe stop manoeuvre results

in lane changes to an emergency lane and braking manoeuvre to a full stop. If there is a lead

vehicle the follow me function will be started. The follow me manoeuvre results in following

the lead vehicle and then in a lane change with a safe stop. These functions were implemented

in the same vehicle as use case C functions, i.e. DLR’s test vehicle.

At first relevant driving scenarios for this use case were identified and agreed on. In a next step,

data collections were conducted to gather more insight in the inducement procedure of the

different driver states and collect sensor data for the algorithm development. Furthermore,

interaction strategies for smooth and safe transitions were developed and assessed in simulator

studies. As a last step the ADAS&ME functionalities were implemented in DLR’s test vehicle

(FASCarII), a modified VW Passat which was equipped with multiple computers and sensors

necessary for driving in highly automated mode (SAE3). The FASCar II was equipped with

project specific sensors and computers from project partners to detect driver states and elements

in the driving environment. Software (automation and environmental sensing) as well as

hardware prototypes were also included. Further, a second braking pedal was installed at the

passenger seat. When the FASCar II was driven in highly automated mode a safety driver was

required in the passenger seat to bring the vehicle to a safe stop in case of an unlikely event of



a system failure. Further information regarding technical and hardware architecture of the

FASCar II can be found in ADAS&ME internal Deliverable D2.1. For the automated functions

in use case D an expert rating was performed to find the most accepted thresholds for the safe

stop manoeuvre, see Figure 44.

Figure 44 The UC C/D Car.

4.3.3 Achievements

The results of the evaluation reveal the potential of the developed driver state adaptive HMI

transition strategies in Use Case C and D. Regarding transitions for control from manual to

highly automated driving, the results show that using the ADAS&ME HMI with driver state

specific HMI strategies reduces the time of manual driving in an inadequate driver state. A key

factor is the “knowledge” of the system regarding the actual driver state. If the driver is in an

inadequate driver state, the automation can prompt the driver to activate the automation earlier.

This strategy results in a shortened time of manual driving in an inadequate driver state.

However, we could not find an effect in reducing take-over times by using the developed

ADAS&ME HMI. The ADAS&ME HMI does not reduce the time to take-over vehicle control

after automated driving compared to a standard HMI without driver state adaptation.

Nevertheless, take-over times in both HMI conditions were very fast (< 6 seconds) so both HMI

variants seam to support the driver in performing the transition of control. The main reason for

not finding differences between the HMI designs could be the arrangement of the scenario. Due

to test track limitations, we could not let participants experience a longer period of pure

automated driving (> 20 min). Further, for safety reasons, there was not a dangerous situation

in front of the car while performing the take-over. Since there was no urge to intervene fast (no

urgency of the situation) participants may have reacted a bit later which led to prolonged take-

over times. A major point for the evaluation of a transition design is the success rate. The

comparison of both HMI strategies shows important differences in these criteria. By using the

ADAS&ME HMI with driver state specific strategies lead to a higher rate in successful

transitions of control compared to the standard HMI. Especially in the distraction trials the

ADAS&ME HMI helps the driver to take over the vehicle control after driving in an automated

mode more successfully than a standard HMI. Again, this illustrated the potential of the

ADAS&ME HMI design to enhance traffic safety. Use Case D evaluated the HMI design if the

driver was not reacting to the take-over request by the system. Participants experienced a

minimum risk manoeuvre by the vehicle ending in a full stop. Participants rated the HMI

solution positively and trustful.

Some of the key achievements in UC C/D included:

▪ Planning and realisation of V2X communication.



▪ Planning and realisation of the agreed software architecture.

▪ Development of a driver state specific HMI for transitions of control from highly

automated to manual driving and vice versa.

▪ Development of a tailored interaction strategy for 4 different driver states.

▪ Development and integration of strategies for personalized HMI in demonstrator

vehicle using results from 2 HMI studies.

▪ Realisation and integration of the specific HMI strategies including hardware and

software into the demonstrator vehicle.

▪ Integration of partner Hardware (5 PCs) into physical architecture of one demonstrator

vehicle.

▪ Integration of a project wide software middleware into the demonstrator vehicle.

▪ Integration and realisation of automated driving functionality in demonstrator vehicle.

▪ Planning and realisation of the final evaluation at IDIADAS test track in Barcelona.

4.3.4 Innovations

Some primary innovations developed in UC C/D include:

▪ Development of well accepted and safe driver state adapted interaction strategy for

vehicle HMI.

▪ Tailored interaction timing for criticality of driver state and environment.

▪ Personalisation of interaction strategies.

▪ Integration of hardware and software for the assessment of 4 different driver states

simultaneously in one demonstrator vehicle.

▪ Realisation of driver state assessment for automated driving.

▪ Connected automated functions for driver state related minimum risk manoeuvre in

case of a non-reacting driver.

4.3.5 Limitations

Even though the evaluation study on a test track provided important insight into the

effectiveness of the developed system there is still a lack of understanding of its effectiveness

on the real road. In addition, the evaluation study suffered from practical problems during the

testing which resulted in a lack of objective data on system effectiveness. More detailed

research is needed for investigating the developed HMI strategies in different road types and in

different (also urgent) scenarios. Further, the driver state specific HMI strategies for the

transitions need to be adjusted continuously to provide results in this research area and the

development of driver state assessment needs to continue. Not all sensors used in the prototype

system are practically useful for the automotive industry or in line with legal conventions (e.g.

large computer in the trunk for processing data, microphones in driver’s field of view).

4.4 Use Case E/F - Motorbike and protective gear

During long-range motorbike touring, environmental conditions (e.g. extremes of temperature),

combined with rider fatigue, can affect the rider’s physiological (e.g. resulting in rider

dehydration) and psychological state and lead to high-risk situations. Further in situations where

the rider is losing consciousness the motorcycle becomes uncontrollable resulting in serious

accidents. For the above situations the rider states of physical fatigue, inattention and stress

were studied.



4.4.1 Aim

The aim was to develop a system that assist the rider during long range touring, in case of

physical fatigue and inattention. In case the rider faint the system should also activate a support

system including automated stabilization of the bike and speed reduction.

4.4.2 Approach

The function developed for long-range attentive touring in use case E supports riders during

long-range motorbike touring. Based on environmental information and rider state, the system

detects signs of physical fatigue, distraction or stress and sends a warning to suggest the rider

to make a pause. The detection is based on sensors integrated in the protective gear together

with some of the HMI solutions used for warning and information the rider, see Figure 45.

Figure 45 Rider protective gear with integrated sensors.

If the rider does not comply, a recovery mode is activated limiting the motorcycle performance

without stopping it. The function limits the power and, consequently, the motorbike’s

longitudinal acceleration in order to not exceed a predefined speed. To deactivate this mode

and have the full motorcycle potential again, the rider must take a break or to power off the

motorbike.

For the development of the detections system a data collection was required to understand the

development of rider states, especially physical fatigue, inattention and stress. Those data were

collected both in simulators and on real road.

The final evaluation of the function used for UC E was tested at IDIADA’s proving ground and

surroundings by using a Ducati Multistrada 1260 equipped with the rider state detection

algorithms and HMI as well as other communication systems. The stabilisation system to be

used during rider faint in use case F was evaluated at DUCATI due to safety issues and time

constraints.

4.4.3 Achievements

The main results for the long-range touring and rider faint perspective are the fact that an

innovative capsize control algorithm capable of stabilizing a motorbike was designed. The

system also works in the presence of unbalanced loads or errors affecting the roll angle

estimation. In addition, an HMI devices manager, tailored to get the best feedback combination

to inform the rider, were developed and integrated. Also, the integration and interaction with a

wearable electronic sensor and unit outside the back protector is an innovation to be mentioned.



From a technical perspective, a full set of wearable electronic devices with a wireless

connection, allowing rider physiological state monitoring and visual, acoustic and haptic

feedback, was developed. The HMI devices aimed to offer effective rider support in case of

fatigue and distraction and the results show a close to production solution. From a safety

perspective, a set of functions to mitigate high-risk conditions such as torque limitation and

capsize control was developed, but also a system that uses an automated HMI strategy for

smooth transitions and effective rider warnings.

The main achievements in UC E/F are:

▪ Design of innovative capsize control software capable of stabilizing the motorbike even

in presence of unbalanced loads or errors affecting the roll angle estimation

▪ Integrated and coherent HMI devices management, tailored to get the best feedback

combination to inform the rider

▪ First integration of wearable electronic sensor and unit outside the back protector

4.4.4 Limitations

A clear limitation is the fact that a new system like the once develop is very safety critical to

evaluate with real riders. Only professional test riders were allowed to drive and to safety

aspects since rider faint and rider distraction manipulation was not possible to do due to safety

reason.

4.5 Use Case G - City bus

Driver fatigue has received increased attention during recent years and is now considered to be

a major contributor to approximately 15–30% of all crashes. The main cause of driver fatigue

is sleepiness due to sleep loss, being awake for too long, and driving during the circadian low.

Also, work-related factors such as stress and shift work contribute to driver fatigue. In addition,

it is important to consider the type of task, as both cognitive underload and overload contribute

to demanding situations influencing the drivers. The goal of doubling the public transportation

travels by 2020 requires more efficient operation, and already now working as a bus driver

involves a lot more than just driving the vehicle. The responsibilities to control where to go,

keep track of the timetable, make sure that the bus is on time, oversee and support ticketing,

communicate with the operator and interact with the passengers can be overwhelming. On top

of that the bus driver occupation is associated with negative physical and psychosocial factors

related to driver’s health. The factors described as most important are poor in-vehicle

ergonomics, risk of injury, shift work, working alone, time pressure, manoeuvring within

increasingly congested urban environments, risk of intimidation and violence, accessibility

problems as well as poor health, in particular obesity. Many of these factors are also expected

to become more severe in the future and lead to an even more stressful work environment. High

levels of work-related stress and disturbed sleep is a dangerous combination contributing to

diseases and poor workplace performance. One possible way to support bus drivers is by

introducing automated functionalities that release them form some tasks, to avoid overload and

at the same time improve the comfort and safety for passengers and people outside the bus.

Such a functionality might be automated docking at a bus stop.

4.5.1 Aim

The aim of this use case was to develop and evaluate the effect of a future system using

automation during docking at bus stop.



4.5.2 Approach

The automated docking function was system initiated and the drivers were asked to give control

to the bus when approaching the bus stop, but also to take back control when the bus stop zone

was coming to its end. When leaving the bus stop zone the system decided if the drivers were

fit to get the control back, taking into consideration detection of sleepiness and distraction but

also confirming that the drivers had at least one hand on the steering wheel, see Figure 46.

The development of the HMI strategy for the involved scenarios included three different steps.

The first step was an exploratory study with the aim to understand bus drivers’ working

conditions and problems. This was done during an experiment on real road with 15 city bus

drivers involved. The second step was a virtual reality (VR) simulation study with 10 bus

drivers in which a first HMI concept was evaluated. The outcome of the VR study was then

modified and integrated in a moving-base driving simulator at VTI, Sweden. The third step was

a pre-pilot with 7 city bus drivers driving both in an alert and an expected sleepy condition.

Tuning and adjustment of the HMI, detection algorithms and automated functionalities were

made based on the outcome of the pre-pilot.

The final evaluation took place using an urban city bus route scenario with in total 20 bus stops

(10 with and 10 without automated docking available). In total 16 participants drove twice, once

in an alert and once in a sleepy condition. The design was a within-subject design and the order

of driver state was balanced. The same simulator as in the pre-pilot was used.

Figure 46 Use Case G scenario with HMI for an automated docking functionality

integrated

4.5.3 Achievements

It can be concluded that the results from the final evaluation of the ADAS&ME system with

automated docking at bus stop showed that automation did not result in a significant difference

in self-reported stress and blink duration. There was however a significant increase in KSS and

in HRV (RMSSD) indicating lower arousal. In addition, there was a significant reduction in

speed.

In general, there was an effect of what the driver experienced first, manual driving or

automation. The drivers starting with manual driving showed higher KSS and longer blink

duration during automation. The highest levels on sleepiness indicators were seen during

automation with expected sleepy drivers, supporting an effect of time on task. For those starting

with automation there was also an effect of automation on blink duration and HRV indicating

more sleepiness signs and less arousal during automation. Looking into driving behaviour, it

could be noticed that departing from the bus stop showed slightly lower accelerations with the



automated system activated. A large increase in the amount of lost tracking and glances towards

“other” was revealed when the bus was about to take over control from the driver. The drivers

liked the system, wanted it and thought it could contribute to increased safety.

4.5.4 Innovations

Some primary innovations developed in UC G include: ▪ A system for automated docking at bus stop.

▪ An integration of automation and driver state detection that only gives back control if the

driver is fit to drive defined by being alert and/or attentive and has a least one hand on the

steering wheel.

▪ An HMI strategy for transition that is easy to understand and to use.

▪ An automated function that reduces the speed while docking (deceleration and acceleration).

4.5.5 Limitations

There were differences in the results depending on if the driver started to drive with or without

automation. It seems that there is a carry-over effect and an interaction effect if the driver was

expected to be alert or sleepy. The results clearly indicated that the system first has one effect

which is changing with time on task. How to avoid this needs further investigation.

The evaluation of automated docking at bus stop suffered from some limitations. One is that

the system was not evaluated in a real road environment. Therefore, it is difficult to say if the

results are valid also for real road situations. This is something that needs further investigation.

To be able to understand the effect of the developed system it is essential to implement it in a

bus on real road and look at real stress levels and on the user’s opinion of the HMI. In the

conducted study the focus was on the drivers, but in future research also the passengers’

perspective is relevant to include. It is important to understand how to communicate that and

how the bus is docking automatically to the passengers to guarantee trust and acceptance among

users.



5 Assessment of the impact

The assessment of the impact has been undertaken from a multicriteria perspective which

reflects the macro areas of impact: Safety, mobility and environment, economic and social,

Legal and regulatory issues. For more details see Deliverable 8.1 Impact assessment (Meta,

Shingo Usami, Azarko, Jackson, & Verschuur, 2020a).

Except for the Legal area, for which the analysis is conducted across the UCs, the impact

assessment has been undertaken for each use case separately considering safety, mobility and

environmental impact as well as the socio economic one. The methodology applied for the

assessment cover mainly the following:

▪ Safety, mobility and environmental impact assessment ▪ Analysis of user-related aspects (from drivers and potential users’ point of view) ▪ Cost-Benefit Analysis (CBA) ▪ Stakeholders’ analysis

5.1 Safety, mobility and environment

The impact assessment is closely related to several parts of the project as described in Figure

47.

Figure 47 Impact assessment and its relations to other parts of the work in ADAS&ME

ADAS&ME has its priority on safety and therefore also the safety impact is of high priority.

The work with safety impact followed the procedure described in Figure 48.



Figure 48 Safety impact assessment procedure

From the safety impact point of view, it can be said that the ADAD&ME systems meet the

principal aim of counteracting the underlying causes behind risks that affect the safety of all

road users, contributing to increased safety for both the driver and passengers of vehicles fitted

with the ADAS&ME system, and other road users that come into contact with these vehicles.

There is evidence from all the use cases supporting the success of the system in mitigating

effects of adverse driver states. Thus, it is possible to conclude that the ADAS&ME systems

can contribute to the prevention of resultant accidents that cite these errors as route causes.

Moreover, it is possible to say that there is an improvement in lateral and longitudinal

accelerations that is seen through integration of the system on buses (UC G). In addition,

evidence is found to support the possibility of heightened trust and acceptance associated with

the ADAS&ME systems. In some cases, this can be viewed as a positive impact – improving

the likelihood of users using the ADAS&ME system such that the other safety benefits can

manifest. There is, however, evidence found within the literature to suggest a link with reliance-

associated behaviour which could have the potential to increase risks of driver error in some

situations.

Concerning the mobility impact, with the evident reduction in aggregate journey time

associated with use of the ADAS&ME system in UC G, an immediate improvement to mobility

of passengers can be supported. This is in conjunction with the evidence for reduced journey

time having a positive effect on mobility.

For environment, data collected in the ADAS&ME evaluations generally supports the

hypothesis of improved efficiency following its implementation. It can be said there will be a

positive impact on emissions following increased battery electric vehicle use. Also tested were

potential increases in efficiency brought about by influences on driving behaviour.

5.2 Economic and social impact

The socio-economic impact assessment has it starting point in the user related aspects. The

results show a positive an overall attitude regarding the acceptance of ADAS&ME systems

among the respondents (drivers who experienced the system and survey respondents). The most

desirable Use Case is UC C/D which has the high rate for willingness to use as well as for the

trust and willingness to pay extra money. The cross-analysis among Use Cases shows that all

systems were rated positively but there were differences between the Use Cases. The systems

developed in use case A, use case C/D as well as use case G were assessed as somewhat positive

1. Definition of target accident

population

2. % of target accident in

EU28

3. Forecast of target accident

in 2030

4. Market penetration

5. Safetybenefits

estimation

6. Cost – unit rates



with a tendency to very positive. The system developed in use case B and part of the system

developed in use case E/F were assessed as somewhat positive with a tendency to neutral.

Overall, the CBA has shown different results according to each UC. Cost-Benefit Ratio (CBR)

values in low and medium scenario are lower than 1, showing a socio-economic inefficiency.

The main reason for these poor results is the fact that the costs of the systems are quite high,

especially for UC A and UC C-D. The other use casess show a CBR greater than 1 in the high

scenario. This means that the target year 2030 may be too early to reach any significant

penetration rate and, thus, benefits for all the systems.

The results of the Stakeholders Analysis have shown that Authorities and Policy makers were

the most skeptical among the Use Cases as well. The main barriers to the full scale deployment

of such systems reported were mostly related to the acceptance and trust aspects, confusion

about HMI operations, increase of a complexity of the systems which lead to doubts in

reliability and safety of the vehicle equipped with ADAS&ME systems.

5.3 Legal and regulatory impact assessment

The Legal assessment has highlighted that the General Data Protection Regulation (GDPR)

does not impede ADAS&ME but imposes several significant obligations. In particular, the

commercialisation of certain personal (such as health data) may be incompatible with the

GDPR. However, no specific solutions required at this stage.

Concerning Traffic Rules compliance, most of the national systems assessed in the present

impact study currently allow for completely autonomous driving (SAE levels 4 and 5), i.e.

without a driver being able to control the car (within the car or remotely). Legislative change

seems necessary to allow for (temporary) control by automated driving systems. In particular,

the definitions of 'control' and 'guardian' may have to be changed in various (national) legal

systems. Finally, regarding Insurance and liability issues, the existing notions of 'control',

relevant for attributing liability, might not be adequate for automated driving. Specific rules

should be considered in respect of (i) the allocation of liability when vehicles are in automated

driving mode and (ii) the burden of proof for consumers in case of deficiencies of automated

driving systems.



6 Exploitation of the results

The main goal of this activity was the creation and validation of the business model concepts

of ADAS&ME innovations, including the Use Cases. A total of 26 innovations were selected

for the realisation of the safety, economic and environmental potential of integrated technical

solutions and were further analysed, see Table 4.

Table 4 Exploitable results of ADAS&ME

Exploitable Results UCs Partner(s)

New tools for customer insights via unobtrusive monitoring A SCANIA, DENSO

Sustainable product development via VR based HMI development

tools

A SCANIA, DENSO

Human Controller state modules as individual components A SCANIA, DENSO

Resting module for haulage company A SCANIA, DENSO

External HMI for incapacitate driver A SCANIA, DENSO

Automatic calling of 112 A SCANIA, DENSO

HMI personalization package A SCANIA, DENSO

Smart route planning for individual drivers based on physiological state A SCANIA, DENSO

EV range anxiety B VALEO, VEDECOM

Driver impairing emotions detection system C EPFL

Environmental situation awareness module C TOMTOM, DENSO

Heart rate and respiration detection steering wheel radar1 C VALEO

Camera based sleepiness and inattention detection system2 C SEYE

Stress and discomfort detection seat sensor C FORD

HMI and automated function steering wheel3 C AUTOLIV

Driver state-based smooth and safe automation transitions C DENSO

Non-reacting driver emergency manoeuvre function D DLR, DENSO,

TomTom, FORD

Long range attentive touring function with motorbike E DUCATI, DAINESE

Rider faint function F DUCATI, DAINESE

Passenger pick-up/drop-off automation function for buses G VTI, SCANIA

Passenger pick up/drop off automation for buses G VTI

HMI steering wheel with LED bar and 2-zone hands on detection

sensor

G AUTOLIV

Driver identification module B, C, D SEYE

Driver personalisation module A, B, C, D, FORTH

1 Previously referred to as: "Inattention/distraction detection steering wheel sensor". 2 Previously referred to as: "Camera based drowsiness and inattention detection system". 3 Previously referred to as: "HMI and automated function modules".



G

Rider identification module E, F DUCATI, DAINESE

Rider personalisation module E, F DUCATI, DAINESE

The following conclusions were reached:

▪ As to IP rights, some of the innovations are patentable, while some others are not

(mainly due to existing patents). In some cases, the technology developed will be

publicly disseminated so that all market players can further develop and exploit the

innovations.

▪ The expected market prices differ substantially (depending on the nature of the products

and the markets involved), but for most products they are expected to amount to several

hundreds of EUR.

▪ The market potential is generally seen as (relatively) positive for most products.

▪ With the exception of a limited number of products (which are expected to be ready for

the market within the next 12-24 months), the time market for many products is at least

3-5 years and for some even 5-10 years. A number of products are still in the

development phase.

▪ The market risks identified mainly concern a perceived low added value of the products

(in the eyes of the potential users), the novelty of the products (and the associated risk

of a perceived lack of reliability), health concerns (electromagnetic waves) and a lack

of standardization/interoperability. Furthermore, the Legal Impact Assessment (part of

WP8) identified potential legal impediments in the area of Data Privacy (GDPR),

Traffic Rules, and Insurance and Liability.

▪ The exploitation routes differ in function of the state of development of the products

(many of which require further research and/or development) and the intentions of the

consortium partners involved (for example, research institutions do not have the

intention to exploit the innovations on the market, while some of the partners intend to

use the innovations for internal purposes rather than externally).



7 Dissemination

Communication actions and dissemination activities have been of high importance from start

to end of the ADAS&ME project. In order to achieve high impact of ADAS&ME results several

offline and online tools were developed, highlighting the project website, social media networks

and videos tailored for all the audience. In addition, dissemination of results at key events,

collaborations with other EU-projects, and writing scientific papers with ADAS&ME partners

but also with externa collaborators have been important. The exploitation of the results obtained

during the project lifetime was also a key aspect for the project and its legacy, a market analysis

was carried out to support the business plan of the ADAS&ME innovations. Firstly, a wide

socio-economic impact analysis was created through a CBA (cost-benefit analyses) considering

the stakeholder consultation and finally the Business and Exploitation plans for 17 potential

ADAS&ME developments. All dissemination activities are describe in Deliverable 9.7

(Figuals, 2020).

The dissemination plan and activities were developed at the beginning of the project and then

updated every year. Definition of our audience and the creation of the project image with

logotype, roll-up and all the promotional aspects including the use cases’ illustrations took

place. The website was developed at the very beginning of the project and reached 50,000

webpage visits during the project lifetime well supported by the social media networks (Twitter,

LinkedIn and Youtube channels) that exceed the KPIs included at the dissemination planning

with more than 1,000 followers. One of the most important tools to promote the project and

explain the progress was the videos, three animated videos explaining the ADAS&ME concept,

the motorbike use cases and the car use cases were produced with the objective to create

awareness to a wide audience. At the end of the project the use case demonstration videos were

developed to secure a legacy of the developments achieved during the project, Figure 49.

Figure 49 ADAS&ME project Youtube channel.



A list of key stakeholders was created. Those stakeholders were invited to the Stakeholder

Workshop set in Brussels where the selection of the ADAS&ME use cases and the

corresponding use case scenarios along with their impact were defined. This first contact was

successful for the project development and it built a close contact with several external

stakeholders who have been used for promoting the project through social media, increasing

the performance of user survey used for the impact assessment and inviting participants to the

Final Event, see Figure 50.

For videos from the final event please visit the following links:

• Final Event Use Case Demonstration: https://www.youtube.com/watch?v=qpQvBG8XNwM

• Final Event promo: https://www.youtube.com/watch?v=rGFgjb55nOo

• Interview Anna Anund: https://www.youtube.com/watch?v=A4EkxFDorfU

• Interview with Angelos Bekiars: https://www.youtube.com/watch?v=O8xIQMo9TiU

Figure 50 Participants at the ADAS&ME Final event December 2019.

The most relevant aspect of dissemination was the large participation of ADAS&ME in national

and international events/conferences. The total was 35 direct participations including the most

relevant of the European framework: EUCAR Conference (Brussels), stand at the EUCAR

Conference (Brussels), special sessions at ITS Europe Congress (Copenhagen and Brainport),

the co-organization of Autonomous Vehicle & Development Symposium (Stuttgart) and

participation in INEA stand at TRA (Vienna) among other international workshops and events.

In addition, 25 scientific papers have been published during the project lifetime, see also

Deliverable 9.7 (Figuals, 2020).

Throughout the project ADAS&ME partners have reached out and shared knowledge with the

EU funded projects I-DREAMS, PROSPECT, AutoMATE, MAVEN, TrustVehicle, VI-DAS,

TransAID and BRAVE. As an example, ADAS&ME co-organized with AutoMATE the

AutomotiveUI (Oldenburg) and Intuitive Partially and Highly Automated Driving Conference

(Aachen). In addition, this Activity was also extended to collaboration with other initiatives

such as CardioID Technologies which gave ADAS&ME interesting inputs during the project

lifetime and resulted in a conference proceedings titled “Driver drowsiness detection: a

comparison between intrusive and non-intrusive signal acquisition methods”. Finally, during

the last stage of the project ADAS&ME were in contact with the new awarded Automated Road

Transport projects Suaave, Trustonomy and Drive2theFuture in order to share the knowledge

generated by the project.

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DqpQvBG8XNwM&data=02%7C01%7Canna.anund%40vti.se%7C63451347eff34ce5faab08d7b49f3e92%7C44f529522a66495880d4da9db72ad662%7C1%7C0%7C637176468360606990&sdata=lNWmP9VEBsbYVWqP9df3XRRG4a7vXStifr4O9Az4Jj0%3D&reserved=0

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DrGFgjb55nOo&data=02%7C01%7Canna.anund%40vti.se%7C63451347eff34ce5faab08d7b49f3e92%7C44f529522a66495880d4da9db72ad662%7C1%7C0%7C637176468360606990&sdata=kW8b1MRbeq3p1DAuEZX%2BS2fNnG0JcNg83b95P0iHXOE%3D&reserved=0

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DA4EkxFDorfU&data=02%7C01%7Canna.anund%40vti.se%7Cd983fe2acc9346953ffc08d7b6543ce2%7C44f529522a66495880d4da9db72ad662%7C1%7C0%7C637178345267274427&sdata=qH7Ppk07K0BXQfJdGuUc8OH%2BhttZ%2BTPnv3ugDXx8FEM%3D&reserved=0

https://www.youtube.com/watch?v=O8xIQMo9TiU



The market and socio-economic analyses had an objective of demonstrating the economic

viability in the current uncertain society and it paved the way for the future exploitation. Some

market challenges were detected and analysed in other activities of the project (data and

security, legal issues and regulation and ethics). Fully analysed challenges were GPS

technology, safety, infrastructure connectivity and, especially important, economic aspects and

adoption of end users. The sources of information for the market and socio-economic analyses

were ERTRAC, JRC and other EU funded projects. The analyses showed valuable conclusions.

In economic terms, it is expected that Cooperative, Connected and Automated Mobility

provides profitable opportunities for sectors like automotive, electronics and software,

telecommunication, data services, digital media and freight transport. Sectors like insurance,

maintenance and repair are identified as businesses that might suffer important decreases in

revenues in the future, especially as a result of decreasing numbers of accidents. At a societal

level, a Cooperative, Connected and Automated Mobility could bring important safety and

productivity gains. Nevertheless, some important concerns exist, such as users’ acceptance,

ethics, social inclusion, and labour. ICT competences will be increasingly demanded in the

future, e.g. in manufacturing, maintenance and transport-related jobs. The skills required for

driving a vehicle will also change as automation gains full control of the vehicle, e.g. requiring

more supervision and selective skills.

During the project lifetime changes of data regulation at European level have appeared (i.e.

GDPR) and a Question & Answer has been developed including all knowledge gathered during

the project and useful information from project partners. As a conclusion of this activity, the

following key aspects were developed:

▪ GDPR took effect and additional considerations were added to data principles.

▪ Special focus on sensitive data: the knowledge of what is personal data is a key for an

effective business and exploitation plan.

▪ There are many differences between pilots and exploitation in the field of automated

vehicles.

The main objective of the ADAS&ME Advisory Board has been to supervise the project results

and key outcomes and support the dissemination of knowledge generated by the project. The

Board members are Jim Sayer (UMTRI), Ashleigh Filtness (Loughborough University), Olivier

Lenz (FIA Region I) and Masao Nagai (JARI). At the beginning of the project the Advisory

Board supported the selection of Use Cases at the Stakeholder workshop organized by

ADAS&ME. In addition, they reviewed and provided expert feedback on the project mid-term

results and the development of the systems. Finally, they validated the final project results at

the final project demonstration event in Barcelona.



8 Conclusions

ADAS&ME aimed to develop an Advanced Driver Assistance Systems that incorporate

driver/rider state, situational/environmental context, and adaptive interaction to automatically

transfer control between vehicle and driver/rider and thus ensure safer and more efficient road

usage for all vehicle types (conventional and electric car, truck, bus, motorcycle).

The work was built around seven use cases using in total five demonstrators. A generic system

architecture and HMI strategy were used and adapted to each use case. Driver/rider state

detection algorithms useful not only when driving manual but also during automation were

developed. The detection system was then the trigger for adapting the HMI solutions needed

for both manual driving and transitions (both driver/rider and system initiated). The knowledge

of the environment added information that made especially the takeover accurate and safe. The

seven use cases were evaluated, and driver state was manipulated if safety was possible to

guarantee. If not possible still the HMI was evaluated. The results where then used to assess

the impact and to identify new business and possible exploitation of results.



9 Contacts

9.1 Coordination team

Coordinator

Dr Anna Anund, VTI

Email: [email protected]

Technical manager

Dr Evangelos Bekiaris, CERTH/HIT


Innovation manager

Stella Nikolaou, CERTH/HIT


Quality manager

Dr Lena Nilsson, VTI


9.2 Dissemination manager

Marc Figuls, RACC


9.3 WP leaders

WP1 Dr Anna Anund, VTI Email: [email protected]

WP2 Sri Venkata Naga Phanindra Akula, TUC Email: [email protected]

WP3 Dr Karel Kreuter, DENSO Email: [email protected]

WP4 Dr Mathissen, Marcel (M.), FORD Email: [email protected]

WP5 Dr Frederik Diederichs, IAO Email: [email protected]

WP6 Stella Nikolaou, CERTH/HIT Email: [email protected]

WP7 James Jackson, IDIADA Email: [email protected]

WP8 Dr Eleonora Meta, CTL Email: [email protected]

WP9 Marc Figuls, RACC Email: [email protected]

WP10 Dr Anna Anund, VTI Email: [email protected]

9.4 Use Case leaders

UCA – Truck: Dr Stas Krupenia, SCANIA Email: [email protected]

UCB – Electrical vehicle: Kevin Nguyen, MOVEOTEC Email: [email protected]

UCC/D – Conventional vehicle: Marc Wilbrink, DLR Email: [email protected]

UCE/F – Motorbike: Davide Sette, DUCATI Email: [email protected]

UCG – Bus: Dr Anna Anund, VTI Email: [email protected]






















10 References

Anund, A., Solis, I., Rauh, N., Jackson, J., Ahlström, C., Abser, A., . . . Georgoulas, G. (2020).

Use ase Test results and consolidation (Deliverable 7.2 ADAS&ME EU project).

Retrieved from

Cocron, M., Vallejo, A., Delgado, B., Wilbrink, M., Anund, A., Krupenia, S., . . . Harous, C.

(2019). Evaluation frame work - ADAS&ME project (Deliverable 7.3 ADAS&ME EU

project). Retrieved from

Diederichs, F., Knauss, A., Wilbrink, M., Lilis, Y., hrysochoou, E., Anund, A., & Krupenia, S.

(2018). Adaptive Transitions for Automation in Cars, Trucks, Busses and Motorcycles.

IET Intelligent Transport Systems.

Dukic Willstrand, T., Anund, A., Strand, N., Nikolaou, S., Touliou, K., Gemou, M., & Faller ,

F. (2017). Driver/Rider models, Use Cases and implementation scenarios (D1.2

ADAS&ME EU project). Retrieved from

Figuals, M. (2020). Report on project dissemination and international cooperation activities –

2nd Update ADAS&ME (Delivearble 9.7 EU Project ADAS&ME). Retrieved from

Hennes, N., & Mathissen, M. (2020). Deliverable 4.2 Driver/rider state detection module

(Deliverabel 4.2 ADAS&ME EU project). Retrieved from

Meta, E., Shingo Usami, D., Azarko, A., Jackson, J., & Verschuur, S. (2020a). Deliverable 8.1

Impact Assessemnt (Deliverabel 8.1 ADAS&ME EU project). Retrieved from

Meta, E., Shingo Usami, D., Azarko, A., Jackson, J., & Verschuur, S. (2020b). Impact

assesement ADAS&ME (Deliverabel 8.1 ADAS&ME EU project). Retrieved from

Touliou, K., Maglavera, M., & Britsas, C. (2017). SoA and Benchmarking ADAS&ME (D1.1


Verschuur, S. (2020). Deliverable 9.4 Buisness and Exploitation Plan (Delivearble 9.4


Deliverable 10.4 Project Final Report€¦ · ADAS&ME (688900) D10.4 – Final project report...

Documents

Transcript of Deliverable 10.4 Project Final Report€¦ · ADAS&ME (688900) D10.4 – Final project report...