D3.4.1 - Two vehicle demonstrators for elderly drivers’...

Open architecture for Accessible Services

Integration and Standardization - G.A. 215754

CRF D3.4.1 – version. 2.0 CONFIDENTIAL

15/12/2010 of 63

OASIS – Open architecture for Accessible

Services Integration and Standardization

GRANT AGREEMENT # 215754

D3.4.1 - Two vehicle demonstrators for elderly drivers’ support

Deliverable No. D3.4.1

SubProject No. SP3 SubProject Title Autonomous Mobility and Smart Workplaces Applications

Workpackage No. WP3.4 Workpackage Title Personal mobility

Activity No. A3.4.2

A3.4.3

Activity Title A3.4.2 Driver telematic support services

A3.4.3 Driver comfort support services

Authors (per company, if more than one company provide it together)

Filippo Visintainer, Mirko Muro, Andrea Carlino CRF; Kostas Kalogirou, CERTH-HIT; Jordi Contreras, CTAG; Giovanni Pioggia, Gennaro Tartarisco, Marcello Ferro, UNIPI

Status (F: final; D: draft; RD: revised draft):

F

File Name: OASIS D3.4.1_v2.0.doc

Project start date and duration 01 January 2008, 48 Months




15/12/2010 of 63

Document History

Version

Date Comments Author

0.1 190710 Structure F. Visintainer, CRF

0.2 290710 Input on CRF demonstrator F. Visintainer, A. Carlino, CRF

0.3 290710 Integrated input by CERTH, CTAG,

UNIPI on Greek demonstrator

K. Kalogirou, HIT, G. Pioggia,

UNIPI, J. Contreras, CTAG

1.0 260810 Completed contents for Italian

demonstrator

F. Visintainer, A. Carlino, M.

Muro, CRF

1.4 010910 Completed all contents F. Visintainer, A. Carlino,

CRF, K. Kalogirou, HIT, G.

Pioggia, J. Contreras, CTAG

1.5 030910 Version for Peer Review F. Visintainer, A. Carlino, CRF

1.6 200910 Addition of paragraph 2.3 Giovanni Pioggia, Gennaro

Tartarisco, UNIPI

1.7 101110 Addition of contents as a result of

CRF and CERTH internal review

Andrea Carlino, CRF

Kostas Kalogirou, CRF

1.8 131210 Addition of sections 1.1.2, 1.1.3, 1.3

and 1.4. Finalisation of the

Document

F. Visintainer, CRF

2.0 151210 Final check F. Visintainer, Andrea Carlino,

CRF




15/12/2010 of 63

Executive Summary ........................................................................................................... 7

1. Italian pilot vehicle ...................................................................................................... 8

1.1. Demonstrator overview ...................................................................................................................................... 8 1.1.1. Introduction ..................................................................................................................................................... 8 1.1.2. CRF demonstrator in the OASIS context: state of the art and innovation ....................................................... 9 1.1.3. Working methodology ................................................................................................................................... 10

1.2. CRF demonstrator integration ........................................................................................................................ 11 1.2.1. Physical architecture ...................................................................................................................................... 11 1.2.1.1. OASIS Support PC ................................................................................................................................... 14 1.2.1.2. Vehicle Embedded Platform - VEP .......................................................................................................... 17 1.2.1.3. Mobile device (PDA) ................................................................................................................................ 18 1.2.1.4. Wearable sensors ...................................................................................................................................... 20 1.2.1.5. Touch screen (navigator) display .............................................................................................................. 22 1.2.2. On board installation ..................................................................................................................................... 23

1.3. OASIS connectivity .......................................................................................................................................... 24

1.4. Technical testing ............................................................................................................................................... 24

1.5. Demo scenario ................................................................................................................................................... 25

2. Greek pilot vehicle .................................................................................................... 27

2.1. Description ........................................................................................................................................................ 27 2.1.1. Main characteristics ....................................................................................................................................... 28 2.1.2. Summary of equipment.................................................................................................................................. 28 2.1.3. Sensors connectivity ...................................................................................................................................... 29

2.2. The Sensing Seat ............................................................................................................................................... 30 2.2.1. Sensing Seat technology ................................................................................................................................ 30 2.2.2. Static and dynamic characterization of conductive elastomeric sensor ......................................................... 30 2.2.3. Electrical model and acquisition electronic design ........................................................................................ 34 2.2.4. Electrical model and acquisition electronic design ........................................................................................ 34 2.2.5. Sensing Seat Experimental results ................................................................................................................. 35 2.2.5.1. Recording protocol and data analysis ....................................................................................................... 35 2.2.5.2. Sensing seat-based biometrics recognition entity ..................................................................................... 37 Enrolment..................................................................................................................................................................... 37 Recognition .................................................................................................................................................................. 37 Communication interfaces ........................................................................................................................................... 37

2.3. Personal driver fatigue safety system ............................................................................................................. 41

2.4. Architecture ...................................................................................................................................................... 46 2.4.1. USB connection architecture ......................................................................................................................... 46 2.4.2. Bluetooth wireless architecture ...................................................................................................................... 49

2.5. Equipment installation and demo scenarios ................................................................................................... 51 2.5.1. Seat sensors ................................................................................................................................................... 51 2.5.2. Blind spot detection ....................................................................................................................................... 53 2.5.2.1. BSD system overview ............................................................................................................................... 53 2.5.2.2. 24 GHz Radar ........................................................................................................................................... 54 2.5.2.3. BSD Processing Unit (ECU) ..................................................................................................................... 54 2.5.2.4. BSD connection ........................................................................................................................................ 55 2.5.3. Lane Departure .............................................................................................................................................. 56




15/12/2010 of 63

2.5.4. Frontal Radar ................................................................................................................................................. 57 2.5.5. OASIS application and vehicle speed ............................................................................................................ 59 2.5.6. Type of messages ........................................................................................................................................... 59

2.6. Impacts and further considerations ................................................................................................................ 59 2.6.1. General impacts ............................................................................................................................................. 59 2.6.2. Potentials for a future vehicle communication protocol ................................................................................ 60

Abbreviations ................................................................................................................... 62

References ....................................................................................................................... 63




15/12/2010 of 63

List of figures

Figure 1: CRF demonstrator vehicle ................................................................................................................................... 9 Figure 2: Physical architecture ......................................................................................................................................... 12 Figure 3: Overview of demonstrator components and their position ................................................................................ 13 Figure 4: Overview of the connections among the demonstrator components ................................................................. 14 Figure 5: eBOX is the OASIS Support PC ....................................................................................................................... 14 Figure 6: Fiat Blue&Me Board ......................................................................................................................................... 17 Figure 7: The PDA/Smartphone used in the demonstrator is the HTC HD2 .................................................................... 18 Figure 8: Wearable sensor used for health data measurement .......................................................................................... 20 Figure 9: The Touch Screen display used in the OASIS demonstrator ............................................................................ 22 Figure 10: Interior of CRF demonstrator, highlighting the main ...................................................................................... 23 Figure 11: PC installed on the car boot ............................................................................................................................. 23 Figure 12: Thesis 2.4 20V Emblema ................................................................................................................................ 27 Figure 13: Sensors‟ connectivity ...................................................................................................................................... 29 Figure 14: The static calibration curve. ............................................................................................................................ 31 Figure 15: Response of a CE sensor excited by step ........................................................................................................ 31 Figure 16: Electric model of a conductive elastomeric strain sensor ................................................................................ 32 Figure 17: Values of the parameters of the equivalent electric model extracted from ten cycles ..................................... 33 Figure 18: Response of a CE sensor excited by trapezoidal ramps in deformation .......................................................... 33 Figure 19: Sensing cover electrical model (A) and electronic acquisition front-end (B).................................................. 34 Figure 20: The Sensing Seat system prototype: ................................................................................................................ 35 Figure 21: SensingSeat authentication error: FAR and FRR (20 subjects, 6 actions, 6 repetitions per subject per action).

.................................................................................................................................................................................. 36 Figure 22: SensingSeat authentication error: mean and standard deviation of FAR and FRR along subjects (left) and

actions (right); 20 subjects, 6 actions, 6 repetitions per subject per action. .............................................................. 36 Figure 23: Modules of the sensing seat system................................................................................................................. 39 Figure 24: The NI-DAQmx API ....................................................................................................................................... 40 Figure 25: Seat-based biometrics. ..................................................................................................................................... 41 Figure 26: Multi-parametric monitoring system block scheme ........................................................................................ 42 Figure 27: sEMG transducers and pre-processing electronic board ................................................................................. 42 Figure 28: sEMG module position .................................................................................................................................... 43 Figure 29: Chest strap ....................................................................................................................................................... 43 Figure 30: ECG and activity module position .................................................................................................................. 44 Figure 31: Comparative ECG recordings ......................................................................................................................... 45 Figure 32: Chest strap dynamic ECG recordings .............................................................................................................. 45 Figure 33: Driver fatigue analysis model .......................................................................................................................... 46 Figure 34: Modules‟ physical connection ......................................................................................................................... 47 Figure 35: Communication between applications ............................................................................................................. 48 Figure 36: Nokia USB connection as a “COM” port connection ..................................................................................... 48 Figure 37: PC suite USB mode ......................................................................................................................................... 49 Figure 38: PC vehicle application; collects CAN signals and send to smart phone device .............................................. 49 Figure 39: Modules in the wireless approach ................................................................................................................... 50 Figure 40: The sensing seat within vehicle ....................................................................................................................... 51 Figure 41: Device‟s location (1) ....................................................................................................................................... 52 Figure 42: Device location (2) .......................................................................................................................................... 52 Figure 43: 24 Ghz Radar ................................................................................................................................................... 54 Figure 44: BSD ECU ........................................................................................................................................................ 55 Figure 45: Connection schema ......................................................................................................................................... 56 Figure 46: Camera housing glue to the windscreen .......................................................................................................... 56 Figure 47: Frontal radar .................................................................................................................................................... 58 Figure 48: Future in-vehicle protocol based on sensors‟ ECUs ........................................................................................ 60 Figure 49: OBD-II plug .................................................................................................................................................... 61




15/12/2010 of 63

List of tables

Table 1: Specifications of the support PC ......................................................................................................................... 15 Table 2: Type of information routed via Bluetooth .......................................................................................................... 16 Table 3: Vehicle OBU specifications ............................................................................................................................... 17 Table 4: PDA/Smartphone specifications ......................................................................................................................... 19 Table 5: Wearable sensors specifications ......................................................................................................................... 21 Table 6: Touch screen display specifications ................................................................................................................... 22 Table 7: Demo scenario .................................................................................................................................................... 26 Table 8: Basic vehicle characteristics ............................................................................................................................... 28 Table 9: EER calculated for every actions considering all the subjects. ........................................................................... 37 Table 10: UNIPI_SensingSeat API interface (external) .................................................................................................. 38 Table 11: BSD technical specifications ............................................................................................................................ 53 Table 12: Lane departure system‟s characteristics............................................................................................................ 57 Table 13: Frontal radar technical characteristics .............................................................................................................. 58




15/12/2010 of 63

Executive Summary

The current deliverable is an outcome of WP3.4 Personal Mobility [1], whose aim is to offer to the

elderly appropriate mobility services, including journey planning and support while driving,

accessible tourism, leisure and social activities. Specifically the activities A3.4.2 and A3.4.3 regard

the vehicle environment: A3.4.2 is dedicated to telematic support services for elderly users, such as

emergency call, in-car navigation, vehicle assistance and medical assistance; A3.4.3. is dedicated to

the driver monitoring, comfort and safety during the trip, responding to the elderly users

requirements of OASIS. The outcomes of these activities have are two car prototypes,

corresponding to the deliverable D3.4.1 Two vehicle demonstrators for elderly drivers’ support.

This document is an accompanying report illustrating the prototypes, and it is therefore structured in

two main chapters, dedicated to the Italian pilot demonstrator and Greek pilot demonstrator

respectively.

The Italian pilot vehicle (A3.4.2) has been developed and set-up by CRF, and provides the elderly

users with telematic assistance services (E-Call, medical and vehicle assistance, navigation) based

on the OASIS framework. The main topics faced in the activity concerned the wireless connectivity

among different subsystems into the vehicle, the integration of OASIS platform, the Human

Machine Interface and the OASIS-assisted navigation based on elderly friendly route guidance.

The Greek pilot vehicle (A3.4.3) is provided by CERTH-HIT and CTAG and integrates several

sensing subsystems as well as the OASIS application. Here, beyond the OASIS adaptation of

several sensing systems, the main task concerned the integration and in-vehicle testing of the

sensing seat, including the wearable driver fatigue safety, provided by UNIPI and its subcontractor.

In addition to commercial components, some prototype components have been provided by OASIS

partners and properly adapted/interfaced to OASIS, among which UNIPI health sensors and sensing

seat and PTV on board navigator. Further cooperation with partners will happen in the cross-

scenarios, for instance the one involving in-vehicle navigation (WP3.4) and pedestrian navigation

(WP3.3).

Rather than going into software details (reported in the future D3.4.2), the description of each

system illustrates the main objectives, the system architecture and components in terms of main

characteristics and functionalities, the technology highlights, and finally the demo scenarios. For

completeness sake, also the component datasheets and the installation overview have been reported.




15/12/2010 of 63

1. Italian pilot vehicle

1.1. Demonstrator overview

1.1.1. Introduction

From the end-user point of view, in-vehicle services offered by CRF demonstrator can be

summarized as follows

o Emergency call

o Medical Assistance

o Remote Vehicle assistance

o In-vehicle navigation

Emergency Call – This service allows contacting the nearest 112 emergency centre [2]. This will

be done autonomously as soon as an accident or an emergency event occurs, or alternatively it can

be started by the driver, employing the same E-Call device. In both cases, this will lead to a voice

connection, which is an actual E112 call, between the crashed vehicle and the emergency operator,

and to the automatic transmission of data containing useful information about the vehicle, to the

rescue centre. The service should comply with the basic pan-European E-Call specifications

(Minimum Set of Data), as reported in Annex1, and it will be enriched with data on elderly user

profile and medical information (Full Set Of Data). Though the service will have a direct

connection to PSAP, OASIS service centre(s) will also be informed (with another, lower priority

call) so that they can exchange more information with the involved PSAP if needed. The latter

mechanism, i.e. the cooperation between service centres, is out of the scope of in-vehicle services

(A3.4.2).

Health monitoring – This service offers health monitoring and medical assistance to the driver. If

the driver is provided with health sensors (and this will be tested in the demonstrator) his/her health

status is monitored by the in-vehicle system, and correlated with driving condition, to infer a

general condition with respect to the current or future driving tasks.

In safety-uncritical conditions (with respect to driving tasks) this service will not generate any

warnings, but respond to user requests; it shall be capable of displaying his health parameter values

in real time, of searching the nearest sanitary structures (hospital, pharmacies), of connecting to the

latter (voice, data), of advising whether to undertake trips or not depending for instance on the trip

length, user profile and time of the day. All these functionalities, excluding the health parameters

display, will likely be based on a connection to a remote centre.

In safety-critical conditions this service will either warn the driver or provide him suggestion, or

even call the local Emergency Call service for the generation of an E-Call.

Remote Vehicle assistance – This service allows searching for and connecting to a point of

assistance for the vehicle: a garage, a road administration service centre, a traffic information

service, the nearest gas station, the closest tow truck service, etc. Depending on the kind of problem

with the vehicle (monitored through the CAN network) this service will provide the most suitable

assistance service, let it be the rescue fleet of the road providers, or the nearest garage or gas station.

This selection is supposed to be based also on user‟s profile. For instance it has to take into account

the uneasiness of very old drivers whenever a system malfunction happens in rural, isolated places.




15/12/2010 of 63

Navigation – This service provides a navigation service targeted to elderly Users. In terms of

browsing option it will be designed in analogous way to standard navigation, but the service it

provides is specific for OASIS: the system will connect to an elderly friendly off-board routing

service, in order to receive the waypoints of the most indicated route. The waypoints will be

automatically fed to an on board navigator, the latter being piloted by the OASIS system.

Furthermore, the service allows to find POIs through the OASIS platform. In terms of HMI, the

Navigation service may be called directly by other services (Health Monitoring, Remote Vehicle

assistance) whenever the user needs to reach a destination.

Figure 1: CRF demonstrator vehicle

1.1.2. CRF demonstrator in the OASIS context: state of the art and

innovation

CRF demonstrator is meant to prove the integration of existing automotive ICT components within

OASIS framework, as well as a few innovative features with respect to the state-of-the art. The

integration of in-vehicle systems with OASIS concerns the following aspects.

o The OASIS framework (in cooperation with CERTH), i.e. the integration of Ambient

Intelligence (AmI) to connect to the OASIS middleware and obtain the best service provider

o The elderly friendly HMI, using a common User Interface component set in cooperation

with FORTH and based on FhG/IAO guidelines

o The in-vehicle navigation availing of the OASIS service Elderly Friendly Route Guidance of

PTV

o The provision of vehicle position information for easy car localization, used in cross

scenarios, e.g. in PTV pedestrian navigation to go back to the parked car.

The project gave to CRF the opportunity to introduce innovations with respect to the state-of art,

mainly in the field of telematics. Although the connectivity via Bluetooth was already existing, the




15/12/2010 of 63

following functionalities – enabled by the connectivity itself – have been created ad-hoc for OASIS

and are rather significant from an OEM exploitation viewpoint.

o The capability of browsing and pushing the virtual keys displayed on the PDA via the

physical steering-wheel keys

o The extension and adaptation of vocal interface, enabling to browse the OASIS application

menu and perform a subset of the services (E-Call, Health Monitor, start/stop navigation)

o The limitation of the PDA User Interface with vehicle in motion (for example disabling

some buttons if the speed is above a certain threshold)

o The connectivity between the on board unit and external sensors, such as health sensors,

allowing to use both health data and vehicle data for extended E-Call and health monitoring.

o A Bluetooth gateway application, needed to connect different Bluetooth peripherals,

exploitable as software component for several purposes

Finally, while the pure on board navigation and pure off board navigation functionalities are already

existing, CRF utilised a mixed on-off board navigation approach, in which off board calculated data

are given as input to an on board navigator. Although this aspect introduces extra processing, it is a

good approach if we consider the integration of existing components (e.g. existing on board

navigators) with novel remote-based services (OASIS off-board services).

1.1.3. Working methodology

The nature of the project, involving several applications integrated into a general OASIS

framework, suggested to take particular care for an effective work in parallel, without blocking or

creating too many dependencies. While the technological choices and adaptations of the vehicle

system had to be designed, developed and tested building extension on existing solutions (e.g.

Bluetooth connectivity, data gathering tools) at the same time, the services had to be defined within

the OASIS consortium, with the final goal of using the OASIS-dedicated tools (e.g. AmI, HMI

Widgets) which in turn would need their time to be developed. All this had to be done in parallel

with reiterated testing on the HMI, thanks to the User Interface partners (e.g. FhG/IAO) assessing

CRF concepts on elderly user groups. For the Italian demonstrator, the approach thus consisted in a

multi-step development, foreseeing already from the beginning the use of temporary schemes and

solutions without being afraid of modifying, porting and improving the results during the activity

course. A similar approach has been adopted for the Greek demonstrator.

Hereafter the work phases of the Italian demonstrator set-up are briefly illustrated. After the user

needs and requirements definition, the in-vehicle architecture was outlined, utilising as much as

possible the functionality of the Lancia Delta telematic unit (Blue&Me, called also Vehicle

Embedded Platform or VEP) with the aid of a nomadic device (the model that is used also outside

the car); even though an additional support PC was needed, all functionalities were developed in a

way that they can be ported with ease in future automotive platforms.

Following this scheme, a first version of the prototype, aimed at verifying with the consortium and

the Commission the soundness of the main concepts to be developed, was issued in February 2010

for the 2nd

OASIS review: this version demonstrated the main features of the OASIS in-vehicle

services such as Bluetooth connectivity, user interaction, navigation, but it was not connected

remotely and not integrated into OASIS framework; all software components were developed in C#,

the language for Blue&Me applications. By that time the User Interface had been structured and

validated on users in paper version, and a first issue of HMI application on nomad device was

available and integrated also with the steering wheel and vocal commands.

After the concept was validated at the review, the integration with the OASIS framework took place

(see previous section for details); some logical components had to be ported in JAVA and




15/12/2010 of 63

integrated into the OASIS framework, to obtain the final prototype. In parallel, the HMI was

assessed for a second iteration at FhG/IAO, this time with a running software, to receive further

guidelines for the layout. These have been implemented in the final HMI application, maintaining

the same structure for the in-vehicle interaction, but now being part of an integrated OASIS HMI

structure which enables the access to other services, and utilising FORTH widget libraries as

development tool.

1.2. CRF demonstrator integration

1.2.1. Physical architecture

In the following UML scheme (Figure 2) the integration of the software module in the demonstrator

hardware components is shown.




15/12/2010 of 63

Figure 2: Physical architecture

The overall telematic assistance system is composed of three devices (large boxes in the picture)

whose main features are summarised below, the details being reported in the next sections.

o Mobile Device (PDA)

Integrates the OASIS framework

Provides connectivity and interoperability with OASIS world (POI, Off Board

Routing, Driver Assistance)

Handles a subset of User Profile data




15/12/2010 of 63

Hosts and is interfaced to SP3 applications

Implements Visual User Interface

o OASIS support PC

Implements main application, i.e. decision manager

Is the center of telematic system network

Is interfaced with wearable health sensors

Hosts PTV On Board Navigation

o Vehicle Embedded Platform (VEP)

Is the embedded platform of the Lancia Delta (Blue&Me)

Is the gateway to vehicle data

Is interfaced with vehicle HMI peripherals (microphones, steering wheel keys,

dashboard display)

Implements vocal interface

Performs E-Call

Figure 6 shows how the components are placed in the vehicle. It‟s worth highlighting that the only

extra elements of this prototype with respect to ordinary vehicles are represented by the Support PC

in the vehicle boot and the aftermarket Touch Screen display.

Figure 3: Overview of demonstrator components and their position

Figure 7 shows the connectivity among components. As can be seen all devices (excluding CAN

and display) are connected wirelessly via Bluetooth, in a star network centred on the OASIS support

PC.




15/12/2010 of 63

Figure 4: Overview of the connections among the demonstrator components

1.2.1.1. OASIS Support PC

This component is an industrial computer whose characteristics are reported in Table 1.

Figure 5: eBOX is the OASIS Support PC




15/12/2010 of 63

OASIS Support PC features

Processor - CPU Socket P for Intel® Core™2 Duo/Celeron® M up to 2.2 GHz

Processor

System Memory - RAM 2 x 240-pin DDR2 DIMM max. up to 4GB

Chipset Intel® GME965+ICH8M

BIOS Phoenix-Award 16Mbit (SPI) with RPL/PXE LAN Boot ROM,

SmartView and Customer CMOS backup

EOS Support XPE, WinCE, Linux

System I/O Outlet 3 x RS-232 (COM 2/3/4)

1 x RS-232/422/485 (COM 1)

1 x VGA

1 x Audio (Mic-in/Line-out)

1 x PS/2 keyboard

1 x PS/2 mouse

2 x 10/100/1000Mbps Ethernet

6 x USB 2.0

1 x VDC power input connector

Watchdog Timer 255 levels, 1~255 sec.

Storage Supports 1 x 2.5" SATA/IDE HDD drive bay

Expansion Interface 1 x 32bit/33MHz PCI slot (max. length 200mm)

1 x PCI Express x16 slot

(max. card length 167mm, height 14.4mm, backward

compatible with x1, x4, x8)

Graphic Card

Serial Ports

Power Supply VDC 10-30V, 150W ATX power supply

Construction Aluminum extrusion and heavy-duty steel

Operating Temperature -10°C to 40°C (-14°F~104°F)

Storage Temperature -20°C~80°C (-4°F~176°F)

Humidity 10%~90%, non-condensing

Vibration Endurance 1Grms (5-500Hz, X, Y, Z directions) operation w/ HDD

Dimensions 182mm (W) x 230mm (D)

x 140mm (H)

Weight (net/gross) 4.5 kg (9.92 lb)/5.0 kg (11.02 lb) (w/o HDD)

Table 1: Specifications of the support PC

Within the computer, the PC Main App, based on health sensors data, vehicle data, external data

(from OASIS world or from other SP3 applications) and User Input, has the main task of triggering

specific actions based on the health and driving situation or based on user requests. The kinds of

action triggered are:

o Driver warning

o Enabling and disabling of HMI functionalities

o Emergency call

o Data exchange with OASIS world

o Start/stop/update on board navigation

o Local data exchange with SP3 applications

The PC Main App includes three main submodules:

o bluetooth gateway

o interface to on board navigation

o Remote Service Interface Module

The bluetooth gateway routes the data among the several peripherals (PDA, VEP, PC Main App

inside the PC and Health sensors). The following kind of data exchange is foreseen:




15/12/2010 of 63

Receiver

Sender

Support PC Vehicle Embedded

Platform

(VEP)

Mobile device

(PDA)

Health Sensors

Support PC E-Call trigger

Processed data for E-

Call (health and

vehicle data)

Driver Warning

trigger

Driver Warning

trigger

HMI Enable and

disable trigger

Steering Wheel Keys

selection

Processed data

(health, vehicle) for

remote OASIS

services

Processed data for

local OASIS

applications

Vehicle Embedded

Platform

(VEP)

Raw vehicle data

(speed, airbag status,

…)

GPS data ( timestap,

position, speed,

heading, satellite

count and sea level

height, …)

HMI/ User input data

(Steering wheel keys

, voice commands,

…)

Driver distraction

evaluation (based on

vehicle dynamics)

Power Notifications

and commands for

the Car Navigator


Mobile device

(PDA)

HMI /User input data

Data from OASIS

remote services

Data from local

OASIS applications


Health Sensors Raw health sensor

data

Table 2: Type of information routed via Bluetooth

The interface to on board navigation provides data received from the off-board elderly-friendly

routing service (in the OASIS world) to the on board navigation and starts/stops the latter, using the

libraries provided by PTV.

Finally, a specific submodule, called Remote Service Interface Module, is dedicated to cases

involving simultaneously emergency calls and data exchange with OASIS world. In fact, it is




15/12/2010 of 63

responsible for aligning information contents provided to remote services internal to OASIS (e.g.

subscribed service providers) with external services (e.g. PSAP in a direct emergency call).

1.2.1.2. Vehicle Embedded Platform - VEP

The Vehicle Embedded Platform is based on Fiat Blue&Me. Fiat Auto and Microsoft have

produced “Blue&Me”, a system based on Windows Mobile for Automotive, that conjugates safety

and comfort, and introduces novel standards of in-car infotainment. Blue&Me is based on an open,

updatable system with modular contents that can be associated with new services in the future.

Figure 6: Fiat Blue&Me Board

The main specifications of the Fiat Blue&Me Board are reported in the following table.

Fiat Blue&Me board

Processor -CPU ARM9 at 300MHz

Memory 32NAND flash, 32MB SDRAM (optionally 64MB flash, RAM)

Stand-by current consumption 1.5 mA

Support for Vehicle IO 1 CAN interface (optional 2nd and 3rd interface)

K-Line (optional)

R-Ladder

A/D and GPIO

Bluetooth module

Audio interface for microphone, phone, and MP3

USB host connection

GPS receiver

Phone Module

Other Noise and echo cancellation in an FPGA

Software highlights

All underlying drivers for hardware listed below

Windows Mobile for Automotive Middleware and application framework

Device management tools for over-the-air updates

Bluetooth hands-free phone profile

USB port for digital music and device programming at dealerships

Power management system

Server connectivity support

Speech software for managed voice-command features that support nine languages

Physical Dimensions

Size 182mm (W) x 140mm (D) x 40mm (H)

Weight ~ 500 gr

Table 3: Vehicle OBU specifications




15/12/2010 of 63

Within the VEP, the OASIS VEP App is the main software module, which has the following roles:

o Pre-processing of vehicle data (from CAN and from GPS positioning module, respectively)

and delivery of pre-processed data to the support PC via the Bluetooth Communication

module.

o Vocal User Interface in vehicle, utilising the Voice Recognition and Text-To-Speech engine

of the Blue & Me.

Vocal output is given for each step of the interaction, based on the HMI flowchart

User voice input is recognised comparing it to a set of keywords, which is loaded

according to the HMI flowchart

HMI flowchart information is synchronised with the PDA based on HMI data

exchanged via Bluetooth with the PDA (Table 2).

o Tactile User Interface in vehicle, reading and interpreting User input from keys. Information

is synchronised with the PDA based on HMI data exchanged via Bluetooth with the PDA

(Table 2).

o Generation of E-Call. Based on trigger and data received by the OASIS Support PC, the

emergency call is performed through the GPRS/GSM communication module in the VEP.

1.2.1.3. Mobile device (PDA)

The mobile device is supposed to be the OASIS assistant in the displacements of Elderly Users,

providing pre-trip information services, pedestrian navigation and the gateway to OASIS world

when the Elderly User is in the car (or walking outside). The device has been selected in agreement

with SP3 and SP4 partners, and will integrate other SP3 applications: specifically the demonstrator

owned by CRF and used in the Italian test site, beyond the WP3.4 components reported hereafter,

will also integrate pedestrian navigation (WP3.3).

Figure 7: The PDA/Smartphone used in the demonstrator is the HTC HD2




15/12/2010 of 63

PDA/Smartphone

Battery Lithium – 1230mAh

Talk time WCDMA: Up to 320 mins

GSM: Up to 380 mins

Standby time WCDMA: Up to 390 hours

GSM: Up to 490 hours

Video playback Up to 8 hours

Audio playback Up to 12 hours

Processor – CPU Processore Snapdragon™ 1GHz

ROM Memory 512 MB

RAM Memory 448 MB

Operative System Windows Mobile 6.5 Professional

Screen Size 4,3‟‟

Resolution 480 × 800 WVGA

Touch Screen Capacitive

Connectivity

HSPA/WCDMA (Europe)

900/2100 MHz

GSM (Europe) 850/900/1800/1900 MHz

3G

Up to 7.2 Mbps - download

Up to 2 Mbps – upload

GPRS: Up to 114 kbps – download

EDGE Up to 560 Kbps – download

Wi-Fi IEEE 802.11 b/g

Bluethoot 2.1 with Enhanced Data Rate

USB 5-pin micro USB 2.0

Other

Sensors G-sensor

Proximity sensor

Environmental light sensor

GPS receiver

Digital compass

Physical Dimensions

Size 67mm (W) x 120.5 mm (D)

x 11mm (H)

Weight 157 gr (with battery)

Table 4: PDA/Smartphone specifications

Concerning WP3.4, two main software modules are implemented: the PDA GUI module and the

OASIS PDA App. Three more general modules are: the Profile Manager, the Medical Assistance

and the GPRS/GSM Communication Module.

The PDA GUI module provides the visual interface to the Elderly User, allowing him/her to access

OASIS services described in the introduction. The visual HMI is structured according to specific

flowcharts reported in ID3.4.1, and is coherent with the vocal and tactile HMI, thanks to the HMI

data exchanged with the VEP via Bluetooth gateway. As an example, in the main menu the user can

either

o select the HMI keys via the touch-screen display

o scroll across the buttons of the application home page via the up/down keys of the steering

wheel, and perform a selection via the OK key

o say one of the words from the keyword menu which is listed by the Blue&Me voice.




15/12/2010 of 63

The OASIS PDA application is responsible of the complete control of the PDA. In particular it

manages directly the PDA GUI Module, the Profile Manager and the Medical Assistance. It

receives the requests for the management of the warnings given to the driver and the messages for

enabling or disabling the HMI and gets the processed data regarding the vehicle and the health of

the driver for the remote OASIS services and some other processed data for local OASIS

applications.

The Profile Manager is the responsible for the management of the profile data and the

synchronization between the local copy and the remote one: whenever a profile preference changes

on the PDA, it should be updated on the remote OASIS profile, in order to make it immediately

available for all the applications which need it. Moreover, whenever a local application needs a

particular profile parameter the Profile Manager has to request it to the Profile Repository of the

OASIS Service Provider, in order to be sure to have its last updated value.

The Medical Assistance should give the possibility to contact the OASIS Service Provider in case of

health assistance need and to provide it with all the possible information about the location of the

vehicle, the status of the vehicle and all the useful data in case of an accident. Furthermore, all the

health data of the driver should be attached. The call to request assistance is performed through the

GPRS/GSM Communication Module.

The GPRS/GSM Communication Module is the module that enables the connection between the

vehicle and the OASIS Service Provider. It is controlled by the Medical Assistance and the Profile

Manager modules, for E-Call generation and for the profile data exchange between the vehicle and

the OASIS Service Provider.

1.2.1.4. Wearable sensors

The wearable sensor is an external module interfaced to the in-vehicle system. It has been provided

for testing purposes to CRF by UNIPI-CNR [3] . It comes from SMARTEX-CSEM prototype of

MY HEART project. The sensor integrates provides via Bluetooth to the OASIS support PC in the

car the following parameters: a full electrocardiogram, the heart rate, respiration rate and 3-D

acceleration.

Figure 8: Wearable sensor used for health data measurement




15/12/2010 of 63

Wearable sensor features

Signal Inputs

ECG

Input sensitivity ±5mV

Frequency range 0.05Hz to 30Hz (depending upon software filter

configuration)

Signal resolution Equivalent to 12 bits

Respiration

Input type

DC resistance measure, constant current or constant

voltage

Input range 100KΩ to 1MΩ

Frequency range DC to 10Hz

Signal resolution Equivalent to 12 bits

Activity

Sensor

3D Accelerometer

Sensor range

±2g or ±6g

Frequency range

DC to 10Hz

Signal resolution 12 bits

Signal processing

ECG • Heart rate

• Signal quality

• Enhanced ECG signal

• R-R intervals

• Heart rate variability

Respiration • Breath rate

• Breath amplitude

Activity • Energy

• Classification

Interfaces

Wireless Bluetooth 2.0, class 1/2/3

Serial port profile

Wired USB 2.0 FS

Storage

Memory micro-SD card, 1GB

Autonomy

Battery Lithium-polymer, 340mAh

Charge through USB

Autonomy while recording Up to 19 hours

Autonomy while streaming Up to 8 hours

Stand-by time Up to 28 days

Memory capacity (with 1GB) Up to 9 days continuous logging

Device

Dimensions 63 x 52 x 14 [mm3]

Weight 42 gr

Table 5: Wearable sensors specifications




15/12/2010 of 63

1.2.1.5. Touch screen (navigator) display

The following aftermarket touch screen display has been connected to the OASIS Support PC. It

was chosen among those models usually deployed for R&D activities.

The display is used to show the on-board navigation route, but it can be used by the researchers for

testing purposes (e.g. Bluetooth connectivity verification, sensor system data check, etc.).

Figure 9: The Touch Screen display used in the OASIS demonstrator

Touch screen (navigator) display

Power Supply DC 12V

Operating Voltage DC 11-13 V

Power consumption ≤ 8W

Screen Size 7‟‟

Contrast ratio 500:1

LCD brightness 300cd/m2

Supported resolution 800 × 480, 16:9 wide screen format

AV input VGA and composite video

Audio output Speaker built-in, 100mW

Touch screen interface USB interface for use with PC Composite video and audio

Viewing Angle 45/70(U/D),70/70(L/R)

Size 188W x 125H x33D mm

Weight 0.495 (Kg)

Table 6: Touch screen display specifications




15/12/2010 of 63

1.2.2. On board installation

The following pictures show the on board installation. In a possible future industrialization, the on

board display could be integrated beneath the current position, in a DIN module close to the radio.

The PC installed on the boot (Figure 11) is already very small but in the future it should become an

embedded component underneath the dashboard (bottom right in Figure 10) where the Blue&Me is

now placed.

Figure 10: Interior of CRF demonstrator, highlighting the main

peripherals. On the right, wearable health monitoring T-Shirt is shown

Figure 11: PC installed on the car boot




15/12/2010 of 63

1.3. OASIS connectivity

OASIS connectivity is performed via GPRS and/or GSM network. Remote service invocation

happens on user‟s request or automatically in critical cases. In case of emergency call, a direct call

to the Public Safety Answering Point is foreseen and performed through a vocal call on the GSM

network, if the user can talk to the PSAP operator, and through GPRS data call; the OASIS call is

performed after the latter, for synchronisation purposes. Though the type of request varies from

service to service, the following classification of remote service invocation can be done:

1. Ask for assistance: the user asks for remote assistance, the system returns a service and

autonomously connects to it, providing a set of data to the service and then performing a

voice call displaying the provider information (type, name, location, phone number, etc.)

2. POI search: the user asks for a Point Of Interest of a certain type and the remote system

returns a set of POIs in the vicinity; the user can select one of them, read relevant

information and decide whether to make a phone call or navigate to it.

3. Elderly Friendly Route Guidance: the user asks the route for a particular destination; the

remote system returns the route waypoints, calculated based on user‟s profile (safe/calm

route) which are then utilised by the on-board navigator.

The connectivity is managed by the Mobile device, as described in section 1.2.1, through the AmI

framework, in compliance with the OASIS guidelines.

1.4. Technical testing

During the development phase and then after the demonstrator was set-up, a series of technical tests

was performed, in order to verify the correct functionality of each subsystem. Being a functional

testing performed just after each component was developed or integrated, it followed the general

work methodology outlined in section 1.1.3. Furthermore, following a consolidated approach, the

testing typically took place first in the laboratory (having the relevant components working on a

development desk) then in the vehicle in static conditions, finally in the vehicle in dynamic

conditions. In particular the following functional tests have been performed.

Bluetooth connectivity. Test the correct data transfer

between mobile device and support PC

between health sensors and support PC

between Vehicle Embedded Platform and support PC

Human Machine Interface (layout) performed by FhG/IAO

1st iteration with elderly users, utilizing HMI printscreens on paper

2nd iteration with elderly users, utilizing software application on PDA

Human Machine Interface (software) performed by CRF

Visual HMI on PDA, standalone

Correct functioning of 1st version application in C#

Correct functioning of 2nd

version application in JAVA

Vehicle keys, standalone

Commands given via steering wheel keys

E-Call performed via dedicated key

Capability of browsing and selecting PDA commands through steering wheel keys

Vocal HMI, standalone




15/12/2010 of 63

Interpretation of commands via Voice Recognition and correct performance of the

related functions

Capability of providing commands to the PDA

Feedback to the driver via text to speech

Integrated HMI: capability of providing command and browsing through the HMI options in

3 ways, namely vocal, vehicle keys and touch-screen PDA,

in single mode (e.g. only vocal)

in multi-mode (e.g. vocal, then steering key, then touch screen)

HMI disabling policies with vehicle in motion

Route guidance

Check correct functioning of on-board navigator, specifically

the provision of single coordinates from the Vehicle Embedded Platform to the navigator

the provision of addresses from the OASIS application to the navigator, and the return of

geo-coded information from the navigator to the OASIS application

the provision of a set of waypoints from the OASIS application to the navigator

the capability of starting and stopping the navigation via the OASIS application

Test the complete service chain, with waypoints stored locally (not connected to the remote

centre) and the possibility of selecting safe and calm route from the settings

Key-off functionality

Check correct storage of vehicle position on the PDA upon key-off operation

Currently the connectivity via GPRS between mobile device and OASIS service centre is being

tested. A further testing activity is also planned for the piloting phase, in the Italian Test-Site of

Florence.

1.5. Demo scenario

Hereafter the demo scenarios are presented in a step-by-step description, allowing to demonstrate

the system core features.

Step Title Concept to be proven Highlights

1 In-vehicle system

overview

Explanation of vehicle architecture and

deployment choices.

2 Entering the car The user enters the car. The PDA is connected to

the on board system via Bluetooth and OASIS in-

vehicle application is on.

PDA connectivity.

3 In-car services

overview

The user is in front of the main menu on the PDA.

The different functionalities will be illustrated.

Here, browsing by graphic interface, vocal

interaction and dashboard-button will be shown.

HMI can be browsed in different

ways (vocal, steering wheel

buttons, PDA touch screen )

4 Emergency Call E-Call: activation by graphic interface, vocal

interaction, car E-Call-dedicated-button.

Different ways to start emergency

call; data that are sent to PSAP

(simulated) and to OASIS.

5 User Profile From the main menu the user enters Settings.

Here, the changing of HMI layout will be shown.

When the User Selects Visual Impairment, the

screen appearance will change to high contrast

text and icons on a dark background. Optionally

the user can select calm or safe route.

In the OASIS system these

features may be transparent to the

user. Anyway a demo version will

show the profiling of services

(e.g. layout of HMI, service

configuration, etc.).

6 Health Monitor From the main menu, the user enters the option Health assistance Use Cases.




15/12/2010 of 63

(inside Health

Assistance)

Health Assistance. In the next screen the user‟s

health parameters will be shown, with simulated

values.

7 Find Hospital

(inside Health

Assistance)

The user selects the option to find a hospital, and

the off board system search for an hospital

through OASIS. The system returns a Hospital.

POI search through OASIS

8 Routing to

hospital

The user wants to find the route to the hospital. A

route calculated off board via OASIS is provided

to the on board navigator.

Hybrid on-off board navigation

9 Start navigation The user selects to navigate to the hospital. The

interface with on board navigator (OBN) is

demonstrated: the OASIS system will pass the

waypoints to the OBN and will command the

latter to start the navigation.

The OASIS application can

command the on board navigator

10 During the trip The HMI will change when the vehicle is moving:

specific options will be disabled. The enabled

options are still selectable through the steering

wheel buttons and vocal interface.

Steering wheel buttons browsing

and selection will be the focus of

this part.

11 Remote Vehicle

Assistance

A malfunction of the system is emulated. A

warning pops up and then proposes the user to

contact or go to a vehicle assistance provider

Semi-autonomous vehicle

assistance through vehicle

sensors and user interaction

12 Stop at parking

place

The user will stop and turn the key off. When this

operation is performed, a message will pop up on

the PDA screen reporting that the last GPS

coordinate has been saved and is available.

System functionality of storing

last car position

13 Return to

parked car

(WP3.3 scenario)

User selects to go back to parked car. The

position of the car is utilized by the pedestrian

navigation service (reference to test site).

Cross scenario with PTV

14 Re-Entering the

car

The user enters the car. The PDA is connected to

the on board system via Bluetooth and OASIS in-

vehicle application is on again.

Easy switching between

pedestrian scenario and car

scenario

Table 7: Demo scenario




15/12/2010 of 63

2. Greek pilot vehicle

2.1. Description

The vehicle demonstrator that will be used for the Greek pilot site is the Lancia Thesis 2.4 20V

Emblema and it is owned by CERTH/HIT. This research car contains several systems that have

already used and validated in several EU research projects (e.g.: IM@GINE-IT, AIDE, IN-

SAFETY, SENSATION, ASK-IT). These systems can provide information regarding the driver‟s

actions and the vehicle‟s operations.

Most of the sensors and actuators are controlled by ECUs. The industrial PC residing at the trunk of

the vehicle acquires and reads the values of all sensors‟ signals.

Figure 12: Thesis 2.4 20V Emblema




15/12/2010 of 63

2.1.1. Main characteristics

Its main characteristics are depicted below.

Model Lancia Thesis 2.4 Emblema

Engine 5 cylinders 2446 cm3 Euro4

Motor fuel Petrol

Power 125KW (170HP) at 6000 rpm

Max Torque 226 Nm @ 3500 rpm

Traction Front

Gear Manual 6 gear

Max Speed 217 Km/h

Dimensions 488 x183 x147cm

Weight 1760 Kg

Equipment ABS+EBD

ASR

EPS (Electronic Stability Program)

EPB (Electric Parking Brake)

Electronic Controlled Shock Absorber Rain-sensor

6 Airbags

Multi-zone Climate control

Connect system (navigation system + GSM + radio

+ CD charger)

Hi-Fi Bose sound system

Table 8: Basic vehicle characteristics

2.1.2. Summary of equipment

The vehicle includes the following systems:

a. Basic electrical equipment to provide safe power supply to all additional equipment such as

auxiliary battery, fuses, relays, switches and lamps to configure and monitor the system.

b. An additional electronic unit (gateway) that retrieves information from the vehicle (CAN buses).

c. Many vehicle signals such gas, brake pedals position, longitudinal speed/acceleration, yaw rate,

steering angle, lights status, wiper status, external temperature are available through the CAN bus

protocol.

d. A frontal ultrasonic radar to acquire information about the leading vehicle (distance, relative

speed, obstacle‟s measurements); again, these data are available through CAN bus network.

e. An industrial PC for data acquisition and real-time data processing. It contains:

Main unit (case main-board, video card, LAN and CAN card)

LCD Monitor, keyboard and mouse installed in the rear seat for the technical operator

CAN PCI card

Application software for data acquisition from all the sensors, recoding, display and

processing.

A small (7”) LCD VGA screen installed over the vehicle dashboard to display data to the

driver.

f. Lane Departure Camera System; its messages are published via CAN

g. Vibration system; it is used for enable vibration warning to driver. It is located in the seat belt.




15/12/2010 of 63

h. An auxiliary battery which is used by all the additional electronic units within car such as

sensors, industrial PCs, ECUs, LDC monitor, ACC and LDW.

2.1.3. Sensors connectivity

The following diagram shows the system connectivity: all the devices (sensors and actuators)

expose their signal through CAN bus network. The industrial PC acquires all the corresponding

signals.

Figure 13: Sensors’ connectivity




15/12/2010 of 63

2.2. The Sensing Seat

2.2.1. Sensing Seat technology

The UNIPI module is an unobtrusive and versatile sensing seat system for human authentication

that can be employed in different scenarios such as truck and car pilots, airplane pilots, plant and

office personnel, and, in general, environments where the security is mandatory and a soft seat is

available. It is an anthropometric system based on pressure sensors integrated in seats, in order to

enhance the security and reliability of the other biometric system but also increase its applicability

to scenarios where the physiological profile of an individual cannot be obtained.

The sensing seat is realized by a seat coated by a removable Lycra sensing cover. The sensing cover

is able to respond to simultaneous deformations in different directions by means of a piezoresistive

network which consists of a mixtures of polymers deepened with coal directly printed onto the

fabric. The strain sensors developed by the University of Pisa are realized by means of Conductive

Elastomers (CE) composites. CE composites show piezoresistive properties when a deformation is

applied and can be easily integrated into fabric or other flexible substrate to be employed as strain

sensors. The used CE is based on a WACKER Ltd (Elastosil LR 3162 A/B) product. It consists in a

mixture of graphite and silicon rubber. WACKER Ltd guarantees the non-toxicity of the product

that, after the vulcanization, can be employed in medical and pharmaceutical applications. It can be

smeared on flexible and elastic substrate or arranged in films applicable on elastic supports. Sensors

were realized by directly smearing the CE on a Lycra®-cotton fabric previously covered by an

adhesive mask. The mask is designed according to the shape and the dimension desired for the

sensors. The production phase is structured as follows: after depositing the material, the mask is

removed and the treated fabric is placed in an oven at about 130 degrees centigrade. During this

phase the cross-linking of the solution speeds up and, in about 10 minutes, the sensing fabric is

ready to be employed. It is important to underline that the integration of CE materials does not

change the mechanical properties of the underlying texture, thus maintaining a good comfort for the

user. For a correct determination of sensor positions and orientations, experimental trials were

performed in order to validate the proposed design. To obtain a specific sensor topology over the

fabric, an adhesive mask representing the drawing of sensors and connections was realized. The

mask is designed starting from the desired sensor positions and orientations traced on a three-

dimensional model of the human body. Then, the mask is realized by cutting a sheet of adhesive

paper with a laser milling machine.

2.2.2. Static and dynamic characterization of conductive elastomeric

sensor

The piezoresistive properties of CE composites, in literature, are statically described by using

percolation theory. In our work, the attention has been focused on the behavior of the electrical

resistance of CE during the transient time and other non-linear phenomena occurring after a

deformation. In our application, CE has been integrated into fabric and employed as strain sensors.

The main objective of the CE characterization has been to determine the relationship between the

electric resistance R(t) of a CE sample and its actual length L(t), where t is the time.

In terms of quasi-static characterization, a CE sample of 5cm in length and 1.7cm in width presents

an non-stretched electrical resistance of about 3KOhm per cm, and its gauge factor (GF) is about

2.9.




15/12/2010 of 63

0 0

0 0

L ( R R )GF

R ( L L )

where R is the electrical resistance, L is the actual length, R0 is the electrical resistance

corresponding to L0 which represents the unstressed length of the specimen. To obtain the gauge

factor, it is necessary to construct the static calibration curve shown in 1.

Figure 14: The static calibration curve.

The curve is realized by stretching the CE sample with the step in deformation having different strain (

Figure 15) and at the same time acquiring the final resistance value. The GF represents the angular

coefficient of the static calibration curve.

Figure 15: Response of a CE sensor excited by step

in deformation to build the static calibration curve




15/12/2010 of 63

Capacity effects were investigated and they were found to be negligible up to 100MHz. The

dynamic analysis of the electrical trend of CE sensors, when deformations are applied, has been

performed by using a system realized in our laboratories which can provide controlled deformations

and at the same time can acquire the resistance value assumed by the specimen. By using this

device, several deformations, which differ in their forms versus time, amplitudes and velocities,

have been applied to CE specimens. Sensor response shows a peak in correspondence to every

mechanical transition. Sensor responses during constant pressure time intervals may be

approximated by decreasing exponential, assuming the local minimum as the steady-state value.

The longer the pressure time interval, the more the above mentioned approximation is accurate. In

order to remove the contribution of high order exponential, the first order time constants were

extracted by means of a window filter. This choice allowed quantization errors introduced by the

acquisition device in response to rapid transitions to be avoided and sensor steady state

deformation, related to slower frequency components, to be maintained. Taking into account the

first-order components of the sensor response (resistance variation) to a rectangular stimulation

(applied deformation), the equivalent circuit represented in Figure 16 can be derived.

Figure 16: Electric model of a conductive elastomeric strain sensor

The power supply V is the electrical equivalent of the imposed deformation. The switch T1

(initially open) is closed and opened in correspondence to the beginning and the end of the imposed

deformation respectively. The switch T2 (initially open) is closed when T1 is opened again.

Following a simple analysis of this circuit, it is easy to recognize that the variation of the charging

and discharging currents of the capacitance in consecutive phases of stimulation are equivalent to

the variation of the resistance of the sensor during its deformation and the following release

respectively. The circuit parameters R1, R2, R3 and C can be derived by using the features,

extracted from reference experimental signals. A circuit voltage of 1 V was assumed as the

equivalent of a deformation of 1 mm, while a circuit current of 1 A was assumed to correspond to a

variation of the sensor resistance of 1 kΩ. Values of the features listed above were extracted from

ten cycles of a reference experimental signal and were used to derive the circuit parameters by

means of the following system of equations:

According to these ten cycles of stimulation, the solution of this system provided the results reported in

Figure 17, showing the time constants τ1 and τ2 have values of the order of 1 second.




15/12/2010 of 63

Figure 17: Values of the parameters of the equivalent electric model extracted from ten cycles

of a reference experimental signal.

Figure 18, which has been reported as an example of this analysis, shows the output of a sample

stretched with trapezoidal ramps in deformation having different velocities (where is the

length of the sample).

Figure 18: Response of a CE sensor excited by trapezoidal ramps in deformation

The main remarks on sensor behaviour are summarized in the following:

Both in the case of deformations which increase or decrease very quickly the length of the

specimen and in the case of deformations which reduce it, two local maxima greater than

both the starting and the regime value are shown.

If the relationship between R(t) and L(t) were linear, one of the extreme described in the

previous point would be a minimum.

The height of the overshoot peaks increases with the strength velocity .

The relaxing transient time, which lasts up to several minutes, is too long to suitably code

the human movements.




15/12/2010 of 63

2.2.3. Electrical model and acquisition electronic design

2.2.4. Electrical model and acquisition electronic design

The sensors and their connections to the acquisition unit was realized by means of the same materials, in this

way no metallic cables are needed on the seat. This is an advantage in terms of user comfort as we maintain the

fabric original elasticity. Moreover, by using this approach, the electrical contacts on the CE material can be

placed in areas where the fabric deformations and stresses are reduced

Figure 19 (A) and

Figure 19 (B) represent the generic configuration of a sensing cover and the electronics acquisition

front-end respectively. The sensors are connected in series thus forming a single sensor line (larger

line of scheme (A)) while the connections (represented by the thin lines of scheme (B)) intersect the

sensor line in the appropriate points.

Figure 19: Sensing cover electrical model (A) and electronic acquisition front-end (B)

Since connections and sensors are made by the same material, both of them change their electrical

resistance when the user moves. For this reason, the front-end of the acquisition unit had to be

designed in order to compensate the connection resistance variations. To obtain this result, the

sensor line is supplied with a constant current I and the voltage falling across two consecutive

connections are acquired using high input impedance amplifiers (i.e., instrumentation amplifiers), as

shown in 6B. Considering the example of sensor Si, if the amplifier is connected between Ck and

Ck+1, only a little amount of current flows through the connection lines compared to the current that

flows through the sensor line (and in the sensor Si). In this way, if the current I is well dimensioned,

the voltage read by the amplifier is almost equal to the voltage fall across the sensor (that is

proportional to the sensor Si electrical resistance). Taking into account the above described strategy,

the analog front-end of the electronic unit included a number of instrumentation amplifiers equal to

the sensors number. The data coming from the front-end, are low pass filtered, digitalized and

acquired in a PC by means of a general purpose card or transmitted by a dedicated electronic

interface. Moreover, the power consumption is near zero resulting in a completely safe system. The

fabric equipped with distributed and redundant unobtrusive strain sensors guarantees to address

plasticity, low dimension, lightness, and low cost. Since the strain sensors can be directly printed on

the fabric, specific cover layouts may be designed to coat different seat shapes obtaining a good

adherence to the seat. As a result, the sensing seat system does not interfere with the mechanical

structure of the seat and it is designed as an extension of the seat itself.




15/12/2010 of 63

Figure 20: The Sensing Seat system prototype:

Details of the sensor connections on the bottom side of the sensing cover

2.2.5. Sensing Seat Experimental results

2.2.5.1. Recording protocol and data analysis

Several topology layouts were taken into account (series, parallel and quadrupole network of

sensors) and finally the best compromise between the technical complexity and the classification

performance of the system was found using the series network. The sensing cover prototype is

equipped with 32 strain sensors: 16 sensors in the bottom side and 16 in the upper side. The existing

sensing cover prototype was tailored to a real office seat. Different layouts could be developed to

handle different types of seats (e.g. office seats, car seats). It should also be remarked that the seat

must be soft enough to guarantee the sensors to be adequately stretched as a human subject is

seated. Moreover, as it will be explained below, since the signals supplied by the sensors depend on

the positioning and the initial stretching of the cover, data are consistent only after the cover is

mounted over the specific seat (i.e. the enrolment signatures are no longer valid if the cover is

dismounted and mounted again even on the same seat). Some methods to extract the features were

tested, and the one, among the others, having the best performance in term of classification, was the

method using as features the minimum, maximum and mean values, and this one was implemented.

In this way during an action the 32 signals from the sensorized cover are acquired and stored, and

the start and stop action time are taken in account. The initial and final 5% of the entire time period

are not considered for the analysis in order to bypass the transitory signal phenomena. The

remaining part of signal is elaborated in order to extract the minimum value, the maximum value

and the mean value. This process is done over all the 32 signals. At this point, this matrix of values

recorded during the enrolment phase forms the biometric signature of a particular subject

performing a particular action, and all the signatures are stored into a database. This process is done

over all the subjects and over all the actions. During the authentication phase, the features are

extracted with the same strategy and then they are compared with the stored signature using a

classifier. Some classifiers were tested in order to evaluate the recognition phase. On the base of the

results, the classifier based on the Euclidean vector distance was chosen. No specific protocol is

required to extract the features, since the recording protocol is very adaptable. Only the subject is

asked to start from full-seated position and to return in the same position after to have performed

the action. Since these elements are linked to the deformation of the sensors due to the subject

pressure, each signature represents the deformation of the seat. As a result, the voltage vectors

available for each measurement and for each predefined position are related to the pressure exerted

by the subject on the seat.

The recordings gave the opportunity to have real data to be used to evaluate the system

authentication performances. Relevant data for the SensingSeat system were collected in a fixed

seat car scenario, giving to the users the opportunity to act according to a specified protocol. The

actions have been chosen in order to simulate in the best way a real car scenario. The list of the

actions for the events triggering are shown below:




15/12/2010 of 63

ActionID Action description

1 User is Seated

2

3

4

5

Correct Posture

Hands on the wheel

User is Reactive

User is Disturbed

6 No Activity

Table 8: The actions used for the event-related

authentication analysis.

As previously explained, the data have been analyzed with some algorithms in the way to test and

chose the one with the best performance. The classifier VDC, based on Euclidean vector distance,

was trained on 80% of the available examples for each action and subject, while the remaining 20%

was used as the test set. The results in term of FAR and FRR for each action and for each subject

are shown in the figures below. The data were analyzed in order to classify each subject, with

respect to all the others, performing each action.

Figure 21: SensingSeat authentication error: FAR and FRR (20 subjects, 6 actions, 6 repetitions per subject per

action).

Figure 22: SensingSeat authentication error: mean and standard deviation of FAR and FRR along subjects (left)

and actions (right); 20 subjects, 6 actions, 6 repetitions per subject per action.




15/12/2010 of 63

The results show a mean FAR = 3.3% +/- 3.9% and a mean FRR equal to 0.4% +/- 1.24%. Even if

the results are encouraging, more tests must be conducted since the number of repetitions per action

and subject is currently too small to assess the system reliability. The EER is calculated for the

considered subject performing the specific action with respect to all the other subjects. This

parameter permits to see how the system is able to recognize a subject during a specific event, and

to see how it is able to distinguish the considered subject respect to all the others.

Considering the global performance of the system, the best useful parameter is the EER calculated

for each action considering all the subjects. In the following table the results are reported.

ActionID Action description EER per action

1 User is Seated 1.367

2 Correct Posture 1.086

3 Hands on the wheel 3.547

4 User is Reactive 0.436

5 User is Disturbed 0.854

6 No Activity 0.835

Table 9: EER calculated for every actions considering all the subjects.

2.2.5.2. Sensing seat-based biometrics recognition entity

Enrolment

The user is asked to seat in each predefined position as it is defined in the measurement protocol.

For each of the predefined positions, the data of the sensing cover are acquired in real-time. The

user is asked to stay still until the steady state value is gained for each signal. This process takes a

few seconds (2 or 3 seconds) for each of the predefined position. Data are subsequently stored into

the PCs RAM and the whole process is repeated until all the predefined positions are taken into

account.

The acquired data belonging to the new user for all the predefined positions are used to train the

classification modules in order to take into account the new user inside the system. These

information will be used in the recognition procedure.

The operator must be able to enrol the new user into the central database (e.g. the user name is

required) and the new user identifier must be supplied to the sensing seat modality module.

Recognition

During the monitoring state, according to the conceptual design, the signals from the sensing seat

are acquired in real-time. The status of the internal classifiers is loaded from the database during an

initialisation step. The classifiers will perform the classification task in real time. Each classifier

will supply a user identifier and an error level as result of the authentication process. The

authentication output of each sub-module will be taken into account by a final classifier in order to

obtain a properly weighted authentication result.

Communication interfaces

The Sensing Cover system connects to the PC via a USB type “A” male connector, using USB 2.0

protocol. The system is configured and controlled from the client side using the provided extension

of UNIPI_SensingSeat API.




15/12/2010 of 63

External

External Interface Point UNIPI_SensingSeat API interface

Description The one and only external interface of the Sensing Seat system is represented

by the API exported by the UNIPI_SensingSeat class API. The provided

functions makes the client able to configure and control the modality during

the enrolment and recognition tasks.

Block diagram

Sub-systems using the interface - Data exchange layer

- Recognition layer

Message format Communication is based on exported dll C++ classes and functions. All

functions are called by the Core System.

Function prototypes InitSystem(…); Initialize the hardware system. Configuration values

are passed.

Update(dt, …); A new loop cycle is beginning. The inter-frame

time is passed.

SetEnvironmentInformation(…); Inform the system about

environmental information: event id (start/stop time window),

subject id (enrolment/recognition), …

SetSignature(…); Provide to the system a signature stored into the

DB.

GetSignature(…); Get the signature extracted by the system

GetRecognitionResult(…); Get the classification result

Special restrictions / considerations

Table 10: UNIPI_SensingSeat API interface (external)




15/12/2010 of 63

Internal

The sensing seat system is composed by many modules that have to cooperate:

A) Two sensing covers (resulting in a total of 32 CE strain sensors); sensing cover communication

cable (black thin wires; IN: wired buttons with elastic strips; OUT: 44-pins socket)

B) Symmetric double cable (black; IN: 44-pins plug; OUT: 44-pins plug)

C) Feeder device (black); 15 V symmetric power cable (black; IN: round 4-pins socket; OUT:

round 4-pins socket); 220 V power cable (grey; IN: CEE 7/7 plug, OUT: IEC C13 socket)

D) Front-end device (black, coloured strips with grey SUBCON 37L connector)

E) National Instruments NI-9205 board (blue/grey)

F) National Instruments NI cDAQ-9172 chassis (white); USB cable (black; IN: type A; OUT: type

B); 220 V power cable (black; IN: CEE 7/7 plug, OUT: IEC C13 socket); 12 V power cable with

voltage transformer (black; IN: IEC C14 plug; OUT: NI socket with screw tapping)

A) B)

C)

D)

E) F)

Figure 23: Modules of the sensing seat system




15/12/2010 of 63

The Sensing Cover (A) is connected to the Feeder device (C) through the 44-pin cable (B). The

Feeder device (C) is connected to the power supply (220V AC) and it provides the power (15V CC)

to the Front-end device through the symmetric power cable (D). The Front-end device (D) performs

a high-impedance voltage measurement from the Sensing Cover and it supplies the analogue

information to the NI-9205 DAQ (E) which is plugged into the NI cDAQ-9172 chassis (F). The

chassis is responsible to dispatch the digital data to the high-level platform (PC, palm…) through a

USB 2.0 connection. The provided API internally communicates with the Sensing Seat hardware

system through the NI-DAQmx API.

Figure 24: The NI-DAQmx API

Seat-based biometrics

The UNIPI_SensingSeatSignature is stored in binary format.




15/12/2010 of 63

Figure 25: Seat-based biometrics.

2.3. Personal driver fatigue safety system

UNIPI, with the support of its subcontractor (IFC-CNR), realized a wearable multi-parametric

monitoring system, integrated in the pilot CAN bus network, for the personal ergonomic wireless

detection of driver fatigue. The multi-parametric monitoring system block scheme is reported in

Figure 29. The system helps to automatically identify and address major driver fatigue deficits.

During his/her driving, the performance of a subject are continuously evaluated in order to

recognize immediately the onset of possible problems related to reduced performances correlated to

fatigue. The system is able to acquire surface electromiographic (sEMG) and electrocardiographic

(ECG) signals, as well as behavioural activity through micro-accelerometers and intelligent signal

processing algorithms. The system consists of two modules: i) the sEMG module; ii) the ECG and

activity level module.




15/12/2010 of 63

Figure 26: Multi-parametric monitoring system block scheme

The sEMG module

The unobtrusive sEMG transducers and pre-processing electronic board (figure 30) allows

information regarding the overall muscle function and condition to be collected from the surface of

the skin. The pre-processing electronic board includes a low-power standard Bluetooth and

802.15.4 communications, large storage capacity (2 Gb), transduction, amplification and signal pre-

processing blocks. The sEMG signal acquisition method is non-invasive and non-painful to the

subject.

Figure 27: sEMG transducers and pre-processing electronic board

The aim of the sEMG is the analysis of the function and coordination of muscles in different

movements and postures in healthy subjects as well as in the disabled, in sedentary as well as in

athletes, under laboratory conditions as well as in the field. Since EMG is a direct measure of alfa

motor neuron activity, it provides information about the central nervous system command that




15/12/2010 of 63

generates the movement, and in case of hierarchical models of intentions of the mirror neuron

system. The timing information from the EMG signal, instead, is much more accurate. The sEMG

signal detected on the skin surface includes information from a greater proportion of the muscle of

interest than conventional clinical EMG, acquired using needle electrodes. sEMG is widely used to

evaluate muscle activity and can determine which muscles are active, their degree of activity, and

how active; it can also used to estimate muscle force. Furthermore, they are easy to use and provide

objective inter-intra-subjects comparable data, suitable for statistical analysis. sEMG based studies

have provided relevant contributions to the understanding of human movement, neuromuscular

adjustments and adaptations occurring while - or following - exercise and training and, in a wider

sense, to the clarification of some aspects of neural control of movement. Physiological fatigue is

not necessarily accompanied by self‐ perceived fatigue, nor viceversa. The loss of force-producing

capacity can both (and simultaneously) have a peripheral and a central origin. Median and mean

spectral frequencies are commonly used as indicators of muscle fatigue. Analyzing the sEMG

spectra during spontaneous contractions while driving, the system which automatically reveals the

occurrences of contractions, extracts a personal muscular fatigue index versus time. The

unobtrusive sEMG electrodes are placed at the latissimus dorsi muscle (figure 31) of the driver.

Figure 28: sEMG module position

The ECG and activity level module

The ECG and activity level module consists of an ergonomic wireless unobtrusive sensorized soft-

textile chest strap. ECG is the best way to measure abnormal rhythms of the heart, particularly

abnormal rhythms caused by damage to the conductive tissue that carries electrical signals, or

abnormal rhythms caused by electrolyte imbalances and any damages in specific areas. The

sensorized chest strap embodies three electrodes, the related electrical connections and a dedicated

portable integrated electronic board placed at the level of the thorax. The electronic board includes a

low-power standard Bluetooth and 802.15.4 communications, large storage capacity (2 Gb), three

axis accelerometer, transduction, amplification and signal pre-processing blocks. The chest strap is

mainly made of flexible biocompatible elastomer, cotton and elastan; it is fully washable and

guarantee an optimal and comfortable contact between the strap and the thorax adapting itself to the

body shape (figure 32). The soft-textile chest strap can be integrated in an elastic comfortable t-shirt

to enhance usability.

Figure 29: Chest strap




15/12/2010 of 63

An embedded electronic board acquires the ECG signals and transmits data via Bluetooth to the car

industrial pc for analysis, storage and relay transmission. The system is powered by a 3.6-V

rechargeable battery. The battery life allows up to 8 h of continuous monitoring per charge.

Figure 30: ECG and activity module position

The chest strap registers the heart‟s electrical signal mimicking the bipolar lead II of the classical

ECG measurement configuration. The system is able to visualize the normal sinus rhythm,

arrhythmias (non-normal heart rhythms) and extrasystoles (an ectopic, usually premature

contraction of one of the chambers of the heart, resulting in momentary cardiac arrhythmia), and to

assess heart rate (HR) and heart rate variability (HRV) through dedicated algorithms. A dedicated

algorithm is able to extract time-domain and frequency-domain features of ECG signals through

advanced signal processing techniques in order to localize the sympathetic and parasympathetic

nervous system activities associated with different frequency bands, quantitatively assessing HRV

signals by means of power spectral analysis. The powers of high frequency (HF) and low frequency

(LF) components of HRV, as well as the LF/HF ratio, allow cardiac vagal and sympathetic

activities as markers of autonomic interaction to be estimated.

The chest strap has been tested on subjects. The performance of the system was evaluated both

motionless and driving. The ECG was simultaneously recorded by the chest strap and two gold

standard ECG clinical devices. A comparison of the recordings performed by expert cardiologists of

the Institute of Clinical Physiology (IFC-CNR) has shown that the chest strap recordings with

respect to the traditional ECG electrode placements and recordings had a similar waveform and a

negligible artifact rate. The discrepancy in the estimation of the RR interval was lower than 1 ms.

An analysis of the signals acquired while driving performed by expert cardiologists has shown that

the chest strap recordings had excellent sinus rhythm visualization and a negligible artefact rate.

Key features of the chest strap:

Ergonomic and comfortable body shape adaptation

Lightweight

Negligible artifact rate in dynamic conditions

Wireless Bluetooth connection

Up to 8 h of continuous monitoring per charge

Flexible biocompatible elastomer




15/12/2010 of 63

ECG recordings of two different electrocardiographs:

Figure 31: Comparative ECG recordings

Figure 32: Chest strap dynamic ECG recordings

In order to classify the human activities during driving, relevant features over time and frequency

are extracted from the 3-axis accelerometer signal. The signal is continuously segmented with

partially overlapping windows of different size. A model based on an artificial neural network is

able to identify different driver activity: low, normal, high, excessive.




15/12/2010 of 63

A hierarchical neural network based model is able to analyse the input variable coming from the

sEMG and the ECG and activity level modules in order to classify the driver fatigue index while

driving.

Figure 33: Driver fatigue analysis model

2.4. Architecture

The driver will be informed and warned from the Symbian OS device. The Symbian device will

retrieve the values of the CAN signals from the vehicle industrial PC and then it will generate the

corresponding type of message to the driver. The Symbian vehicle application will be integrated on

the main OASIS MIDlet using the LWUIT graphic library. The OASIS vehicle application is

assumed as an OASIS local application and it will use the User Profile API in order to access the

corresponding parameter‟s value.

The physical connection between Symbian OS device and the industrial PC will be wired and

wireless as an alternative solution. The next paragraphs describe these two architectures.

2.4.1. USB connection architecture

The physical connection between Symbian OS device and the industrial PC will be wired via a USB

cable. The approach assumes as prerequisite the installation of PC suite or Ovi suite application

from Nokia. The device‟s actual USB connection takes place via PC (Ovi) suite program which

opens a virtual COM connection.

A USB 2.0 repeater cable of 3 meters is also used in order to connect the device to industrial PC

located at the vehicle‟s trunk.

The following schema depicts the connection between Symbian device and industrial PC.




15/12/2010 of 63

Figure 34: Modules’ physical connection

The OASIS JME application running in Symbian OS device communicates with the industrial PC

application which acquires vehicle‟s CAN signals. The following figures show the interaction

between these applications.




15/12/2010 of 63

Figure 35: Communication between applications

As it is mentioned above, the prerequisite are:

The Symbian device has to be connected to vehicle PC via a virtual COM port (see figure

below).

The installation of the PC (Ovi) suite has to be installed for Nokia devices or a similar one

such as the Sony Ericsson PC Suite 6.0 for Sony Ericsson Symbian devices in the vehicle

PC.

Figure 36: Nokia USB connection as a “COM” port connection




15/12/2010 of 63

Figure 37: PC suite USB mode

Figure 38: PC vehicle application; collects CAN signals and send to smart phone device

2.4.2. Bluetooth wireless architecture

This architecture is an alternative one and is totally dependent on the electronic board unit which

will be attached to the OBD-II plug within vehicle. This board contains a micro controller and its

main functionality is to collect the CAN signals provided by OBD-II and then transmits their values

to smart phone. It contains also an integrated modules and chipsets such as CAN, UART for the

serial communication, Bluetooth, EEPROM and a microcontroller.




15/12/2010 of 63

Figure 39: Modules in the wireless approach

The Symbian OS application software will implement the Service Port Profile from

Bluetooth protocol. It receives the vehicle CAN signals and displays them in G.U.I. form.

This approach assumes that the Symbian OS device is the “master” within the above Bluetooth

network. The prerequisite is that the board unit has to be paired before communication took place.

The pairing procedure consists of the following steps:

Bluetooth must be enabled on the smart phone.

The board unit has to be turned on.

The board unit will be showed in the Bluetooth screen of the smart phone as “Not Paired” or

“Not Connected”.

The appropriate button from board unit has to be pressed in order to connect to smart phone

device.

This approach is totally relied on the available signals provided by the OBD-II.




15/12/2010 of 63

2.5. Equipment installation and demo scenarios

The next paragraphs describe the scenarios will be developed for OASIS project in conjunction with

the related equipment that will be installed in the vehicle.

2.5.1. Seat sensors

One of the main objectives of OASIS WP3.4 is to provide comfort support services to elderly

driver. The sensing seat will be installed in Lancia Thesis and the vehicle PC will acquire data in

order to analyze them further.

Figure 40: The sensing seat within vehicle

On the other hand, the Symbian OS device will be located next to driver as it is displayed in the

following figure.




15/12/2010 of 63

Figure 41: Device’s location (1)

Figure 42: Device location (2)

The vehicle PC software application which will collect data from seat sensors using the

UNIPI_SensingSeat API Interface, then it will estimate the driver‟s stress status and finally it will

send the information to the Symbian device. On its turn, Symbian device will enable the

corresponding warning or informative message to the driver.




15/12/2010 of 63

2.5.2. Blind spot detection

2.5.2.1. BSD system overview

A Blind Spot Detection (BSD) system has the goal to warn the driver in case another vehicle is

placed in the blind spot which is not controlled by the rear-view mirrors. Thus, it counts with some

sensors (commonly, short range radars @ 24 GHz or image processing units) which monitor

constantly the area placed in the lateral blind spots of the vehicle. These sensors provide

information to a Control Unit, which decides the susceptibility to provide the driver with a warning.

This warning can be acoustic, visual or haptic.

Some systems can warn continuously on the existence of objects in the blind spot. Some others only

warn when the driver expresses his will to change lane, using the correspondent blinker. They

usually work over a certain speed and are capable to filter parked vehicles or those driving on the

opposite direction, in order to reduce the false alarm rate. The detection area can measure around 10

meters behind the rear view mirror and 4 meters wide, enough to cover the blind spot.

For the purposes of OASIS project, the BSD model will be installed in Greek pilot vehicle. The

following table describes its technical specifications.

Topic Description

Sensor/Subsystem Blind Spot Detection System

Supplier: CTAG

The main components of the system are: a processing unit with BSD

algorithms, two 24Ghz radars and a group of leds (left and right side) to

visualize system warnings.

Brief technical description The rear radar information (Objects speed and location) is processed by the

ECU. The ECU makes the data fusion of both radar information, as well as

the tracking of the detected objects. After processing, the ECU can send an

output in order to warn the driver of any risk. This output is connected to a

group of LEDs in order to visualize this warning (left or right side). All

communications are implemented via CAN-bus (500kbps). The system

allows detecting objects in the blind spot zone of the car, this detection allows

warning the driver in case of a certain risk when changing lane.

Physical characteristics

Radar Size: 140x117x42 mm

Processing Unit Size: 173x78x 57mm

Led boards(2) Size: 50x20x1 mm

Position in the car The two radars must be allocated behind the rear bumper of the car each one

of radar in one corner.

The processing unit is allocated In the luggage area.

The led board is allocated in each rearview mirror position (left an right)

Power consumption 24Ghz Radar(2):

Power supply: 12 V – 200 mA

Processing unit:

Power supply: 12 V – 100 mA

Led Board(2):

Power supply: 12V – 10 mA (supplied by the processing unit)

H/W requirements CAN bus interface

S/W requirements Embedded software

Table 11: BSD technical specifications




15/12/2010 of 63

2.5.2.2. 24 GHz Radar

Two radars of 24 GHz gives information to the BSD ECU related to the speed and location of the

detected objects in the rear area of the vehicle.

Figure 43: 24 Ghz Radar

The main specifications of the radar are:

o Power Consumption: ~ 200 mA 12V

o Physical Dimensions: 140x117x42 mm

o Communication interface: 500 KBits/s CAN Bus

o Range Detection: 0.5 – 70 m (objects with relative velocity not zero)

o Speed Interval: -250 …. 250 km/h

o Angle Accuracy: typ. +/- 0.75 degrees

o Cycle Time: < 40ms

o Other: Waterproof housing and connectors.

2.5.2.3. BSD Processing Unit (ECU)

The processing unit receives data from both 24Ghz radar and from the CAN Bus of the vehicle

(speed, blinkers…). The ECU makes the object tracking and the fusion between both sensor data.

With the processed data and the car information, the ECU can determine if it have to send a

warning to the driver (with the group of leds) or not.




15/12/2010 of 63

Figure 44: BSD ECU

The main specifications of the processing unit are:

Power Consumption: ~ 100 mA 12V

Physical Dimensions: 173x78x 57mm

Communication interface: 500 KBits/s CAN Bus

Minimum CAN Vehicle Information required: Speed, Yaw-rate and blinkers

Other: Waterproof housing and connectors.

2.5.2.4. BSD connection

Communication between all systems devices are via 500Kbits/s CAN Bus. Power supply of 12V is

required for the ECU and both 24 GHz Radar (See Figure 3).




15/12/2010 of 63

Figure 45: Connection schema

The BSD‟s warnings will be also published to the Symbian device in order to ensure that the driver

will be definitely informed.

2.5.3. Lane Departure

As it has already mentioned in the above chapters, CERTH/HIT research vehicle is instrumented

with a Lane Departure System. The use and the results of this sensor have already been validated in

previous research European projects.

Figure 46: Camera housing glue to the windscreen




15/12/2010 of 63

Its main technical chrematistics are shown in the following table.

Topic Description

Sensor/Subsystem Lane detection system and lane departure warning

Supplier: CRF

The main components of the system are: a CCD camera and a processing unit.

Brief technical

description

A standard CCIR video signal (from CCD camera) is processed by our system in order to

recognize lane borders and to estimate, in addition to lane width, the position and the

orientation of the car respect to them.

The types of lane border we are able to recognize are continuous or dashed. The system is not

able to recognize road limit without any delineation line (white/yellow painted lane markers).

The system is also able to work in all weather conditions, apart from heavy rain, fog or snow on

the road.

Lane detection information and Lane warning information is provided for speed higher than 50

Km/h (extra-urban, highway enviroment)

Physical

characteristics

Camera Size: 32x32X30 mm

Processing Unit Size: 150x110x40 mm

Position in the car The CCD camera is fixed behind the windshield, near to the central rearview mirror.

The processing unit is placed in the luggage van.

Power consumption CCD camera:

Power supply: 9 V - 120 mA (provided by the processing unit)

Processing unit:

Power supply: 12 V - 1.0 A

H/W requirements CAN bus interface

S/W requirements Embedded software

Table 12: Lane departure system’s characteristics

The Lane Departure unit is controlled by ECUs and their messages are also retrieved by an

industrial PC. In case there is a Lane Departure warning, the industrial PC will send it to Symbian

OS device.

2.5.4. Frontal Radar

The frontal radar will be used in order to measure the headway parameter. The headway is given by

the formula of the distance/time between vehicles in a transit system [5]. It is located in the

vehicle‟s frontal bumper.




15/12/2010 of 63

Figure 47: Frontal radar

The radar sensor‟s main features are shown in table below.

Topic Description

Sensor/Subsystem RADAR SENSOR

Supplier: FUJITSU TEN.

Brief technical description RADAR SENSOR is a detector able to track the position of

obstacles in front of the vehicle.

Emission frequency: 76 - 77 GHz

Modulation: FMCW

Antenna type: Mechanical scanning

Antenna gain: < 33 dB

Output power: < 10 mW

Horizontal field of view (azimuth): ± 8°

Max operating range (car): 140 m

Minimum operating range (car): 5 m

Target of tracked obstacles: Max 20

Physical characteristics Weight: < 700 g

Size (W x H x D): 107 x 89 x 86 mm

Operating temperature: -30°C to +65°C

Storage temperature: -40°C to +105°C

Position in the car Frontal bumper

Power consumption Operating Voltage: 9 to 16 V

Current: 1 A @ 12V

Communication requirements CAN bus interface (500 kbps)

Data refresh rate: 100 ms

Contents sent from vehicle‟s industrial PC: vehicle speed, curve radius

Contents sent to vehicle‟s industrial PC: range, range rate, angle

Table 13: Frontal radar technical characteristics

The driver is warned when the headway is below the limit threshold. The limit has been declared

with the default value of 1.1 seconds.




15/12/2010 of 63

2.5.5. OASIS application and vehicle speed

It has been noticed that many times the driver interacts with his/her smart phone device while

he/she is on the way. This scenario assumes that one of the OASIS applications is running on

driver‟s smart phone (e.g. routing or navigation service is running). When the driver exceeds the

limit of 15 km/h, the smart phone application does not allow him/her to interact with his device for

safety reasons. The Symbian OS application disables specific G.U.I. components such as button and

menus.

This “safety” scenario is applied only when OASIS application is running and not in the whole

functionalities of smart phone. For example, the user can receive calls and SMS messages while

he/she is driving with more than 15 km/h.

2.5.6. Type of messages

The messages can be informative or warning. They will be enabled in Symbian OS device when

one of the OASIS applications is running and the driver is on the road. These messages will be

appeared to driver in the following forms:

Acoustic

o Sound messages will be enabled from Symbian device.

Haptic

o Vibration of the device.

Visual

o Icons will flash on top of running OASIS application on the device.

The smart phone application will provide an additional G.U.I. form to user in order to select the

type of message he/she preferred.

2.6. Impacts and further considerations

2.6.1. General impacts

Advanced age affects cognitive perception and consequent visual field limitations increase accident

risk for elderly drivers. The application of a mobile device controlled system targeting to increase

comfort and alleviate stress would enhance quality of stress control and allow elder drivers to

manage every day activities with increased confidence. Self-assuring information on health status

is critical and personalized information may lead for the device to play a protective and mitigating

role in daily living. Moreover, dynamic driving variables (e.g. lateral and longitudinal acceleration)

allow the driver to adjust „risky‟ behaviour and increase driver and road safety. The system will be

cooperative and supplement driving skills which are now declining. In addition, familiarization

with the system will enable driver to create his/her driving profile. In other words, if he/she has

used the system for a while, then they learn (learning effect) how they behave and where they need

to be more cautious (e.g. if they close follow leading vehicles because their vision is impaired, the

system informs them about it and they correct it).

The above paragraphs also describe the type of the signals that are provided by the vehicle sensors

and how they will be transmitted to the driver‟s Symbian OS device. The main aim is to warn

drivers that their vehicles do not contain industrial PC and thus, a real time data processing is not




15/12/2010 of 63

available. The handheld device is responsible for the processing of vehicle‟s data and for serving

the appropriate output message to the driver. The industrial PC is used only1 as a “message broker”

between vehicle sensors and handheld device (paragraph 2.4).

Moreover, the management of the messages by driver‟s Symbian device can be applied to vehicles

that do not contain dashboard screen and also to vehicles that although they may include a

dashboard screen, it is not feasible to access it and display any software application on it. The

Greek pilot vehicle contains a dashboard screen which is totally accessible and many applications

can run on it. Despite that, the warning messages will be totally provided by the driver‟s Symbian

device for the purposes of OASIS project.

2.6.2. Potentials for a future vehicle communication protocol

The Greek pilot scenarios‟ sections investigates whether additional signals -which are critical to

driver‟s safety (lane departure, frontal radar, blind sport detection) and driver‟s biometrics (ECG,

fatigue‟s measurement (sEMG)) - have to be provided as a new standard protocol by any vehicle

and not only the OBD-II protocol‟s ones [6].This idea can be extended and implemented with the

definition of another standard in-vehicle protocol which collects vehicle peripheral data such as the

above. The validation of these scenarios can be used for further analysis and consider whether it

could be useful for the driver. This protocol can be independent or it can be integrated with the

existing OBD-II (considering custom versions of it among vehicle manufacturers). Consequently,

peripheral vehicle sensors‟ ECUs may substitute the vehicle‟s industrial PC; hence data processing

is diverted directly from sensors towards the driver‟s Symbian device. The following figure depicts

this idea.

Figure 48: Future in-vehicle protocol based on sensors’ ECUs

The vehicle peripheral sensors‟ data can be provided as an additional plug (a hardware “output”)

similar to the OBD-II one as it shown below.

1 Some of the parameters‟ calcualtions such as the headway and the time to line crossing are processed by the vehicle‟s

industrial PC.




15/12/2010 of 63

Figure 49: OBD-II plug




15/12/2010 of 63

Abbreviations

ACC Adaptive Cruise Control

AmI Ambient Intelligence

BSD Blind Spot Detection

CAN Controller Area Network

CE Conductive Elastometers

DB Data Base

DIN Deutsche Industrial Norms

E-Call Emergency Call

ECG Electrocardiogram

ECU Electronic Control Unit

EEPROM Electrically Erasable Programmable

Read-Only Memory

GPS Global Positioning System

GPRS General Packet Radio Service

GSM Global System for Mobile

Communications

GUI Graphical User Interface

HMI Human Machine Interface

LCD Liquid Crystal Display

LDW Lane Departure Warning

OBD-II On Board Diagnostic - II

OS Operating System

PC Personal Computer

PCI Peripheral Component Interconnect

PDA Personal Digital Assistant

POI Point Of Interest

R&D Research and Development

sEMG Surface ElectroMioGraphic

SP Sub Project

UART Universal Asynchronous Receiver-

Transmitter

UI User Interface

UML Unified modeling Language

USB Universal Serial Bus

VEP Vehicle Embedded Platform

VGA Video Graphics Array

WP Work Package

http://encyclopedia2.thefreedictionary.com/General+Packet+Radio+Service




15/12/2010 of 63

References

[1] OASIS Description of Work, last update September 1, 2009, EC Approved January 27 2010

[2] http://www.esafetysupport.org/en/ecall_toolbox/ecall_standardisation/

[3] http://www.smartex.it/smartex.htm?label=&clickHandler=&width=550.1&height=400.1&te

xtStyle=[object+Object]&enab

[4] “HIT experimental vehicle”, Antonello Per Claudio, April 2004

http://www.smartex.it/smartex.htm?label=&clickHandler=&width=550.1&height=400.1&textStyle=%5bobject+Object%5d&enab

http://www.smartex.it/smartex.htm?label=&clickHandler=&width=550.1&height=400.1&textStyle=%5bobject+Object%5d&enab

D3.4.1 - Two vehicle demonstrators for elderly drivers’...

Documents

Transcript of D3.4.1 - Two vehicle demonstrators for elderly drivers’...