Directeur : Mr S. PERREY (PR).

Improving Usability in Human Computer Interfaces: an investigation into cognitive fatigue and its influence on the

performance of hybrid brain computer interfaces

Presented by Gérard Derosière

06/14/2012

2

I. Supervisors of the project and partners

Codirection in the context of a PhD thesis :

Supervisors Partners Specifities / Competences Logo

Tomas Ward

Department of Electronic

Engineering, National University of Ireland

Maynooth.

Specialized in the development of Brain

Computer Interfaces (BCIs).

Stéphane Perrey

Movement to Health (M2H) Laboratory,

Montpellier 1 University.

Understanding of the cerebral correlates during :- A motor task with fatigue

occurrence;- The cognitive tasks.

3

II. Introduction

Brain-Computer Interfaces (BCIs): allow the control of an electrical system without

voluntary movement needed, just by the mean of the cerebral activity (Gouy-Pailler, 2010).The user have to :- Do mental imagery or mental arithmetic;- Focus his attention to a light.

How is it possible?

These tasks require sustained attention.

One main characteristic of the sustained attention:

The time-on-task (TOT) effect.

Characterized by:

- A progressive decrease of the attentional resources with time measured through the increase of the reaction times (RTs) (Lim et al., 2010);

- A modification of the cerebral activation (e.g., Paus et al.,

1997) non-stationarity.

Attentional resources

4

II. Introduction

TOT effect must be reduced during the use of a BCI for 2 reasons:

- Decrease of attentional capacities exhaustion, inability to use the BCI;

- Modification of cerebral activation ↓ of classification accuracy.

Proposed solutions (Baï et al., 2010): to develop BCIs requiring minimal training, and thus, less mental load

Pfurtscheller et al. (2010): the hybrid BCIs (hBCIs) allow to enhance classification accuracy.Allison et al. (2010): development of a simultaneous hBCI. Event-Related Desynchronization (ERD): mental imagery. Steady-State Visual Evoked Potential (SSVEP): focalization of attention to a light.

Enhancement of the classification accuracy in the simultaneous condition.

5

III. Definition of the problematic

Summary

BCIs require sustained attention;

Sustained attention is characterized by a "Time-On-Task" effect; TOT effect must be reduced during the use of a BCI; A new type of BCI to enhance classification accuracy: the simultaneous hBCI;

Incompatibility between simultaneous hBCI and the aim to reduce TOT effect.

Questions:

- Does the simultaneous condition generate an early TOT effect?

- If so, is it linked to an early modification of the cerebral activation?

- Finally, does it generate an early decrease of the classification accuracy?

Simultaneous hBCIs require divided attention the mental load = higher than a simple

condition.

6

IV. Proposed protocol

PASAT PASAT PASAT PASAT

RTs RTs RTs RTs

...

Time

RTs RTs RTs RTs

...

Time

SSVEP SSVEP SSVEP SSVEP

...

Time

SSVEP + PASAT SSVEP + PASAT SSVEP + PASAT SSVEP + PASAT

RTs RTs RTs RTs

Session 2: Steady State Visual Evoked Potential (SSVEP) only

Session 3: hybrid condition

Session 1: Mental arithmetic onlyPASAT: Paced Auditory Serial Addition Test (mental arithmetic task).RTs: Reaction times.

Concomitant measurements:Electroencephalography (EEG), functional NIRS. Subjective aspects of fatigue.

Classification accuracy tested in the three conditions, in a monomadal and multimodal manner.

RTs

RTs

RTs

Comments:

7

V. Expected results and conclusion

Expected results:

RTs

TOT

SSVEP + Mental arithmetic

Mental arithmetic

SSVEP

SSVEP + Mental arithmetic

Mental arithmetic

SSVEP

TOT

Classification accuracy

Project partly financed by the LABEX NUMEV this will conduct to new development for pathological people.

Classification accuracy

TOT

Multimodal (EEG+fNIRS)

Monomodal fNIRS

Monomodal EEG

Tested in the three conditons:

Directeur : Mr S. PERREY (PR).

Thank you for your (sustained) attention.