1

Tactile Brain-computer Interface Using Classification of P300

Responses Evoked by Full Body Spatial Vibrotactile Stimuli

Tactile Brain-computer Interface Using Classification of P300

Responses Evoked by Full Body Spatial Vibrotactile Stimuli

@APSIPA ASC 2016@APSIPA ASC 2016

1

Takumi Kodama , Shoji Makino and Tomasz M. Rutkowski1 1 2, 3

Life Science Center of TARA, University of Tsukuba ,The University of Tokyo , Saitama Institute of Technology

12 3

1: Introduction - What’s the BCI?

● Brain Computer Interface (BCI)○ Exploits user intention ONLY using brain waves

2

1: Introduction - ALS Patients

● Amyotrophic lateral sclerosis (ALS) patients○ Hard to move their muscle due to nerve injuries○ BCI could be a communicating option for them?

3

!!!

…

…

1: Introduction - Research Approach

1, Stimulate touch sensories 2, Classify brain response

AB

A

B

3, Predict user intention

92.0% 43.3%

A B

TargetNon-Target

P300 brainwave response

4

● Tactile (Touch-based) P300-based BCI paradigm○ Predict user’s intentions by decoding P300 responses○ P300 responses were evoked by external (tactile) stimuli

● Previous Tactile P300-based BCI paradigm○ Tactile point-pressure BCI (for usual hands)○ Tactile and auditory BCI (for head positions)

1: Introduction - Previous Research

5

[1] K. Shimizu, S. Makino, and T. M. Rutkowski, “Inter–stimulus interval study for the tactile point–pressure brain–computer interface,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE Engineering in Medicine and Biology Society. IEEE Press, August 25–29, 2015, pp. 1910–1913.[2] H. Mori, Y. Matsumoto, Z. R. Struzik, K. Mori, S. Makino, D. Mandic, and T. M. Rutkowski, “Multi-command tactile and auditory brain computer interface based on head position stimulation,” in Proceedings of the Fifth International Brain-Computer Interface Meeting 2013. Asilomar Conference Center, Pacific Grove, CA USA: Graz University of Technology Publishing House, Austria, June 3-7, 2013, p. Article ID: 095.

● Propose new touch-based BCI modality intended for communicating with ALS patients

● Confirm an effectiveness of the modality in terms of practical applications, stimulus pattern discriminations and classification accuracies

1: Introduction - Research Purpose

6

2: Method - Our Approach

7

● Full-body Tactile P300-based BCI (fbBCI)○ Applies six vibrotactile stimulus patterns to user’s back○ User can take experiment with their body lying down

2: Method - Two experiments

8

Psychophysical EEG● Without attaching

EEG electrodes ● Selecting target

stimulus with button pressing

● To evaluate the fbBCI stimulus pattern feasibility

● With ERP calculation● Selecting target

stimulus with P300 responses

● To reveal the fbBCI classification accuracies

2: Method - Signal Acquisition

9

● Event related potential (ERP) interval○ captures 800 ms long after vibrotactile stimulus onsets○ will be converted to feature vectors with their potentials

Vlength

VCh○○

p[0]

…

p[Vlength - 1]

Ch○○

Condition Details

Number of users (mean age) 10 (21.9 years old)

Stimulus frequency of exciters 40 Hz

Vibration stimulus length 100 ms

Inter-stimulus Interval (ISI) 400 ~ 430 ms

Number of trials 1 training + 5 tests

EEG sampling rate 512 Hz

EEG recording system g.USBamp active electrodes EEG system

Classification algorithm SWLDA with BCI2000

2: Method - Experimental settings

10

3: Result - Behavioral accuracies

● Correct rate exceeded 95% for each stimulus pattern

11

3: Result - Response times

● Response time differences for each stimulus pattern

19

● P300 responses were confirmed (> 4 μV) in every channel

3: Result - ERP (P300) responses

12

TargetNon-Target

3: Result - ERP AUC Scores

21

● Times series of the Target vs. Non-Target AUC scores

3: Result - ERP pattern differences

20

● P300 peaks were shifted to later latencies from #1 to #6

#1 Left arm

#2 Right arm

#3 Shoulder

#4 Waist

#5 Left leg

#6 Right leg

3: Result - Classification accuracy

13

● Grand mean fbBCI classification accuracy: 53.67 %

3: Result - ITR score

14

● Information Transfer Rate (ITR)○ Averaged score: 1.31 bit/minute

4. Conclusions

15

● The validity of fbBCI paradigm was confirmed○ Classification accuracy : 53.67 % by SWLDA○ Expect to help ALS patients

● However, more analyses would be required○ Only 10 healthy users has tried yet○ Applying other machine learning algorithms○ Higher accuracies would be needed for practical

applications

● The HARA Research Foundation Research Grant Project ○ for APSIPA ASC 2016 Participation

Special Thanks

16

17

Many thanks for your attention!