Spatial tactile brain-computer interface paradigm by applying vibration stimulus to large body areas

Spatial Tactile Brain-Computer Interface Paradigm 　　　by Applying Vibration Stimulus to Large Body Areas201213082 Multimedia Laboratory !! Takumi Kodama!Supervisors:!! Tomasz M. Rutkowski, Takeshi Yamada, Shigeki Miyabe, Shoji Makino

Research Background

"2

❖ Healthy and non-disabled people − Can communicate their intentions with others easily

❖ Amyotrophic lateral sclerosis (ALS) patients − Can’t move their muscle by themselves − Thus they can’t communicate their intentions

So they need device not using muscle like “BCI”

・・・

About “Brain-Computer Interface”

BCI could be a communicating tool for ALS patients"3

Brain waves are captured

Machine learning estimates

user intention

Thought based application control

❖ Brain Computer Interface (BCI) − Control the device based on brain signals (EEG)

❖ To input BCI commands: − Not necessary to use your body movements

❖ Stimulus-driven BCI:− Use brain activity only generated in response to external stimuli

❖ Stimulus-driven BCI has five modes (five senses of human): − Difference is how to evoke P300 response

Type of Brain-Computer Interface:

I use “tactile (touch) BCI” in my research project"4

Auditory BCI

Gustatory BCI

Olfactory BCI

Tactile BCI

Visual BCI

Stimulus-driven BCI

Imagery BCIBCI

About “P300 response” ❖ P300 response− Positive signal that occurs 300 ms after receiving a stimulus

P300 response evoked by applying tactile stimulus to human body

"5

: Target

: non-Target

Research Approach❖ Development of a new tactile BCI paradigm

− Propose new communication option by using tactile BCI

"6

Give user tactile stimuli of body parts with larger distances

Tactile stimuli Tactile stimuli

About a “gaming pad”❖ Specification of the gaming pad −Used to deliver tactile stimuli to any body parts

"7

Only the vibrotactile output to stimulate the users is used

SpecificationName “ZEUS VYBE”

Developer Disney ResearchInput Audio signal

Output Sound and vibrationSound Two speakers inside

Vibration 12 vibration motors inside

Input: !Audio signal

♪ ♫

Output:!Sound & Vibration

~~~

~

❖ Define six vibrotactile stimulus positions in gaming pad − Stimulus positions are located in spatial body parts

"8

Users identify six vibrotactile positions individually

How to deliver the tactile stimulus

No. Pos. name1 Left shoulder (LS)2 Right shoulder (RS)3 Left waist (LW)4 Right waist (RW)5 Left leg (LL)6 Right leg (RL)

How to use the gaming pad❖ Gaming pad is used in “laying-position” − Because ALS patients are often bedridden

"9

Users lie on the gaming pad and receive vibrotactile stimuli

“Sitting position” “Laying position” ▲Experiment in laying position

❖ The experiments were carried out in two steps

"10

First: Psychophysical Experiment

#1 Psychophysical Experiment

Purpose: !!!!

!

Experimental Procedure

! ? !

!#2 EEG Experiment

! Purpose: !!!!

　・To check whether the P300 response 　could be evoked after vibrotactile stimuli

　・To check whether the users could 　　　perceive the tactile stimulation correctly

❖ Flow of the experiment in each sequences

"11


3. Users press the button when feel the

target stimulus !!!!!!!

press!

1. Present the target stimulus

!!!!!!

Target! Right Waist

Measure the reaction time from stimulus onset to button press

2. Give six random stimulus

!!!!!

3

1

6

5

2

non-Targetnon-Targetnon-Targetnon-Target

non-TargetTarget!4


One Sequence [11sec]

Vibrate!

③Vibrate!

①Vibrate!

⑥Vibrate!

⑤Vibrate!

④Vibrate!

②time [sec]

1. Present Target Stimulus [5sec] 2. Random Stimulus Trial [6sec]

Vibrate!

④

Vibrate Example

[1sec]

Break Time [2sec]

6 7 8 9 101 2 3 4 50

Vibration Each Stimulus [1sec × 6]

"12

Present Target [2sec]

❖ The detail flow of one sequence press!

❖ Experimental condition

"13


Condition Detail

Subjects 7 persons (3 males and 4 females)

Type of stimulus Only tactile (vibration)

Use device Gaming pad “ZEUS VYBE”

Stimulus positions 6 (Left Shoulder,Right Shoulder , Left Waist, Right Waist, Left Leg, Right Leg)

Number of sequences 120

Stimulus duration 300 ms

Inter-stimulus interval (ISI) 700 ms

"14

Next: EEG Experiment

Experimental Procedure❖ The experiments were carried out in two steps


Purpose: !!!!

!

! ? !

!#2 EEG Experiment

! Purpose: !!!!

　・To check whether the P300 response 　could be evoked after vibrotactile stimuli

　・To check whether the users could 　　　perceive the tactile stimulation correctly

❖ Flow of the experiment in each sessions

1. Present the target stimulus

!!!!!!

"15

Classify the user’s brain responses in each target stimulus

#2 EEG Experiment

2. Give 60 random stimulus (10 Targets + 50

non-Targets) !!!!!

3. Users pay attention to the target stimulus

(Generate P300 response) !!!!!

！1

6

5

2

---

-

6

3

5

--

--

3

5

2

1

----…

-Target!

Right Waist

4 !

4 !

4 !4 !

4 !2

3

❖ Experimental condition (additional & changed item)

"16

#2 EEG Experiment

Condition Detail

EEG recording system g.USBamp active EEG electrodes system

Number of channels 16

EEG electrode position 　Cz Pz P3 P4 C3 C4 CP5 CP6 　P1 P2 POz C1 C2 FC1 FC2 FCz

Number of sessions 5

Target stimulus length 250 ms

Inter-stimulus interval (ISI) 350~370 ms

❖ Confusion matrix of all users’ response − Correct rate of response for each target stimulus

"17

#1 Psychophysical Experiment Results

Response

Targ

et

1 2 3 4 5 6

12

34

56

❖ Boxplots of all subjects’ response − Average and distribution of the response time for each target stimulus

"18

#1 Psychophysical Experiment Results

Command

Res

pons

e tim

e[m

s]

300 ms

　20

0 3

00

400

500

60

0 7

00

800

Median value

Minimum value

Maximum value

1/4 value

3/4 value

1 2 3 4 5 6

❖ EEG experiment SWLDA the accuracy for each of the 7 subjects

0%

20%

40%

60%

80%

#1 #2 #3 #4 #5 #6 #7Ave.

42.7%50.0%

73.4%

16.7%

46.7%43.3%

30.0%

38.9%

"19

#2 EEG Experiment Results

0%

20%

40%

60%

80%

100%

#1 #2 #3 #4 #5 #6 #7Ave.

76.2%

100%100%

50%

66.7%

83.3%

50%

83.3%

Acc

urac

y [%

]

Average accuracy rates Max accuracy rates

Users Users

Chance level: 16.7%

❖ EEG mean result in Cz channels

"20

: Target : non-Target

#2 EEG Experiment Results

❖ Information transfer rate (ITR) − Commonly used to assess the performance of BCI

"21

#2 EEG Experiment Result

R: the number of bits / selection; N: the number of classes (N=6 in this experiment); P: the classifier accuracy (each best result in this experiment); V: the classification speed in selections / minute (V=5.56 in this experiment);

❖ EEG experiment ITR results of 7 uses

"22

Subject Max accuracy [%] ITR [bit/min]

#1 83.3 4.30

#2 50 1.18

#3 83.3 4.30

#4 66.7 2.48

#5 50 1.18

#6 100 7.18

#7 100 7.18

#2 EEG Experiment Result

❖ The effectiveness of proposed method was proven as follows: − This approach could be applicable easily for ALS patients − The BCI command identification performance of each target stimulus has been very high − The P300 response appears very clearly in the all tested electrode channels

"23

Conclusions

Spatial tactile brain-computer interface paradigm by applying vibration stimulus to large body areas

Science

Transcript of Spatial tactile brain-computer interface paradigm by applying vibration stimulus to large body areas