978-1-4244-9350-0/11/$26.00 ©2011 IEEE 583

2011 4th International Conference on Biomedical Engineering and Informatics (BMEI)

A Collaborative Brain-Computer Interface

Yijun Wang, Yu-Te Wang, Tzyy-Ping Jung*

Swartz Center for Computational Neuroscience University of California San Diego

San Diego, USA

Xiaorong Gao, Shangkai Gao Dept. Biomedical Engineering, School of Medicine

Tsinghua University Beijing, China

Abstract—Electroencephalogram (EEG) based brain-computer interfaces (BCI) have been studied for several decades since the 1970s. Current BCI research mainly aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. The BCI technology can also benefit normal healthy users; however, little progress has been made in real-world practices due to low BCI performance caused by technical limits of EEG. To overcome this bottleneck, this study uses a collaborative BCI to improve overall performance through integrating information from multiple users. A dataset involving 15 subjects participating in a Go/NoGo decision-making experiment was used to evaluate the collaborative method. Using collaborative computing techniques, the classification accuracy for predicting a Go/NoGo decision was enhanced substantially from 75.8% to 91.4%, 97.6%, and 99.1% as the number of subjects increased from 1 to 5, 10, and 15, respectively. These results suggest that a collaborative BCI can effectively fuse brain activities of a group of people to improve human behavior.

Keywords-Brain-computer interface (BCI); collaborative computing; Electroencephalogram (EEG); human performance; Go/NoGo decision making

I. INTRODUCTION

The human brain is the most complex system in the world. The functional brain imaging technologies such as functional magnetic resonance imaging (fMRI) and Electroencephalogram (EEG) give us an opportunity to observe brain activities related to thoughts, emotions, and behavior, and therefore, help us understand the relationship between the brain and behavior. Recently, a new technology known as brain-computer interface (BCI) or brain-machine interface (BMI) has made a significant progress in brain science [1]. The BCI study covers the three aspects in exploring the human brain: understanding the brain, protecting the brain, and creating the brain. During the past two decades, the BCI technology has become a hot research topic in the areas of neuroscience, neural engineering, medicine, and rehabilitation [2][3].

In essence, a BCI is a communication channel that bypasses the traditional pathway of peripheral nerves and muscles, and creates a direct link between the human brain and an output device [1]. Currently, the main focus of BCI research lies in the clinical use which aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. In current BCI systems, commonly used neural recording technologies include EEG, Magnetoencephalogram (MEG), Electrocorticogram (ECoG), fMRI, near infrared spectroscopy (NIRS), and neuronal

recording. Among these methods, EEG is the most widely used modality in current BCI studies due to its advantages such as simple and inexpensive equipment, flexibility and mobility, and short time constants. In present-day BCIs, the following EEG signals have been paid much attention: visual evoked potential (VEP), sensorimotor mu/beta rhythms, P300 evoked potential, slow cortical potential (SCP), and movement-related cortical potential (MRCP) [1].

Although the EEG-based BCI technology has achieved great successes, moving a BCI system from a laboratory demonstration to a real-life application still poses severe challenges to the BCI community. Applications of the BCI technology are very limited due to bottleneck problems including high system cost, low communication speed, low recognition accuracy, and easy user fatigue [4]. To overcome these problems, a practical solution is to develop a multi-user collaborative BCI system, which can utilize collective intelligence from a group of users. Recently, we first proposed the framework for a collaborative BCI system and further investigated the feasibility and practicality of the system [5]. The development of group-synchronized neural recording systems and group collaborative cognitive computing methods will open a totally new direction for BCI research.

In this study, we propose to study the feasibility of using a collaborative BCI system to improve human decision making in a Go/NoGo decision-making task. In the Go/NoGo task, the N2 event-related potential (ERP) component, which reflects the processing of motor inhibition, will be used as a feature for identifying the NoGo condition. To evaluate the performance of the collaborative BCI, EEG-based prediction of a Go/NoGo decision will be executed using a single-trial classification paradigm and a collaborative classification paradigm respectively.

II. METHODS

A. System diagram

Figure 1 shows the system diagram of a collaborative BCI. Similar to a single-user BCI, a collaborative BCI consists of three major parts: a data acquisition module, a signal processing module, and a command translation module. Consequently, there are three major procedures in system operations:

1) Brain signals from a group of users are acquired by multiple EEG recording devices, and then are synchronized with common environmental events.

584

2)

3)

ComfromdatanewhavEEGpro

B.

sevdes

1)

2)

3)

IdeparparrecosignA cBCmay

1)

Integrated EEfeatures for d

Extracted fetranslated to to give senso

mpared to a sinm multiple usa recording an

w algorithms ve to be develoG analysis, wh

ocessing modul

Figure

System implem

The implemeveral specific signs:

Multiple EEindependently

Multiple-subjsynchronizedevents.

Multiple-subjprocedures ha

eally, the systeradigm similar radigm, EEG dorded, then thnal processing centralized para

CI system; howy be limited fo

Data transmisystems needsending/receirequires highand storage. standing and EEG systemsare more pre

EG and event ddecoding users’

atures from aoperation comry feedback to

ngle-user BCI, ers will lead t

nd signal-procefor implemen

oped to performhich plays the le.

1. System diagram

mentation

entation of a requirements

EG recordingy and simultan

ject data nd with respect

ject data recave to be perfo

em can be imto a conventio

data from multhrown into a and command

adigm is optimwever, practicar the following

ission: When wd to be conneciving in real th capacities of

Because userwalking are al

s, portable and ferable in natu

data are process’ intentions.

a group of usmmands, which

the users.

the complexityto technical chssing procedurnting collabor

m the proceduremost importan

m of a collaborative

collaborative for hardwar

g systems nneously.

need to be to the commo

cording and ormed in real tim

mplemented usional BCI (Figutiple subjects conventional

d translation usimal for designinality of systemg reasons:

wired EEG systcted to the datime. Thereforedata communirs’ natural belways limited wmobile EEG r

ural environme

sed for extracti

sers are direccan also be us

y of system inphallenges in bores. For examprative computie of collaboratint role in the da

e BCI.

BCI has posre and softwa

need to wo

received aon environmen

data processime.

ing a centralizure 2(a)). In thare received aBCI module fing a data serv

ng a collaboratim implementati

tems are used, ta server for dae, the data sevication, memo

ehaviors such when using wirrecording devicents. Then, a da

ing

ctly sed

put oth ple, ing ive ata

sed are

ork

and ntal

ing

zed his

and for

ver. ive ion

all ata ver ry, as

red ces ata

serwir

2) Comacurreqcolinvinccolcomper

3) Syscondevshoeveconperof

Figure 2collabora

To this stuimplem2(b), thsubsystBCI sucapabilparadigand theprocessbeen wparadigcollaboparadig

rver requires a reless infrastru

omputational cachine learningrrent BCI stuquire large amllaborative BCvolved, the crease. Becaullaborative BCmputation, the rformance CPU

stem robustnesnsists of muvices. To assurould have the aen when subsnnection loss)rformance shoua subsystem or

2. (a) A centralizeative BCI.

solve the probudy proposes

mentation of a he whole systtems and a simubsystem worklity in EEG dgm, the amoune data server, asing, are signif

well studied in pgm is a moreorative BCI. Tgm is that the o

low-latency, hucture, which m

cost: Advanceg techniques h

udies [6] [7]. mount of comCI where a lar

computationaluse real-time CI will leaddata server ha

Us and large am

ss: A collaboraultiple EEG rre the stability ability to keep

sets of the wh). In other wuld not be serior subsystems.

ed paradigm and

lems existing ia distributed collaborative

tem consists omplified data ks independentdata acquisitiont of data transas well as the ficantly reduceprevious studiee practical solThe only disadoverall costs of

high bandwidthmight be very c

ed signal prohave been wiThese approa

mputational resrge amount ofl cost will

data proced to a large as to be equippmounts of mem

ative BCI systerecording andof the system,

p the whole syshole system fawords, the ovously affected

(b) a distributed

in the centralizparadigm to BCI. As show

of multiple disserver. For eatly, each subsyon and processmitted betweencomputational

ed. The single-es. Therefore thlution for impdvantage of thf the system har

h, and reliable ostly.

ocessing and dely used in aches always sources. In a f subjects are

significantly essing in a

amount of ed with high-

mory.

em inevitably d processing , the software stem working ail (e.g., data verall system by the failure

paradigm for a

zed paradigm, facilitate the

wn in Figure stributed BCI ach subject, a ystem has its ssing. In this n subsystems cost for data user BCI has

his distributed plementing a he distributed rdware might

585

incrfor cansys

C.

persingexpbe phoimatargquiappsubwercodfor

D.

eacindremsegremThicon(SVeacesti

E.

colclasvotclasa w

whweiclasthe

A.

imamebetwthe shomig

rease due to theach user. In

n be integrated tem cost, and i

Go/NoGo dec

Following a rformed in altegle-photographperimental setu

found in [otographs werages and flasheget, subjects hickly and accupeared, subjectbject, 32-channre recorded todes at a 1000 Hoffline analysi

Single-trial EE

This study pech subject usingdependent commove eye-movegments in a prmoving the ERPird, the intercncatenated, andVM)-based clach subject, a imate the classi

Collaborative

Using the dislaborative datassifier [11], whting system. Insses are labeled

weighted voting

ere m is the nuight and y(i) ssifier was traitraining accur

Event-related

As shown in age onset, the dial frontal coween the Go aFz electrode)

owed a larger Nght reflect the

he employmenpractice, portinto the EEG

improve system

ision-making e

Go/NoGo parnation an "anh recognitionup and the ima9]. During te randomly med for 20 ms ohad to lift theurately as posss had to withh

nel EEG data oogether with stHz sampling rais, totally resul

EG classificati

erformed a sing a standard ma

mponent analyement and musredefined timeP baseline calccepted ERPs d then inputtedssifier to predi10x10-fold c

ification perfor

e classification

stributed systema analysis washich consists o

n the case of a bd as +1 and -1

g can be describ

m

i 1

sign

umber of subjeis the output ined as a sub-

racy was used a

III. R

potentials

Figure 3, durN2 ERP comp

ortex (MFC), sand NoGo cond. Compared toN2 componentmotor inhibit

nt of a data-protable data-procrecording dev

m practicality a

experiment

aradigm, 15 imal" categorin task. Dges used in thethe experimen

mixed with noon a computer eir finger fromsible. When noold their buttonof 10 blocks (timulus/responte and downsalting in 500 tria

ion

ngle-trial EEG achine-learning

ysis (ICA) wascle artifacts [1e window werculated within [

from all 32 d to a supportict the Go/NoGcross validatiormance.

m paradigm (Fs performed wof multiple subbinary classificrespectively, t

bed as follows

iyiw )()(

ects, w(i) is theof a sub-clas

classifier for eas the voting w

RESULTS

ing 180 ms - ponent, which showed a signiditions (paired

o the Go trials,t (-9.8 uV vs. tion process. A

ocessing platforcessing platformvice to reduce ts well [8].

human subjezation task andetails of te experiment cnt, target (G

on-target (NoGscreen. For ea

m the button on-target imagn press. For ea(100 trials eacnse related eveampled to 200 Hals per conditio

classification g paradigm. Firas employed 10]. Second, ERe extracted af[-100 ms - 0 melectrodes we

t vector machiGo decision. F

on was used

Figure 2(b)), twith an ensemb

-classifiers andcation where twthe procedure f:

(

e subject specissifier. An SVeach subject, a

weight.

250 ms afterlocated over t

ificant differend t-test, p<10-5,, the NoGo tri-4.5 uV), whi

A subsequent

rm ms the

cts d a the can Go) Go) ach

as ges ach h), ent Hz on.

on rst, to

RP fter

ms]. ere ine For

to

the ble d a wo for

(1)

ific VM and

an the nce , at als ich P3

componmedial patternconditiodecisio

Figure 3.condition

B. Sing

Thesignificprior tosubjectsubject75.8±6courseswas enh74.4±6ms to 2suggestby sing

Figure 4.line indic

nent also largefrontal and pa

s of N2 and Pons provide

on using EEG [

. Scalp ERP wavens at all electrode p

gle-trial classif

e single-trial cantly higher to mean responts (377±48 msts when using .7%, range: 64s of the N2 andhanced from 6.3% as the len

250 ms, 300 mted that a Go/N

gle-trial EEG cl

. Accuracy of singcates the chance lev

ely differed undarietal areas. TP3 componentthe basis fo

[12].

e forms and differpositions.

ification

EEG classifithan the channse time (RT) s). Figure 4 sthe time wind

4.4% - 85.2%)d P3 componen61.2±4.1% to 6ngth of time w

ms, and 350 ms,/NoGo decisionlassification.

gle-trial EEG classvel (50%).

der two conditThe distinct spats under the Gor predicting

rence waves unde

ication achievnce level (50%

of the Go triashows the accudow of [0 RT). Consistent wnts, the predict68.1±4.3%, 70window increas, respectively. n can be reliab

sification for all su

tions over the atio-temporal

Go and NoGo a Go/NoGo

er Go and NoGo

ved accuracy %) using data

als across all uracy for all

T] (mean±std: with the time tion accuracy .9±4.5%, and sed from 200 These results bly predicted

ubjects. The dash

586

C.

the 1, intetimaccsubnumrespaccacccollsubaro150decmedmo

Figuof thacrodevi

BCclasimpcollmuthiscoll

be softchacollcollof aplatrese

Collaborative

Figure 5 showlength of time5, 10, and 1

eraction betweme. Using the curacy for prebstantially frommber of subjepectively. Th

celeration of decuracy and thlaborative sys

bjects were incund 200 ms af

0 ms earlier thcoding the grodial frontal cotor inhibition.

ure 5. Classificatiohe window length.oss all subjects (37iation.

IV.

This study demCI to acceleratessification accprovement ovelaborative BCI

uch earlier thans study desiglaborative BCI

The prototypedirectly transfe

ftware requiremallenges that laborative BClaborative BCIan EEG recordtform. Becausearch are stil

e classification

ws the classifice windows use5 subjects ween the numbertime window dicting a Go/N

m 75.8% to 91ects increased

he results alsecision-makinghe number otem. As show

cluded, the Go/fter the stimuluhan the subjecoup ERP activortex, which ar

on accuracy of diff. The vertical line 7 ms). The dash li

CONCLUSION

monstrated an e decision-makcuracy of the r that of the sinI allowed the n his/her actuagned and deI technology to

e system demonerred to an onl

ments can be mhave to be

CI system canI needs multiplding system ane commercial l expensive, t

cation accuracyed for data anaere put togethr of subjects an

of [0 RT], tNoGo decisio.4%, 97.6%, a

d from 1 to so clearly shg depended on of subjects inwn in Figure /NoGo decisious onset, whicct’s actual movities arising mre related to th

ferent numbers of sindicates the mean

ines indicate mean

N AND DISCUSS

application of king in a Go/system show

ngle-user BCI. subject’s decil motor responemonstrated th improve huma

nstrated in the line system if t

met. Currently, resolved be

n become a le BCI platformnd a real-time s

EEG productthe total cost

y as a function alysis. Results fher to show tnd the predictithe classificatin was enhanc

and 99.1% as t5, 10, and 1

howed that tboth the desir

nvolved in t5, when all n could be mah was more th

otor response,mainly from the processing

subjects as a functn response time (R

n accuracy ± stand

SIONS

the collaborati/NoGo task. T

wed a significaFurthermore, t

ision to be manse. In summahe use of tan performance

current study cthe hardware athere are seve

efore an onlireality. First,

ms, which conssignal-processits used for EEt for building

of for the ion ion ced the 15, the red the 15

ade han by the of

tion RT) dard

ive The ant the ade ary, the e.

can and eral ine

a sist ing EG g a

collaborequirecommuplatformserver. the dataadvanctechnolcollabo

ThiBioscieArmy Cooperviews aof the aofficialResearcGovernfor Gonotation

Thethe EEG

[1] J. RVauClin

[2] M. pre

[3] N. Clin

[4] Y. basdes

[5] Y. impdoi

[6] D. KruFeaEng

[7] F. Lof cNeu

[8] Y. inte025

[9] A. Botnatu

[10] A. Dof ana

[11] DudYor

[12] K. DynNeu

orative BCI wils specific softw

unication betwms, and betwFurthermore,

a server has toes in biomedlogy, it will soorative BCI sys

is work is suence Inc. ReseResearch Lab

rative Agreemand the conclusauthors and shol policies, eithch Laboratorynment is authoovernment pun herein.

e authors woulG data.

R. Wolpaw, N. Biughan, “Brain-comn. Neurophysiol.,

A. Lebedev, M. esent and future,” T

Birbaumer, “Bran. Neurophysiol.,

Wang, X. Gao, Bsed on visual evsigns,” IEEE EMB

Wang, T. P. Junproving human pi:10.1371/journal.p

J. McFarland, Cusienski, “BCI Meature extraction ang., vol. 14, pp. 135

Lotte, M. Congedclassification algorural Eng., vol. 4, p

T. Wang, Y. Wanerface for commu5018, 2011.

Delorme, G. Routtom-up and Topural scenes,” Cog.

Delorme, S. Makesingle-trial EEG

alysis,” J. Neurosc

da RO, Hart PE, Srk: Wiley Interscie

Yamanaka, Y. Ynamics Associateurosci. vol. 22, no

ll be high. Secware developm

ween EEG systween the BCI

data processin be implement

dical electronicoon be possib

stem.

ACKNOWLEDG

upported by aearch was alsoboratory and

ment Number sions containedould not be inther expressed y or the U.S

orized to reprodurposes notwi

ld like to thank

REFERENC

irbaumer, D. J. Mmputer interfaces vol. 113, pp. 767-

A. L. Nicolelis, Trends Neurosci.,

ain-computer-intervol. 117, pp. 479-

B. Hong, C. Jia, Svoked potentials: B. Mag., vol. 27, p

ng, “A collaboratperformance,” PLopone.0020422, 201

C. W. Anderson, eeting 2005 - Wond translation,” IE5-138, 2005.

o, A. Lecuyer, F. rithms for EEG-bapp. R1-R13, 2007

ng, T. P. Jung, “Aunication in daily

usselet, M. Mace, p-down processing. Brain Res., vol. 1

eig, “EEGLAB: anG dynamics inci. Meth., vol. 134,

Stork DG, Pattern ence Press, 2000.

Yamamoto, "Singed with Voluntary. 4, pp. 714-727, 2

cond, a collaboment, which allotems and sign

subsystems ang in BCI subted in (near) recs and teleco

ble to impleme

GMENT

a gift fund fro sponsored in

was accompW911NF-10-

d in this documterpreted as rep

or implied, oS Governmenduce and distriithstanding an

k A. Delorme

CES McFarland, G. Pfur

for communicatio791, 2002.

“Brain-machine vol. 29, pp. 536-5

rface research: C483, 2006.

. Gao, “Brain-comfeasibility of p

pp. 64-71, 2008.

tive brain-computoS ONE, vol. 6, 11.

K. R. Muller, Aorkshop on BCI siEEE Trans. Neura

Lamarche, B. Arnased brain-comput.

A cell-phone basedy life,” J. Neura

M. Fabre-Thorpeg in the fast vis19, no. 2, pp. 103-

n open source toolcluding independ, pp. 9-21, 2004.

Classification (2n

gle-trial EEG Pory Response Inhib2010.

rative system ows seamless

nal-processing and the data bsystems and eal time. With ommunication ent an online

from Abraxis n part by the lished under 2-0022. The

ment are those presenting the of the Army nt. The U.S ibute reprints ny copyright

for providing

rtscheller, T. M. on and control,”

interfaces: past, 46, 2006.

oming of age,”

mputer interfaces practical system

ter interface for no. 5, e20422,

. Schlogl, D. J. gnal processing: al Syst. Rehabil.

naldi, “A review ter interfaces,” J.

d brain-computer al Eng., vol. 8,

e, “Interaction of sual analysis of 113, 2004.

lbox for analysis dent component

nd Edition). New

ower and Phase bition," J. Cog.