Building an Ear-EEG System by Hacking a Commercial Neck ...
Transcript of Building an Ear-EEG System by Hacking a Commercial Neck ...
HARDWARE METAPAPER
Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the LabMartin G. Bleichner* and Reiner Emkes†
Electroencephalography (EEG) allows measuring human brain activity for scientific, diagnostic and thera-peutic purposes. For scientific purposes, EEG is traditionally used to study the brain-behaviour relationship under well controlled laboratory conditions while the participant sits quietly in a room. Consequently, our knowledge regarding the brain-behaviour relationship was limited to human behaviour under constrained conditions. The development of mobile EEG hardware in the last years allows moving beyond the lab and studying brain activity in everyday life.
Most EEG systems record from electrodes that are distributed over the head, kept in place with an elastic cap or net. For long term EEG recordings, to study natural behaviour in public, these systems are not optimal, as they are rather bulky and unpleasant to wear for longer periods. We argue for “ transparent”, ear-centered EEG systems instead. Transparent in the sense that both the person wearing the EEG system as well as the people interacting with that person are not disturbed by it.
We describe here the integration of a commercially available research-grade mobile EEG amplifier and a commercially available neck speaker as a mounting system to be used in combination with ear electrodes (cEEGrid). This nEEGlace can be used for ear-EEG recordings in public over extended periods of time. The integrated microphone and the integrated speakers provide the possibility to play and record sound. This allows conducting experiments on auditory perception beyond the lab by both studying the brain responses to naturally occurring sounds as well as to experimental auditory stimuli.
We describe the general design, present a working prototype, outline a number of use cases and discuss future improvements of the proposed design.
Keywords: electroencephalography; ear-EEG; mobile EEG; transparent EEG, beyond-the-lab experimentation
Metadata Overview• Maindesign files:https://github.com/mgbleichner/nEEGlace
• Targetgroup:neuropsychologists,neuroscientists• Skills required:advancedsolderingskills,basic skillsinusingprecisionscrewdriversandpowertools forcutting.
• Seesection(4)BuildDetailsformoredetail.
(1) OverviewIntroductionTheunderstandingofpathologicalandnon-pathologicalbrainfunctionsinhumansreliesonin-vivorecordingsofbrain activityusingelectroencephalography (EEG), func-
tional magnetic resonance imaging (fMRI), magnetoen-cephalography (MEG), positron emission tomography(PET), and functionalnear-infrared spectroscopy (fNIRS).Thesemethodsallowstudyingthephysiologicalbasisofhumancognition.However,most of whatwe know about human brain
functions,we learnedfromimmobilizedparticipants.Asthe different brain recording techniques are all highlysensitivetosignaldisturbancesduetobodymovements,participantseithersitorliedownandareaskedtomoveaslittleaspossibleduringanexperiment.Thismeansthatweneglectbrainfunctionsduringmotionandduringnat-uralbehaviourdue to the technical limitationsof thesebrainrecordingtechniques.It has become increasingly clear that there are differ-
ences in brain function during rest and duringmotion.OppezzoandSchwartz(2014)show,forexample,thatwetendtobemorecreativewhilewewalk.Inthelastyearsthenewfieldofmobilecognitiveneuroscience, lookingintotheunderlyingneuralmechanisms,isemerging(Debener,Minow,Emkes,Gandras,&DeVos,2012;Gramann,2011,
Bleichner, MG and Emkes, R. 2020. Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab. Journal of Open Hardware, 4(1): 5, pp. 1–10. DOI: https://doi.org/10.5334/joh.25
* Neurophysiology of everyday life group, Department of Psychology, University of Oldenburg, Oldenburg, DE
† Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, DE
Corresponding author: Martin G. Bleichner ([email protected])
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 2 of 10
2014;Gramann,Jung,Ferris,Lin,&Makeig,2014;Ladouce,Donaldson,Dudchenko,& Ietswaart,2017).This interestinmobilecognitionisparalleledbytechnicaladvancesinbrain recording techniques.Newmobilebrain recordinghardwareallowsleavingtheclassicallabsetupandstudy-ing brain functions duringmotion or in everyday situa-tions. The increasing interest in these questions, on theotherhand,acceleratesthedevelopmentofnewhardware.MobileEEGstandsattheforefrontofthisdevelopment.
Unlike fMRI,MEGor PET, EEG can easily bemadeport-ableandevenwearable.EEGmeasurestheelectricalactiv-ityofgroupsofneuronsusingelectrodespositionedonthescalpandasignalamplifier.Small,lightweight,mobileEEGamplifiersallowrecordingEEGvirtuallyanywhere,forexampleduringbungeejumping(Nann,Cohen,Deecke,& Soekadar, 2019).Mobile EEG setups generally consistofacapwithintegratedelectrodes(Figure 1A),amobileamplifier,andalaptoporasmartphonewhichisusedfordata acquisition and stimulus presentation, e.g. for pre-senting experimental stimuli such as sounds or images(Bleichner&Debener,2017).For long-term EEG recordings in public we envision
‘transparentEEG’(Bleichner&Debener,2017).Transparentinthesensethatneitherthewearersnortheirinteraction
partners notice the recording device. The wearers canforget about thedevice andbehave asnaturally as pos-sible.TheclassicalEEGcap,whichisusedinmanymobileEEGsetups isnotoptimal for long termEEGrecordingsinpublic.EEGcapsposeseveralconstraintsonthedura-tion,comfortandnaturalnessoftherecording.First,EEGcapsareoftenunpleasanttowearoverextendedperiodsof time. Second, the conductive gel which is necessaryforagoodskinelectrodecontactgraduallydriesoutandleadstoareductioninsignalqualityovertime.Dryelec-trode systems exist, but have an inferior signal quality(Radüntz,2018)andareevenmoresusceptibletomove-ment artifacts. Third, EEG capsmay alter the behaviourofparticipants,e.g.bymakingtheminsecureabouttheirappearance. As a consequence, participants may avoidcertainsocialsituationsandtherebychangetheirnormalbehaviour.Finally,capsmaychangethereactionsofoth-ers towards the participant (i.e.making a natural inter-action less likely).While thismay be tolerable inmanyresearchcontexts,itisproblematicforlongtermrecord-ingsineverydaysituations.Forourusecases,theEEGamplifieraswellastheEEG
sensorsshouldbedisguisedorhiddenfromviewasmuchaspossible(Figure 1B),butshouldstillallowrecordingthe
Figure 1:A)MobileEEGsetupusinganelectrodecap.Therecordingselectrodes(red)aredistributedoverthescalpandarekeptinplacebyanelasticcapthatissecuredinplacewithastrapunderthechin.TheEEGamplifierisattachedtothebackofthecap.Thesmartphone,thatisusedforstimuluspresentationanddataacquisition,isattachedwithapouchontheupperarm.ThecommunicationbetweensmartphoneandamplifierrunsviaBluetooth.Audiostimu-lationisdoneviain-earheadphones.B)MobilenEEGlacesetupusingcEEGrids.ThecEEGridsareconnectedtothenEEGlacewithconnectorcable(lightblue)thatsticksoutoftheneckpiece.Themobileamplifierisintegratedintotheneckpiece.Thedataacquisitionisdoneonthesmartphonethecommunicationbetweensmartphoneandampli-fierrunsviaBluetooth.Audiostimulationandaudiorecordingisdoneviatheneckspeakerthatisconnectedwithacinch-cinchcable(white)withthesmartphone.
Pouch with Smartphone for data acquisition and stimulus presentation
EEG cap with attached amplifier
Cap Setup nEEGlace Setup
cEEGrid connected to nEEGlace
nEEGlace: neck speaker with integrated mobile EEG amplifier
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 3 of 10
brainprocessesofinterest.Thesetwocriteriaaredifficulttosatisfysimultaneously.Ontheonehand,thebestsystemtorecordbrainactivitywouldhavealargenumberofelec-trodes,widelydistributedoverthehead.However,tokeeptheelectrodesinplace,somekindofnetorcapisneeded,which tends to be very noticeable. Some attemptsweremadetohidetinyelectrodesinthehair(Nikulin,Kegeles,& Curio, 2010), but the problem of keeping those elec-trodesinplacecomfortablyandinconspicuouslyremains.Ontheotherhand,themostdisguisedsystemwouldnotbe noticeable at all. Consequently, such a systemwouldonlyrecordfromalimitednumberofelectrodesandthere-forewouldonlybesensitivetoalimitedamountofbrainprocesses(Meiser,Tadel,Debener,&Bleichner,2020).Wearguethatin-ear(Kidmose,Looney,Ungstrup,Rank,
&Mandic,2013;Looneyetal.,2011)andespeciallyaround-ear(Debener,Emkes,DeVos,&Bleichner,2015)EEGsolu-tions provide a satisfactory compromise. Ear-EEGs areinconspicuousenoughtobeusedinpublicandaresensi-tivetoavarietyofneuronalsignals(Bleichner&Debener,2017; Bleichner, Mirkovic, & Debener, 2016; Debeneretal.,2015;Denketal.,2018;Mirkovic,Bleichner,DeVos,&Debener,2016),notlimitedto,butespeciallytothosethatoriginateinthetemporalcortex(Meiseretal.,2020).Earelectrodescanstay inplacefor longperiodsoftimeor be re-positioned accurately (e.g. individualized ear-moldsforin-earEEG),whichallowsforconstantrecordingconditions.Inafeasibilitystudy(Hölle,Meekes,&Bleichner,2020),
wecombinedamobileEEGamplifier,withearelectrodes,attachedtoapairofheadphoneswornaroundtheneck(usedtoplaysounds)torecordEEGforanentireworkday.Weshowedthattherecordedbraindatawasofsufficientqualitytostudyauditoryattention.Forthatstudywecom-binedseveralhardwarepartsastherewasnocommercialEEGsystemavailable that fulfilledour requirements.Wefoundthatparticipantstoleratedthesetupwellandcoulddotheirnormalofficeworkduringtherecording.However,thatstudyalsomadeclearthattheusedsetupwasfarfromoptimal,bothinregardtousercomfortandvisibility.Herewepresentamoreintegratedprototypethatsolves
someof these issues.Wedesigneda research-gradeEEGsetupthat issufficiently inconspicuoustoallowforEEG
recordingineverydaysituations,thenEEGlace(Figure 2).ForthisweintegratedmobileEEGhardwareintoaneckspeaker(i.e.aconsumerproducttoplaymusicsimilartoaheadphone;thespeakersareintegratedintoaneckpiecewhich iswornaround theneck, theears remainuncov-ered)andcombineditwithear-EEGelectrodes(cEEGrids).ThisEEGsetupusessmartphonebasedsignalacquisitionand allows for stimuluspresentation (audio) and soundrecording.
Overall Implementation and DesignThenEEGlaceconsistsofthreecomponentsthatarecom-binedintoasingleneckset:aneckspeaker1,anEEGampli-fier2, anda connector to connect to cEEGrid3 electrodes(around-the-earsensorarray).Theneck speaker serves as amount for the amplifier
andbringsthecEEGridconnectorsclosetotheears.Thespeaker also allows playing back sounds, an integratedmicrophoneallowsrecordingallsoundsthepersonwear-ingthenEEGlaceiscurrentlyexposedto.Forourdesign,partsoftheneckspeakerareremoved
tomakeroomfortheamplifierandtheconnectors.Theoriginalelectrodeconnectorof theamplifier is replacedbyspecialcEEGridconnectorsthatarepositionedclosetotheears.
Universality of the DesignTheideaofaneckpiecetomountEEGhardwarecanbeimplemented inmany differentways. The neck speakercanbereplacedbyanotherbrandwithasimilarformfac-tor,orbebuilt fromscratch(e.g.3Dprinted).Anyotherkindofmobileamplifiercouldbeused,ifthespecsofthespecific device are suitable or at least sufficient for theresearchquestion.Thesameholdsfortheelectrodesused.
(2) Quality ControlSafetyTherearenospecificrisksanticipatedforthisdesign.Allrisksoftheendproductarealreadypresentforthesub-componentsintheiroriginalformasdescribedbelow.
EEG AmplifierEEGisawell-establishedtechnologytorecordbrainactivitynon-invasively. It isconsideredsafe,withnonegativeeffectsfortheparticipants.Themobile
Figure 2:nEEGlaceforrecordingear-EEGforextendedperiodsinpublic.A)backview;B)frontview;C)topview.ThemobileEEGamplifierisintegratedintothebackpartoftheneckspeakerwherethebatteryoftheneckspeakerwaslocated.
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 4 of 10
amplifiersusedhereareCEcertifiedintheiroriginalform(however,allwarrantyislostonceitisalteredinthewaywedescribehere).Ifthemodificationsaredonecorrectlyweexpectnoproblems.The largestsafetyrisk is relatedtothebatteries.Themobileamplifiersarepoweredbyalithium-ion battery. These batteries need to be handledwithcare.Batteriesthataredamagedduringtheassemblyprocessmustnotbeused.Adamagedlithium-ionbatterymycauseshortcircuit,leadingtoathermalrunawayandapotentialfirehazard.Asstatedintheusermanualofthesmartingamplifier:4theamplifiermustnotbeusedwhilecharging!
Neck SpeakerTheneckspeakerisacommercialprod-uct,andnoadditionalrisksduetotheredesignareantici-pated. Due to the removal of lithium-ion battery, anybatteryrelatedriskisremovedaswell.Thelargestriskmaybethesharpedgesonthecasingifthealterationsarenotdonewithcare.Thiscaneasilybemitigatedbyworkingcarefully,andfilingoffallproblematicedges.
Ear ElectrodesThecEEGridelectrodesaremadefrombio-compatiblematerialandnonegativesideeffectsareknown(Debeneretal.,2015).Theusedelectrodesarepas-sive.Theelectrodesareplacedwithadoublesidedstickertotheskin.Abitofconductivegel isneededforagoodelectrode-skin contact. Sensitive peoplemay experienceallergic reactions to the glue or the conductive gel. Forthosecases,hypoallergenicalternativesareavailable.
Connector Cables The amplifier is connected to thecEEGridwiththeconnectorcablesandthecEEGrid’stailsare inserted into theconnector.Thisconnection is tightenoughtokeepthecEEGridsinplaceevenduringmove-ments. However, the connection would come loose ifthereiseitherastrongpullonthecEEGridortheconnec-torcable(e.g.byfast,largeheadmovements).Inoddcasesitcouldhappenthatthiscableconnectiongetscaughtinsomething.InthatcaseitmayhappenthatthecEEGridispulledoutoftheconnector,thattheconnectorcabletearsfromtheamplifier,orthecEEGridstearofftheskin.Thiswouldleadtoadamageofthehardwarecomponentsordiscomfortforthewearer,respectively.Thesystemisnotsplashwaterprotectedandcannotbe
usedinwaterorduringprecipitation.
General TestingThe neck speaker, the EEG amplifier, and the ear elec-trodes are commercially available products. The func-tionalities of all sub components are well establishedandhavebeenvalidatednumeroustimes.Obviously,thedis-andreassemblyforourpurposemaycompromisethefunctionalityofanycomponent.Beforeusingthissetupproductively, one should run a couple of simple teststo assure the functionality of the system including thespeakersandthemicrophones.Onequickandeasytesttoseewhetherneurophysiologialsignalscanbemeasured,
Figure 3:SchematicofthecommunicationbetweenthenEEGlaceandasmartphone.ThesmartphonerunstheSmart-ingAppthatcollectsEEGdataandeventmarkerinformation.TheEEG signalisreceivedviaBluetoothfromtheEEGamplifier.ThesmartphonerunsthePresentationApptopresentaudiostimuli.Theaudio signalistransmittedbyanauxconnection(cinch-cinch)totheneckspeaker.ThePresentationAppsendsmarker signal(eventmarkers)totheSmartingApptosynchronizetheEEGsignalandthepresentationofanaudiosignal.Infutureversionstheaudiotransmission,bothaudiopresentationandaudiorecordingshouldbewirelessaswell.
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 5 of 10
istorecordfromapersonsittingquietlyfirstwithopenandthenwithclosedeyesforsomeminutes.Closingtheeyeswill lead toacharacteristic change in the signalataround10Hzwhichisoftenlargeenoughtobeseenintherawsignal.Thischangeinthe10Hzrange(socalledalphaband),isoneofthelargestandmostrobusteffectsthatcanbeobservedinEEG.Thealphaactivityincreaseswhenclosing theeyesanddecreaseswhenopening theeyes.Figure 4(A–D) showsthetimelineofthesimpleexperiment and the expected outcome for a recordingusingthenEEGlace.
(3) ApplicationThenEEGlace incombinationwithear-electrodescanbeused to record neurophysiological data over extendedperiodsoftimeineverydaysituations.Itsprimaryareaofapplicationisresearch.Theneckpieceandtheelectrodesaroundtheeararecomfortabletowearandarenotdirectlyapparent to others. If you build this system yourself,itmust not beused for clinical applications (in Europe,medical devices require CE-Marking to declare that thedeviceisincompliancewithalllegalrequirementsoftheEuropeanmarket.Thisisnotthecaseforthisdevice).
The setup of the system is simple and will not takelonger than5–10minutes.First, the skinof thepartici-pantispreparedforelectrodeplacement.Forthistheareaaroundtheearsiscleanedwithalcoholwipes.Second,thecEEGridsareprepared.Abitofconductivegelisappliedtoeachelectrode.Thecoverofthepre-appliedsticker isremovedfromthecEEGrid,andthecEEGridispositionedaroundtheear.Third,thenEEGlaceisplacedaroundtheneck.Fourth,thecEEGridsareconnectedtothenEEGlaceby inserting the tail of each cEEGrid into the connec-tor. If the audio capabilities of the nEEGlace should beused, a cinch-cinch audio cable canbeused to connectthenEEGlace and the smartphone (or a laptop). Finally,thedatatransmissionbetweenamplifierandtherecord-ing program on the smartphone (or laptop, see sectionOperatingSoft-wareandPeripherals) canbe startedandelectrodeimpedancecanbechecked.Thestimuluspres-entationandsignalacquisitioncanbestarted.Amobilerecordingcancontinueuntilthebatteryofthe
amplifierorthesmartphonegoflat.Aftertherecordings,theelectrodescanbedisconnectedfromtheconnectors.ThecEEGridscanberemovedfromtheskinandtheskincanbecleanedwithtissuepaper.
Figure 4: Simplevalidationexperiment for thenEEGlace:A) ExperimentalSetup–WithallnEEGlacecomponentsinplace (seesection (3)Application), sitquietly inachair for fourminutes,witheithereyesopenoreyesclosed(60secondseach,repeattwice).B)Timecourse–SinglecEEGridchannelasrecordedwiththenEEGlaceduringtheeye-open,eyes-closedexperiment(channelR07).Thedatashownhereisrecordedfromthefirstauthor.Thesignalisband-passedfiltered(0.1to45Hz).TheresultingEEGsignalfluctuatesovertimewithanamplitudebetween–100and100μV.C)Timecourse–SamedataasinB)butfilteredinthe“alphaband”range(7to13Hz).Theclearincreaseinamplitudefortheperiodswithclosedeyesisreadilyapparentinthefilteredsignal(i.e.,comparethefirst60sec-onds,withthesecond60seconds).D)Powerspectra–Powerspectraforeach60secperiodofeyes-open(firstandthirdcolumn)andeyes-closed(secondandforthcolumn).Fortheeyes-closedconditionalargeincreaseinpowerinthealpharange(10Hz)comparedtotheeyes-openconditionisevident.
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 6 of 10
Use CasesStudy of Natural Fluctuations in Brain DynamicsEEG is suitable to studybrainprocesseson several timescales – from brain processes that unfold within a fewmilliseconds, to brain processes that unfold over hoursor even days (e.g. attention, fatigue, circadian rhythms,hormonal cycles). While fast brain dynamics can easilybestudiedinclassicalEEGrecordingssessions,whichdonotlastlongerthan1–2hours,slowerbraindynamicsaremoredifficulttostudy.Labrecordingsofmorethan2–3hoursareverytiringto
theparticipants.Thismakesitdifficulttogatherdatathatis representative of natural brain processes during dailylifeactivities.Thealternative,multipleshorterrecordingswhicharespreadoutoverthecourseofaday,hasseveralproblemsaswell.First,thenecessaryvisitstothelabmightalreadyinfluencethebrainprocessesofinterest.Second,with standard EEG caps, the exact re-positioning of theelectrodesisnotpossible.Theindividualrecordingswouldthereforenotdirectlybecomparablewitheachother.ThenEEGlaceisconvenienttowearforlongperiodsof
time and the signal quality can be expected to be highthroughouttherecording.Theear-electrodesremainathesamespotandnore-attachmentoftheelectrodesisnec-essary, assuring constant recording conditions. PreviousstudieshaveshownthatthecEEGridscanbewornwithoutdiscomfortformorethan8hoursandevenduringsleep(Bleichner&Debener,2017).Thoughnot fully transpar-ent,participantscanmovemoreorlessfreelyandengagein everyday activities. It is hence conceivable that brainactivity canbe recorded in different everyday situationsandthatnaturallyoccurringbraindynamicscanbestud-iedmoreeasily.
Sound Perception in Everyday LifeInourresearch,weareinterestedinsoundperceptionineverydaylife(Hölleetal.,2020).Wewanttounderstandhowthebrainprocessesnaturallyoccurringsounds,andwhywearedisturbedby somesoundsbutnotbyothersounds.Oursoundsensitivityissituationdependentandvariesover time.WithEEGrecordings in the labwecanonlygetsnapshotsofthebrainactivity.WiththenEEGlaceincombinationwithearelectrodeswecanmovebeyondthe lab and study sound perception in natural sound-scapes.TherecordingfunctionalityofthenEEGlaceallowsmonitoring theexisting soundscape.Theplayback func-tion allows presenting experimental sound stimuli inadditiontothenaturallyoccurringsounds.
Reuse Potential and AdaptabilityOur design has a high degree of adaptability. DifferentneckspeakersanddifferentEEGamplifierscouldbeused.Additional(bio-)sensorscouldbeaddedtotheneckpieceforfurthersensingcapabilities.Additionalbatteriescouldbeaddedtoextendtherecordingtime.Thebuilddescrip-tion can be found on GitHub (mbgleichner/nEEGlace)5.Others are free to fork from this project and add addi-tionalinformationordescribealternativeapproaches.ThenEEGlacecouldalsobebuiltusingtheopensource
EEGhardwarecomponentsfromtheOpenBCI6project.
(4) Build DetailsAvailability of Materials and MethodsThe system is built from an EEG amplifier (Smarting,mBrainTrain, Belgrade, Serbia), a JBL neck speaker (JBLSoundgearBTA)andtwocEEGridconnectors.TheSmartingamplifieriscommerciallyavailablefromseveralspecialisedEEG resellers and from the manufacturer (www.mbrain-train.com). The JBL is commercially availableonline. ThecEEGridsarecommerciallyavailablefromdifferentcompa-nies(e.g.www.tmsi.com).ThecEEGridconnectorsarenotcommerciallyavailable,
butneedtobebuiltfromseveralcomponents7.Thecon-nectorPCBisnotcommerciallyavailable.ThePCBdesignisavailableonline,weprovidesourceswherethePCBcanbeproduced(describedinthemaindesignfiles).
Ease of BuildThecurrentdesignrequiresaconsiderabledegreeofdex-terity,afairunderstandingofelectronicsandgoodtools(rotary power tool, soldering iron, precision screw driv-ers,clamp,cutterknife,rasp/file,pliers).Partsoftheneckspeakercasingandoftheamplifiercasingneedtobecutout.Theoriginalconnectorcablesneedtobesolderedoffoftheamplifierboard,thecEEGridconnectorsneedtobesolderedontotheboardoftheamplifier. (Beawarethatthis step can permanently damage the amplifier if notdonecorrectly!Theauthorsofthisarticle,arenotrespon-sibleforanydamagetoyourdevice.Byfollowingthestepsinthisdocument,youwillloseallmanufacturerwarranty.Given the costs for the EEG amplifier, you should havesomeexperiencewithallstepsinvolved).Fortherest,allnecessarystepsarerelativelysimple.
Operating Software and PeripheralsTheSmartingamplifiercomeswithproprietarysoftwareforWindowsandAndroid(seemanufacturerswebsitefordetails). For ourworkwe use the Android version on asmartphone.WeusetheSmartingapptoconnecttotheamplifier via Bluetooth, and store the EEG data on thesmartphone.Forourexperimentsweuseproprietarysoft-ware(Presentation,NeurobehaviouralSystems8)topresentexperimentalstimuli(e.g.sounds)toassurehightemporalprecision(seeFigure 3).Weuseaself-builtapptorecordsound(whichwillbedescribedelsewhere).
Open Source AlternativesThere are open source alternatives to the proprietarysoftwareweuseforinterfacingwiththeamplifierandforstimuluspresentation.TheSmartingamplifierworkswithOpenVibe9 which allows collecting EEG data. OpenVibeis an open source software (Windows, Linux) for braincomputer interfaces and real time neuroscience (AGPLlicense). OpenVibe comes with an acquisition server tocommunicate with various hardware signal acquisitiondevices(includingtheSmartingamplifier)usinghardwarespecificmodules(drivers).Forstimuluspresentationtherearevariousopensource
alternatives.OpenSesame10isacross-platform(Windows,MacOS,LinuxandAndroid(runtimeonly))python-basedstimuluspresentationsoftware(GPL3license).PsychoPy11
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 7 of 10
isanotherpython-based,free,cross-platformpackageforimplementing and running experiments (GNUGPL v3+license).AthirdoptionistousePsychtoolbox12,atoolboxforstimuluspresentationforMATLABandOctave.
Necessary Parts• JBLSoundgearBTA1
• SmartingEEGAmplifier2
• cEEGridconnectors7
• FemaleJackPlugSocket
Hardware Documentation and Files LocationName:nEEGlace�Persistent�identifier:https://github.com/mgbleichner/nEEGlace Licence: The hardware design is licensed under theCERNOpenHardwareLicenceVersion2–PermissivePublisher:MartinG.BleichnerDate published:2.July2020.
(5) DiscussionThenEEGlacecombinesaneckspeakerwithamobileEEGamplifier and connectors for around the ear electrodes(cEEGrid).ThenEEGlacecanbeusedtorecordbrainactiv-ityandforpresentingandrecordingsoundinthelabandbeyond. The integrated gyroscopes of the EEG amplifierallowcapturingsomemovementparametersofthewearer.Wepresentthisprototypetomotivateotherswhoaremoretechnicallyinclinedandwhohaveaccesstobetterproduc-tionfacilitiestodevelopandconstructadedicatednEEGlace.Thisversionisacombinationofexisting,commercially
availableelements,ahardwarehack.Initialtestswiththissetupshowthatitisfunctionalbothforrecordingbrainactivityaswellasforpresentingandrecordingsound.Therearenumerouswaystoimprovethecurrentdesign
aswepointoutindetailbelow.AdedicateddevelopmentofthenEEGlacefromscratchwouldallowoptimizingallcomponents.Thegoalistohavearobustandeasytouseresearchplatformforear-EEGrecordingsbeyondthelabineverydaylife.
Future ImprovementDedicated Neck PieceA dedicated neck piece should provide sufficient roomforallnecessaryparts,sufficientbatterycapacity(>8h,orshouldbequicklyrechargeable),goodsoundqualityandhighusercomfort.Itshouldberobustenoughtobeusedcontinuously outside the lab without supervision of anexperimenter.Theneckpieceshouldhaveasnugfitaroundtheneck
toreduceanymovementofthedevicearoundtheneck,and toprevent the loss of theneckpieceduringmove-ment.Furthermore,specialattentionneedstobeputintoassuringalight-weightdesignwithabalancedweightdis-tributiontomaximisewearingcomfortduringlongtermusage.Furthermore, a dedicated design should pay atten-
tiontotheaudioquality.Forourprototypewemayhavecompromisedtheaudioqualitybyopeninguptheneck
speaker,cuttingintoitandbyremovingsomepartsofthehardware(batteryaswellascontrolelectronics).Thismayhavepotentiallynegativeeffectsonthesoundqualityoftheneckspeakerduetolessresonancespaceandrattlingduetoinsufficientlyfixedparts(wedidnotobserveanyoftheseeffectsforourprototype).Overall,theproductqualityneedstobeimproved.This
includesa (water-tight)housingthatprotectstheampli-fier hardware (at least frommild rainfall). The cEEGridconnectorsneedtobesoftandflexibleenoughtoallowforundisturbedmovementoftheparticipantbutrobustenoughtowithstandsomestrain(e.g.duetosuddenlargeheadmovements).Ideally,theentireneckpiecewouldberobustenoughtowithstandcarelesstreatment(i.e.flungonatable)toaccommodateat-homeusagewithoutexper-imenter supervision.Protrudingparts, (i.e. theamplifierinourdesign),shouldbeavoidedinthisregardaswell.
Additional SensorsOur prototype allows playing and recording sound. Inthe current version the sound recording is mono, witha microphone on one side only. Future developmentsshould have at least one stereomicrophone (preferablymore),torecordthenaturalsoundscapesurroundingtheparticipant,inordertorelatethesoundscapetotheongo-ingEEG.InordertocomplywithEuropeanprivacyprotec-tion laws,whichdonotallowrecordingotherpeople inpublic,itisnecessarytostoretheaudioinsuchawaythatthecontentofconversationsortheidentityofapersonisprotected. There are smartphonebased audio recordingsolutionsthatallowthisbyonlystoringsomeaudiofea-tures(Bitzer,Kisser,&Holube,2016;Kowalk,Kissner,vonGablenz,Holube,&Bitzer,2018).TheSmartingamplifierweusedisequippedwithagyro-
scopetomeasuremovementlikemostmobileamplifiers.BydesignthisamplifierisintendedtobeusedwithanEEGcapandisattachedtothebackoftheheadinapouchonthecap.Inthatcasethegyroscopecapturesheadmove-ments inalldirections (pitch,rollandyaw)verywell. InthecaseofthenEEGlace,thegyroscopesarenowlocatedinthenEEGlaceontheneck.Thismeansthatheadmove-mentsarenotnecessarilycapturediftheshoulderswiththe nEEGlace remain still. Future designs should haveadditionalmotionsensors,e.g.attachedonthecEEGridstocaptureheadmovements.Thismovementinformationisnotonlynecessarytoknowwhatapersonisdoing,butcouldalsoprovideinformationaboutmovementinducedartifacts(e.g.duetoelectrodedisplacementduetoheadmovement), and could consequently help with theirremoval(O’Regan&Marnane,2013).A dedicated neck piece could also mount numerous
additional sensors.Dependingon theuse case a varietyof sensors is conceivable, suchas lightand temperaturesensors togetmore informationabout the surroundingoftheperson.
Fully WirelessThe used Android software for the EEG amplifier doesnotallowtosimultaneouslyuseBluetoothtoreceiveEEGdatafromtheamplifierandtotransmitaudiototheneck
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 8 of 10
speaker. More importantly, we removed the Bluetoothfunctionalityoftheneckspeakerasweremovedthebat-terytomakespacefortheamplifier.Therefore,ourpro-totype requiresa stereocinch-cinchcable to recordandplay back sound. In future designs, all communicationwith the nEEGlace should bewireless, i.e. the transmis-sionofallinformationtoandfromthenEEGlaceshouldbe based either on Bluetooth, WiFi or NFC (near-field-communication) under consideration of the amount ofdata that needs to be transmitted (both directions) andtheexpecteddrainonthebattery.Furthermore,recharg-ingshouldbeinductionbasedtoavoidaconnectorsocket(seesectionBattery).
Data Storage or Data TransmissionFormobileEEGsystems, there is thedesigndecisiononwheretostorethedata.Thesignalcaneitherbetransmit-ted toanextra recordingdevice (herea smartphone)orstored locallyon theEEGamplifier (e.g.onanSD-card).On-boardrecordinghastheadvantagethatnotelemetryis necessaryduring the acquisition. This leads to longerbatterylife,reducestheriskofdatalossduetotransmis-sionproblems(e.g.duetootherinterferingdevices),andiseasiertoimplement.Onthedownside,on-boardrecord-ingcanmakeitmoredifficulttocombinetheEEGsignalwithotherdatastreams(e.g.eventmarker).Forexampletoaddeventmarkers,amarkersignalwouldhavetoberecordedonthedeviceitselfrequiringpotentiallyfurtherhardwireconnections.However, the synchronized acquisition of multiple
data streams is essential for EEG recordings, especiallyforbeyondthelabrecordings.ForEEGexperimentsit isnecessarytorelatetheEEGsignaltosomestimulus(e.g.avisualorauditoryevent).Forthiswehavetoknowwhenthose stimuli occur relative to the EEG signal. FormostofourmobileEEGstudiesweusethelabstreaminglayerframework (LSL13) to integratedata streams fromdiffer-entdevices.Labstreaminglayerallowsthesynchronizedacquisitionofmultipledatastreamswithdifferent(evenirregular)samplingrates.Eachdatasamplereceivesatimestampasearlyaspossible.FuturedesignsshouldconsidertouseLSLfordatasynchronization.
BatteryForbeyond the lab recordingsonewould like tohaveabatterylifeofatleast8hours(nine-to-fiveworkday).ThebatteryoftheSmartingamplifier2allowsarecordingtimeof3–5hours.TheSmartingsleepamplifier14has15hoursofbatterylife,butitisconsiderablylargerandcannotbeintegratedintothecurrentneckpieceduetothesizeofthebatterypack.Itisconceivabletohaveseveralsmallerbatteriesintegratedintotheneckholder,e.g.ateachsideforbalancedweightdistribution (seeabove). Ideally, thebattery would be charged by induction, as this wouldincreasethelongevityofthedeviceforathomeuse.
EEG SensorsThecEEGrid (Debeneretal.,2015) isa researchproductto facilitate ear-EEG research. However, for long termrecordings inpublic it isnot ideal in termsofusability,
re-usabilityandusercomfort.Currently,itisnecessarytoapplyasmallamountofconductivegeltoeachelectrode,what requires some training, andmakes self-applicationdifficult.Furthermore,cEEGridscomeinonesizeonlyanddonotfitallearsequallywell,leadingpotentiallytodis-comfortifusedforlongerperiods.The optimal ear-EEG sensors would be comfortable
towear for longperiodsof timeandshouldnot leadtoskinirritations,evenafterrepeatedusage.Theyshouldbelight-weight,robustandself-applicable.Itshouldbeeasytopositionandshouldallowforeasyre-positioningatthesamelocation.
Choice of AmplifierWeuseaSmartingamplifier2hereaswehaveextensiveexperience with it and because it integrates into ourexperimental ecosystem (Blum et al., 2017). The formfactorof thedevice isnot ideal for the integration intotheneckpiece.Inprincipleeveryothermobileamplifierwould work. However, for applications using ear-EEG itneedstobeassuredthattheamplifierissufficientlysensi-tive to the small signal amplitudes that aremeasurablewithelectrodesaroundtheears.Dependingontheoriginof themeasured signal theamplitudes recordedaroundtheearcanbe10timessmallercomparedtorecordingswithanEEGcap(Meiseretal.,2020).Theneuralsignalsofinterestaretinycomparedtootherinfluences(e.g.duetomusclemovements).Thenoisefloorofmanyresearchgrade amplifiers lies at around 1μV. The acquisition ofhigh-qualityEEGdatarequireshigh-qualityEEGhardware.Goodamplifiersarecharacterisedbylowinputnoise(i.e.≥1μV),high-commonmoderejection,hightemporalpre-cision (this includes reliable and continuous sampling,aswellasthepossibilitytointegratetriggerinformationwith millisecond precision), the possibility to get theraw,unfilteredorotherwisepre-processeddataandhighqualityelectrodes.Forourpurposewealsoneedmobilestoragecapabilitiesorsmartphonecompatibility.
ConclusionTheherepresentedprototypeofanear-EEGsystemthatcanbewornaroundtheneck,showsthegeneralfeasibil-ity and practicality of such a system for beyond-the-labEEGrecordings.Dedicatedfuturedevelopmentisneededto turn this prototype into a commercial product or tomakeitafullyopenhardwareprojecttomakeitavailabletotheresearchcommunity.
Notes 1 https://www.jbl.com/bluetooth-speakers/JBL+
SOUNDGEAR+BTA.html. 2 https://mbraintrain.com/smarting. 3 https://ceegrid.com. 4 https://mbraintrain.com/wp-content/uploads/
2016/08/SMARTING-User-Manual.pdf. 5 https://github.com/mgbleichner/nEEGlace. 6 https://openbci.com/. 7 http://ceegrid.com/home/how-to-connect/. 8 https://www.neurobs.com. 9 http://openvibe.inria.fr.
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 9 of 10
10 https://osdoc.cogsci.nl. 11 https://www.psychopy.org. 12 http://psychtoolbox.org. 13 https://labstreaminglayer.readthedocs.io/info/intro.
html. 14 https://mbraintrain.com/Smarting-sleep.
AcknowledgementsWethankProf.StefanDebener,BjörnHoltzeundDanielHölle for their ideason thedesignof thenEEGlace.WethankDanielHölle forhishelp in acquiring thequalitycheckdata.
Funding InformationThis work was funded by the Deutsche Forschungsge-meinschaft (DFG, German Research Foundation) undertheEmmy-Noetherprogram–BL1591/1-1–Project ID411333557.
Competing InterestsTheauthorshavenocompetingintereststodeclare.
Author ContributionsMartinG.BleichnerconceivedanddesignedthenEEGlacetogetherwithReinerEmkes.ReinerEmkesassembledthehardware.MartinG.BleichnerevaluatedthenEEGlaceandwrotethemanuscript.
ReferencesBitzer, J., Kisser, S., & Holube, I. (2016).Privacy-Aware
AcousticAssessmentsofEverydayLife.AES: Journal of the Audio Engineering Society. DOI: https://doi.org/10.17743/jaes.2016.0020
Bleichner, M. G., & Debener, S. (2017, 4). Concealed,unobtrusiveear-centeredEEGacquisition:cEEGridsfor transparent EEG.Frontiers in Human Neurosci-ence, 11(163), 163. DOI: https://doi.org/10.3389/fnhum.2017.00163
Bleichner, M. G., Mirkovic, B., & Debener, S. (2016).Identifyingauditoryattentionwithear-EEG:CEEGridversushigh-densitycap-EEGcomparison.Journal of Neural Engineering,13 (6),1–13.DOI:https://doi.org/10.1088/1741-2560/13/6/066004
Blum, S., Debener, S., Emkes, R., Volkening, N., Fudickar, S., & Bleichner, M. G. (2017, 11).EEG Recording and Online Signal Processing onAndroid: A Multiapp Framework for Brain-Com-puter Interfaces on Smartphone.BioMed Research International, 2017, 1–12. DOI: https://doi.org/10.1155/2017/3072870
Debener, S., Emkes, R., De Vos, M., & Bleichner, M. G.(2015).UnobtrusiveambulatoryEEGusingasmart-phone and flexible printed electrodes around theear. Scientific Reports, 5, 16743. DOI: https://doi.org/10.1038/srep16743
Debener, S., Minow, F., Emkes, R., Gandras, K., & De Vos, M.(2012).Howabouttakingalow-cost,small,andwireless EEG for awalk?Psychophysiology,49 (11), 1617–1621. DOI: https://doi.org/10.1111/j.1469-8986.2012.01471.x
Denk, F., Grzybowski, M., Ernst, S. M., Kollmeier, B., Debener, S., & Bleichner, M. G. (2018). Event-Related PotentialsMeasured From In and Aroundthe Ear Electrodes Integrated in a Live HearingDeviceforMonitoringSoundPerception.Trends in Hearing, 22, 233121651878821. DOI: https://doi.org/10.1177/2331216518788219
Gramann, K.(2011).Cognitioninaction:Imagingbrain/body dynamics in mobile humans. Reviews in the Neurosciences, 22. DOI: https://doi.org/10.1515/RNS.2011.047
Gramann, K.(2014).Imagingnaturalcognitioninaction.International Journal of Psychophysiology, 91.DOI: https://doi.org/10.1016/j.ijpsycho.2013.09.003
Gramann, K., Jung, T.-P., Ferris, D. P., Lin, C.-T., & Makeig, S. (2014).Toward a new cognitive neurosci-ence: Modeling natural brain dynamics (Vol.8).DOI:https://doi.org/10.3389/fnhum.2014.00444
Hölle, D., Meekes, J., & Bleichner, M. G. (2020, 9).Mobileear-EEGtostudyauditoryattentioninevery-daylife.bioRxiv,2020.09.09.287490.DOI:https://doi.org/10.1101/2020.09.09.287490
Kidmose, P., Looney, D., Ungstrup, M., Rank, M. L., & Mandic, D. P.(2013,10).AStudyofEvokedPoten-tialsFromEar-EEG.IEEE Transactions on Biomedical Engineering,60 (10),2824–2830.DOI:https://doi.org/10.1109/TBME.2013.2264956
Kowalk, U., Kissner, S., von Gablenz, P., Holube, I., & Bitzer, J.(2018).Animprovedprivacy-awaresystemfor objective and subjective ecological momentaryassessment.Proceedings of the International Sympo-sium on Auditory and Audiological Research.
Ladouce, S., Donaldson, D. I., Dudchenko, P. A., & Ietswaart, M. (2017, 1). Understanding minds in real-world environments: Toward a mobile cogni-tion approach (Vol.10).Frontiers.DOI:https://doi.org/10.3389/fnhum.2016.00694
Looney, D., Park, C., Kidmose, P., Rank, M. L., Ungstrup, M., Rosenkranz, K., & Mandic, D. P. (2011). Anin-the-ear platform for recording electroencepha-logram. In Proceedings of the annual international conference of the ieee engineering in medicine and biology society, embs (pp.6882–6885).DOI:https://doi.org/10.1109/IEMBS.2011.6091733
Meiser, A., Tadel, F., Debener, S., & Bleichner, M. G.(2020, 8). The sensitivity of ear-EEG: evaluatingthesource-sensorrelationshipusingforwardmod-eling.Brain Topography,1,1–15.DOI:https://doi.org/10.1007/s10548-020-00793-2
Mirkovic, B., Bleichner, M. G. M., De Vos, M., & Debener, S. (2016). Target speakerdetectionwithconcealedEEGaroundtheear.Frontiers in Neurosci-ence,10 (JUL).DOI:https://doi.org/10.3389/fnins.2016.00349
Nann, M., Cohen, L. G., Deecke, L., & Soekadar, S. R.(2019).Tojumpornottojump–TheBereitschaft-spotential required to jump into192-meter abyss.Scientific Reports.DOI:https://doi.org/10.1038/s41598-018-38447-w
Bleichner and Emkes: Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab
Art. 5, page 10 of 10
Nikulin, V. V., Kegeles, J., & Curio, G.(2010).Miniaturizedelectroencephalographic scalp electrode for opti-mal wearing comfort. Clinical Neurophysio logy,121 (7),1007–1014.DOI:https://doi.org/10.1016/j.clinph.2010.02.008
Oppezzo, M., & Schwartz, D. L. (2014). Give yourideas some legs: the positive effect of walkingon creative thinking. Journal of experimental psychology. Learning, memory, and cognition, 40
(4),1142–52.DOI:https://doi.org/10.1037/a0036577
O’Regan, S., & Marnane, W. (2013).Multimodaldetec-tion of head-movement artefacts in EEG. Journal of Neuroscience Methods. DOI: https://doi.org/10.1016/j.jneumeth.2013.04.017
Radüntz, T.(2018,2).SignalQualityEvaluationofEmerg-ingEEGDevices.Frontiers in Physiology,9 (FEB),98.DOI:https://doi.org/10.3389/fphys.2018.00098
How to cite this article: Bleichner, MG and Emkes, R. 2020. Building an Ear-EEG System by Hacking a Commercial Neck Speaker and a Commercial EEG Amplifier to Record Brain Activity Beyond the Lab. Journal of Open Hardware, 4(1): 5, pp. 1–10. DOI: https://doi.org/10.5334/joh.25
Published: 16 October 2020
Copyright: © 2020 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.
OPEN ACCESS Journal of Open Hardware is a peer-reviewed open access journal published by Ubiquity Press.