186246056 brain-computer-interfacing
-
Upload
homework-ping -
Category
Education
-
view
311 -
download
0
Transcript of 186246056 brain-computer-interfacing
Brain Computer Interfacing Seminar Report 2011
Get Homework/Assignment Done Homeworkping.comHomework Help https://www.homeworkping.com/
Research Paper helphttps://www.homeworkping.com/
Online Tutoringhttps://www.homeworkping.com/
click here for freelancing tutoring sites
SEMINAR REPORT ON
BRAIN COMPUTER INTERFACING
BY
ASHISH VISWANATH PRAKASH
DEPARTMENT OF ELECTRONICS AND COMMUNICATION
1Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
COLLEGE OF ENGINEERING
CUSAT
JULY 2011
ABSTRACT
A brain-machine interface is a communication system that does not depend on the brains normal
output pathways of peripheral nerves and muscles. It is a new communication link between a
functioning human brain and the outside world. These are electronic interfaces with the brain,
which has the ability to send and receive signals from the brain. BMI uses brain activity to
command, control, actuate and communicate with the world directly through brain integration
with peripheral devices and systems. The signals from the brain are taken to the computer via the
implants for data entry without any direct brain intervention. BMI transforms mental decisions
and/or reactions into control signals by analyzing the bioelectrical brain activity.
While linking the brain directly with machines was once considered science fiction, advances
over the past few years have made it increasingly viable. It is an area of intense research with
almost limitless possibilities. The human brain is the most complex physical system we know of,
and we would have to understand its operation in great detail to build such a device. An
immediate goal of brain-machine interface study is to provide a way for people with damaged
sensory/motor functions to use their brain to control artificial devices and restore lost capabilities.
By combining the latest developments in computer technology and hi-tech engineering, paralyzed
persons will be able to control a motorized wheel chair, computer painter, or robotic arm by
thought alone. In this era where drastic diseases are getting common it is a boon if we can
2Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
develop it to its full potential. Recent technical and theoretical advances, have demonstrated the
ultimate feasibility of this concept for a wide range of space-based applications. Besides the
clinical purposes such an interface would find immediate applications in various technology
products also.
Keywords: Brain,Neuro-electronics,Electro encephalography, interfacing,
INTRODUCTION
Picture a time when humans see in the UV and IR portions of the electromagnetic spectrum, or
hear speech on the noisy flight deck of an aircraft carrier; or when soldiers communicate by
thought alone. Imagine a time when the human brain has its own wireless modem so that instead
of acting on thoughts, war fighters have thoughts that act. Imagine that one day we will be able to
download vast amounts of knowledge directly to our brain! So as to cut the lengthy processes of
learning everything from scratch. Instead of paying to go to university we could pay to get a
"knowledge implant" and perhaps be able to obtain many lifetimes worth of knowledge and
expertise in various fields at a young age In the near future, most devices would be
remote/logically controlled. Researchers are close to breakthroughs in neural interfaces, meaning
we could soon mesh our minds with machines. This technology has the capability to impact our
lives in ways that have been previously thought possible in only sci-fi movies.
There is legitimate scientific interest in the possibility of connectingbrains and computers—from
producing robotic limbs controlleddirectly by brain activity to altering memory and mood with
implantedelectrodes to the far-out prospect of becoming immortal by “uploading”our minds into
machines. This area of inquiry has seen remarkableadvances in recent years, many of them aimed
at helping the severely disabledto replace lost functions. Yet public understanding of this research
3Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
isshaped by sensationalistic and misleading coverage in the press; it is coloredby decades of
fantastical science fiction portrayals; and it is distortedby the utopian hopes of a small but vocal
band of enthusiasts who desire to eliminate the boundaries between brains and machines as part
of a larger “transhumanist” project
Neuroelectronics
Neuroelectronics, sometimes referred to as neurotechnology, is the discipline that deals with the
interface between the human nervous system and electronic devices. It is a highly complex and
interdisciplinary field with contributions from computer science, cognitive science, neurosurgery
and biomedical engineering. Neuroelectronics has roughly three related branches: (1)
neuroimaging, (2) brain-computer interfaces (BCIs), and (3) electrical neural stimulation. The
discipline exists for more than half a century. However, in the last decade significant advances
have been made, particularly in neuroimaging, which revolutionized the field by allowing
researchers to directly monitor brain activity during experiments. And it is predicted that
neuroelectronics, particularly neuroimaging and brain-computer interfacing, will be employed
much more in the future.
BCIs, sometimes called brain-machine interfaces (BMIs), are an emerging neurotechnology that
translates brain activity into command signals for external devices. Research on BCIs began in
the 1970s at the University of California Los Angeles (UCLA). Researchers at UCLA also coined
the term brain-computer interface. A BCI establishes a direct communication pathway between
the brain and the device to be controlled. They are mainly being developed for medical reasons,
because there is a societal demand for technologies which help to restore functions of humans
with central nervous system (CNS). Patients for whom a BCI would be useful usually have
disabilities in motor function or communication. This could be (partly) restored by using a BCI to
steer a motorized wheelchair, prosthesis, or by selecting letters on a computer screen with a
cursor. Invasive or non-invasive electrodes are used to detect brain activity, which is
subsequently translated by a signal processing unit into command signals for the external device.
The most common BCI responds to specific patterns detected in spatiotemporal EEGs measured
non-invasively from the scalp. Spatiotemporal EEGs can be controlled by imagining specific
movements (Gasson & Warwick, 2007). So, merely by imagining movements one can steer a
wheelchair, prosthesis or a cursor on a computer screen.
4Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
There are roughly three branches in neuroelectronics. Each branch uses different devices to
interface with the brain, and each of these devices has different features. The first branch,
neuroimaging, uses techniques such as fMRI, PET, MEG or EEG, amongst others, to extract
information from the brain to diagnose disorders or to study the brain. The second branch, BCIs,
uses invasive or non-invasive electrodes to extract information from the brain, not for diagnostic
or research purposes, but to control external devices such as wheelchairs, computers or airplanes.
And the third branch, electrical neural stimulation, uses invasive electrodes to send electrical
signals to specific parts of the brain. The only defining feature these three branches have in
common is that they all interface electrical devices with the brain, either to extract information
from the brain or to send electrical signals to the brain.
HISTORY
The brain has been clearly understood to be the seat of the mind for less than four centuries. A
number of anatomists, philosophers, and physicianshad, since the days of the ancient Greeks,
concluded that the soul was resident in the head. In 1960, at the height of interest in cybernetics,
the word cyborg—short for “cybernetic organism”—was coined by researcher Manfred E.Clynes
in a paper he co-wrote for the journal Astronautics Around the same time, Jack E. Steele,a
polymath doctor-engineer-neuroanatomist serving in the U.S. Air Force,
coined the word bionics for the use of principles derived from living systems to solve engineering
and design problems. But to the most ambitious and most radical advocates of merging brains and
machines, such advances are mere child’s play. These so-called transhumanists long for an age
when human beings will leave the miser ies and limits of the body behind, and achieve new
ecstasies of freedom
and control: We will send feelings and conscious thoughts directly frombrain to brain; we will
delete unwanted memories at will; we will master
complex subjects by “downloading” them directly into our minds; we will“jack in” to virtual
realities; and eventually, we will be able to “upload”
our personalities into computers or robots, where the self could live on indefinitely.
5Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
The history of Brain-Computer-Interfaces (BCI) starts with Hans Berger'sdiscovery of the
electrical activity of human brain and the development ofelectroencephalograpy (EEG).Berger
studied medicine at the University of Jena and received his doctorate in
1897. He became a professor in 1906 and the director of the University'spsychiatry and neurology
clinic in 1919. In 1924 Berger was the first one whorecorded an EEG from a human brain. By
analyzing EEGs Berger was able toidentify different waves or rhythms which are present in a
brain, as the Alpha Wave (8 – 12 Hz),also known as Berger's Wave.
In the 1980s, Apostolos Georgopoulos at Johns Hopkins University found a mathematical
relationship between the electrical responses of single motor-cortex neurons in rhesus macaque
monkeys and the direction that monkeys moved their arms (based on a cosine function). He also
found that dispersed groups of neurons in different areas of the brain collectively controlled
motor commands but was only able to record the firings of neurons in one area at a time because
of technical limitations imposed by his equipment
Prominent research successes
In 1999, researchers led by Garrett Stanley at Harvard University decoded neuronal firings to
reproduce images seen by cats. The team used an array of electrodes embedded in the thalamus
(which integrates all of the brain’s sensory input) of sharp-eyed cats. Researchers targeted 177
brain cells in the thalamus lateral geniculate nucleus area, which decodes signals from the retina.
The cats were shown eight short movies, and their neuron firings were recorded. Using
mathematical filters, the researchers decoded the signals to generate movies of what the cats saw
and were able to reconstruct recognisable scenes and moving objects.
Miguel Nicolelis has been a prominent proponent of using multiple electrodes spread over a
greater area of the brain to obtain neuronal signals to drive a BCI. Such neural ensembles are said
6Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
to reduce the variability in output produced by single electrodes, which could make it difficult to
operate a BCI
By 2000, the group succeeded in building a BCI that reproduced owl monkey movements while
the monkey operated a joystick or reached for food.The BCI operated in real time and could also
control a separate robot remotely over Internet protocol. But the monkeys could not see the arm
moving and did not receive any feedback, a so-called open-loop BCI Other labs that develop
BCIs and algorithms that decode neuron signals include John Donoghue from Brown University,
Andrew Schwartz from the University of Pittsburgh and Richard Andersen from Caltech
Donoghue's group reported training rhesus monkeys to use a BCI to track visual targets on a
computer screen with or without assistance of a joystick (closed-loop BCI).Schwartz's group
created a BCI for three-dimensional tracking in virtual reality and also reproduced BCI control in
a robotic arm
Cell-culture BCIs
Research into techniques for stimulating and recording from individual neurons grown on
semiconductor chips is sometimes referred to as neuroelectronics or neurochips.
World first: Neurochip developed by Caltech researchers Jerome Pine and Michael Maher
Development of the first working neurochip was claimed by a Caltech team led by Jerome Pine
and Michael Maher in 1997. The Caltech chip had room for 16 neurons In 2003, a team led by
Theodore Berger at the University of Southern California started work on a neurochip designed to
function as an artificial or prosthetic hippocampus. The neurochip was designed to function in rat
brains and is intended as a prototype for the eventual development of higher-brain prosthesis
Thomas DeMarse at the University of Florida used a culture of 25,000 neurons taken from a rat's
brain to fly a F-22 fighter jet aircraft simulator.After collection, the cortical neurons were
cultured in a petri dish and rapidly begin to reconnect themselves to form a living neural network.
The cells were arranged over a grid of 60 electrodes and trained to control the pitch and yaw
functions of the simulator
READING THE BRAIN – NEUROIMAGING IN BCIS
Neuroimaging definition
“Neuroimaging includes the use of various techniques to either directly or indirectly image
thestructure, function, or pharmacology of the brain. It is a relatively new discipline within
medicineand neuroscience.” [WIKI_NI]
7Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Direct Neural Contact
This is the most accurate method of recording potentials occuring in the brain as it has
directcontact to every neuron in the brain, e.g. by using nanorobots. Needless to say that this
method ishighly invasive and impracticable with respect to our current technology. However,
with ongoingadvances in nanotechnology this method might become reality.
Electroencephalography (EEG)
This procedure is the first non-invasive neuroimaging technique discovered. It measures
theelectrical activity of the brain. Due to its ease of use, cost and high temporal resolution this
methodis the most widely used one in BCIs nowadays. However, in practice EEGs are highly
susceptible tonoise and thus require a significant amount of user training in order to be operable
in a BCI.Luckily, recent research at the Fraunhofer Society [WIKI_BCI] has shown that this
problem can beovercome by using neural networks to shift the learning overhead from the human
to the computer.
Magnetoencephalography (MEG)
Though similar to the EEG in that it is a non-evasive technology the MEG is a much newer
andmore accurate technology. Instead of measuring the electrical activity in the brain this
technologyrecords magnetic fields produced by it. The main drawbacks of this technology are its
higherequirements in equipment.
“Using MEG requires a room filled with super-conducting magnets and giant super-cooling
heliumtanks surrounded by shielded walls.”
Functional Magnetic Resonance Imaging (fMRI)
This technique measures the haemodynamic response (blood flow and blood oxygenation) related
toneural activity in the brain by the use of MRI (Magnetic Resonance Imaging formerly known
asMagnetic Resonance Tomography MRT). The fact that there is a correlation between neural
activityand the brain’s haemodynamics makes the fMRI a neuroimaging tool. In contrast to the
MRI whichstudies the brain’s structure this method studies the brain’s function. As this method
requires MRItechnology it needs very special equipment and thus is quite costly.
BLOCK DIAGRAM
8Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
9Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
BLOCK DESCRIPTION
The BMI consists of several components: 1.the implant device, or chronic multi-electrode array,
2.the signal recording and processing section, 3.an external device the subject uses to produce and
control motion and 4.a feedback section to the subject. The first component is an implanted array
of microelectrodes into the frontal and parietal lobes—areas of the brain involved in producing
multiple output commands to control complex muscle movements. This device record action
potentials of individual neurons and then represent the neural signal using a rate code .The second
component consists of spike detection algorithms, neural encoding and decoding systems, data
acquisition and real time processing systems etc .A high performance DSP architecture is used for
this purpose. The external device that the subject uses may be a robotic arm, a wheel chair etc.
depending upon the application. Feedback is an important factor in BCI‘s. In the BCI‘s based on
the operant conditioning approach, feedback training is essential for the user to acquire the
control of his or her EEG response. However, feedback can speed up the learning process and
improve performance.
the three types of BCI. These types are decided on the basis of the technique used for the
interface. Each of these techniques has some advantages as well as some disadvantages.
Invasive BCI:
Invasive BCI are directly implanted into the grey matter of the brain during neurosurgery. They
produce the highest quality signals of BCI devices . Invasive BCIs has targeted repairing
damaged sight and providing new functionality to paralyzed people. But these BCIs are prone to
building up of scar-tissue which causes the signal to become weaker and even lost as body reacts
to a foreign object in the brain
Partially Invasive BCI:
Partially invasive BCI devices are implanted inside the skull but rest outside the brain rather than
amidst the grey matter. They produce better resolution signals than non-invasive BCIs where the
bone tissue of the cranium deflects and deforms signals and have a lower risk of forming scar-
tissue in the brain than fully-invasive BCIs.
Electrocorticography(ECoG) uses the same technology as non-invasive electroencephalography,
but the electrodes are embedded in a thin plastic pad that is placed above the cortex, beneath the
dura mater. ECoG technologies were first traled in humans in 2004 by Eric Leuthardt and Daniel
Moran from Washington University in St Louis. In a later trial, the researchers enabled a teenage
boy to play Space Invaders using his ECoG implant. This research indicates that it is difficult to
produce kinematic BCI devices with more than one dimension of control using ECoG.
10Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Non-Invasive BCI :
As well as invasive experiments, there have also been experiments in humans using non-invasive
neuroimaging technologies as interfaces. Signals recorded in this way have been used to power
muscle implants and restore partial movement in an experimental volunteer. Although they are
easy to wear, non-invasive implants produce poor signal resolution because the skull dampens
signals, dispersing and blurring the electromagnetic waves created by the neurons. Although the
waves can still be detected it is more difficult to determine the area of the brain that created them
or the actions of individual neurons
MAIN PRINCIPLE
Main principle behind this interface is the bioelectrical activity of nerves and muscles. It is now
well established that the human body, which is composed of living tissues, can be considered as a
power station generating multiple electrical signals with two internal sources, namely muscles
and nerves.
We know that brain is the most important part of human body. It controls all the emotions and
functions of the human body. The brain is composed of millions of neurons. These neurons work
together in complex logic and produce thought and signals that control our bodies. When the
neuron fires, or activates, there is a voltage change across the cell, (~100mv) which can be read
through a variety of devices. When we want to make a voluntary action, the command generates
from the frontal lobe. Signals are generated on the surface of the brain. These electric signals are
different in magnitude and frequency.
By monitoring and analyzing these signals we can understand the working of brain. When we
imagine ourselves doing something, small signals generate from different areas of the brain.
These signals are not large enough to travel down the spine and cause actual movement. These
small signals are, however, measurable. A neuron depolarizes to generate an impulse; this action
causes small changes in the electric field around the neuron. These changes are measured as 0 (no
impulse) or 1 (impulse generated) by the electrodes. We can control the brain functions by
artificially producing these signals and sending them to respective parts. This is through
stimulation of that part of the brain, which is responsible for a particular function using implanted
electrodes.
BMI COMPONENTS
11Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
A brain-machine interface (BMI) in its scientific interpretation is a combination of several
hardware and software components trying to enable its user to communicate with a computer by
intentionally altering his or her brain waves. The task of the hardware part is to record the
brainwaves– in the form of the EEG signal – of a human subject, and the software has to analyze
that data. In other words, the hardware consists of an EEG machine and a number of electrodes
scattered over the subject‘s skull. The EEG machine, which is connected to the electrodes via thin
wires, records the brain-electrical activity of the subject, yielding a multi-dimensional (analog or
digital) output. The values in each dimension (also called channel) represent the relative
differences in the voltage potential measured at two electrode sites.
The software system has to read, digitize (in the case of an analog EEG machine), and preprocess
the EEG data (separately for each channel), ―understand‖ the subject‘s intentions, and generate
appropriate output. To interpret the data, the stream of EEG values is cut into successive
segments, transformed into a standardized representation, and processed with the help of a
classifier. There are several different possibilities for the realization of a classifier; one approach
– involving the use of an artificial neural network (ANN) – has become the method of choice in
recent years.
. IMPLANT DEVICE
The EEG is recorded with electrodes, which are placed on the scalp. Electrodes are small plates,
which conduct electricity. They provide the electrical contact between the skin and the EEG
recording apparatus by transforming the ionic current on the skin to the electrical current in the
wires. To improve the stability of the signal, the outer layer of the skin called stratum corneum
should be at least partly removed under the electrode. Electrolyte gel is applied between the
electrode and the skin in order to provide good electrical contact.
12Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Figure 4.An array of microelectrodes
Usually small metal-plate electrodes are used in the EEG recording. Neural implants can be used
to regulate electric signals in the brain and restore it to equilibrium. The implants must be
monitored closely because there is a potential for almost anything when introducing foreign
signals into the brain.
There are a few major problems that must be addressed when developing neural implants. These
must be made out of biocompatible material or insulated with biocompatible material that the
body won‘t reject and isolate. They must be able to move inside the skull with the brain without
causing any damage to the brain. The implant must be chemically inert so that it doesn‘t interact
with the hostile environment inside the human body. All these factors must be addressed in the
case of neural implants; otherwise it will stop sending useful information after a short period of
time.
There are simple single wire electrodes with a number of different coatings to complex three-
dimensional arrays of electrodes, which are encased in insulating biomaterials. Implant rejection
and isolation is a problem that is being addressed by developing biocompatible materials to coat
or incase the implant.
One option among the biocompatible materials is Teflon coating that protects the implant from
the body. Another option is a cell resistant synthetic polymer like polyvinyl alcohol. To keep the
implant from moving in the brain it is necessary to have a flexible electrode that will move with
the brain inside the skull. This can make it difficult to implant the electrode. Dipping the micro
device in polyethylene glycol, which causes the device to become less flexible, can solve this
problem
problem. Once in contact with the tissue this coating quickly dissolves. This allows easy
implantation of a very flexible implant.
Three-dimensional arrays of electrodes are also under development. These devices are
constructed as two-dimensional sheet and then bent to form 3D array. These can be constructed
13Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
using a polymer substrate that is then fitted with metal leads. They are difficult to implement, but
give a much great range of stimulation or sensing than simple ones.
Figure 5. Block diagram of the neurotrophic electrodes for implantation in human patients
.
A microscopic glass cone contains a neurotrophic factor that induces neurites to grow into the
cone, where they contact one of several gold recording wires. Neurites that are induced to grow
into the glass cone make highly stable contacts with recording wires. Signal conditioning and
telemetric electronics are fully implanted under the skin of the scalp. An implanted transmitter
(TX) sends signals to an external receiver (RX), which is connected to a computer.
SIGNAL PROCESSING SECTION
Multichannel Acquisition Systems
Electrodes interface directly to the non-inverting opamp inputs on each channel. At this section
amplification, initial filtering of EEG signal and possible artifact removal takes place. Also A/D
conversion is made, i.e. the analog EEG signal is digitized. The voltage gain improves the signal-
to-noise ratio (SNR) by reducing the relevance of electrical noise incurred in later stages.
Processed signals are time-division multiplexed and sampled.
14Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
SPIKE DETECTION
Real time spike detection is an important requirement for developing brain machine interfaces.
Incorporating spike detection will allow the BMI to transmit only the action potential waveforms
and their respective arrival times instead of the sparse, raw signal in its entirety. This compression
reduces the transmitted data rate per channel, thus increasing the number of channels that may be
monitored simultaneously. Spike detection can further reduce the data rate if spike counts are
transmitted instead of spike waveforms. Spike detection will also be a necessary first step for any
future hardware implementation of an autonomous spike sorter. Figure 6 shows its
implementation using an application-specific integrated circuit (ASIC) with limited
computational resources. A low power implantable ASIC for detecting and transmitting neural
spikes will be an important building block for BMIs. A hardware realization of a spike detector in
a wireless BMI must operate in real-time, be fully autonomous, and function at realistic signal-to-
noise ratios (SNRs).
An implanted ASIC conditions signal from extra cellular neural electrodes, digitizes them, and
then detects AP spikes. The spike waveforms are transmitted across the skin to a BMI processor,
which sorts the spikes and then generates the command signals for the prosthesis.
Classical methods for removing eyeblink artifacts can be classified into rejection
methods and subtraction methods [4.6]:
Rejection methods consist of discarding contaminated EEG, based on eitherautomatic or
visual detection. Their success crucially depends on the quality of the
detection, and its use depends also on the specific application for which it is used.Thus, although
for epileptic applications, it can lead to an unacceptable loss of data,for others, like a Brain
Computer interface, its use can be adequate.
Subtraction methods are based on the assumption that the measured EEG is a
linearcombination of an original EEG and a signal caused by eye movement, called
EOG(electrooculogram). The EOG is a potential produced by movement of the eye oreyelid (Fig.
4-4). The original EEG is hence recovered by subtracting separatelyrecorded EOG from the
measured EEG, using appropriate weights (rejecting theinfluence of the EOG on particular EEG
channels).
Artifact rejection based on peak elimination.
As previous works have shown [4.3], the presence of artifacts in EEG signalsproduces a rapid
increase of energy in forehead locations Fp1 and Fp2. The method
15Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
developed here consists of the analysis of these two channels by small overlappingwindows, in
order to check if the energy of the signals surpasses an established blinkthreshold. In case it does
it, the samples coming from the corrupted signal are rejectedfrom all the EEG signals.Despite the
simplicity of this method, the results obtained have been satisfactory enough to consider it as an
initial option for a real-time Brain Computer Interface. It iseasy enough to be implemented on a
low complexity signal processing platform.Nevertheless, this method has the inconvenience of
rejecting some non-corrupted data inother scalp channels, as well as in the frontal channels.With
the purpose of improving the data preservation, we have developed a similarsystem based on a
quadratic Time Frequency Representation of the signal. This analysistakes advantage of the high
resolution of this technique in time and frequency, for
establishing, after an appropriate training, the differences between a corrupted EEGsignal and a
non-corrupted EEG signal, in order to be able of distinguish and reject theeyeblink artifact. This
technique, which is based on energy distributions, is an alternativeto the classical artifact
rejection method presented here, and it should be considered in afuture work.
Blinking artifact recognition using artificial neural network.
The method proposed by Bogacz and colleagues, used a neural based approach tofind artifacts in
EEG signals [4.4]. The input to the neural network was not a raw sampledsignal, but different
coefficients computed for a window of one second of the signal,
expressing some characteristic properties of blinking artifacts. 41 coefficients weredesigned.
Some of them were designed by the authors and were based on their knowledgeabout the artifact
recognition, and a total of 14 were chosen by terms of sensitivity andcorrelation. A large training
set including coefficients for over 27000 windows was used,containing different kinds of blinking
artifacts, pathological and proper waves, andartifacts caused by other sources (e.g. jaw, muscle).
Afterwards, three classificationalgorithms were tested and compared: k-neighbors, RBF networks
and back propagationnetworks. The lowest classification error (1.40%) was obtained for the back
propagationnetwork, with a classification time of the test set (6227 windows) of 2 seconds
[4.4].This method achieves high classification accuracy thanks to two factors:
- Large training set containing different kinds of EEG waves.
- The coefficients delivered to the network’s inputs, express the characteristic
features of artifacts, since they encode large amount of domain expert’s
knowledge.
Unfortunately, the first factor can be problematic in the use of a Brain computer
Interface.
16Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
4. 4. Artifact rejection based in bandpass FIR filters.
The method proposed by Gupta and colleagues used a fixed bandpass FIR filter,followed by a
subject specific eyeblink threshold, in order to remove the eyeblink andeyeball movement
artifacts. This technique, whose block diagram can be seen in Fig. 4-5,consist of [4.1]:
1. Pass the raw EEG samples obtained from analog-to-digital converter through a
digital bandpass filter (BPF) to remove slow baseline drift.
2. Determine the blink threshold (Vt) for specific subject in brief training session.
3. Compare the absolute sample value with Vt.
4. If the value is exceeded then remove N samples from the vicinity of zero crossing
(N/2 on either side of treshold crossing).
5. Shift the following N samples to fill up the gap created by blink removal. These
gaps will, otherwise, grossly distort the spectrum.
Figure 4-5. Scheme of the proposed system.
The experiments carried out through this scheme, for Fp1 and Fp2 electrodes
location, provided interesting results in eyeblink artifact rejection. This method presents
the advantage of working even under baseline drift artifacts conditions, and also is easy
enough to be implemented on a low-cost digital signal processor, on a real – time system.
Nevertheless, it fails if the blink rate is unnaturally high, and the training session for each
individual is quite long: 30 sec (6 eyeblinks on a average for a normal subject).
SIGNAL ANALYSIS
Feature extraction and classification of EEG are dealt in this section. In this stage, certain features
are extracted from the preprocessed and digitized EEG signal. In the simplest form a certain
frequency range is selected and the amplitude relative to some reference level measured
Typically the features are frequency content of the EEG signal) can be calculated using, for
example, Fast Fourier Transform (FFT function). No matter what features are used, the goal is to
17Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
form distinct set of features for each mental task. If the feature sets representing mental tasks
overlap each other too much, it is very difficult to classify mental tasks, no matter how good a
classifier is used. On the other hand, if the feature sets are distinct enough, any classifier can
classify them. The features extracted in the previous stage are the input for the classifier.
The classifier can be anything from a simple linear model to a complex nonlinear neural network
that can be trained to recognize different mental tasks. Nowadays real time processing is used
widely. Real-time applications provide an action or an answer to an external event in a timely and
predictable manner. So by using this type of system we can get output nearly at the same time it
receives input. Telemetry is handled by a wearable computer. The host station accepts the data
via either a wireless access point or its own dedicated radio card.
EEG Signal Pre - Processing.
One of the main problems in the automated EEG analysis is the detection of thedifferent kinds of
interference waveforms (artifacts) added to the EEG signal during therecording sessions. These
interference waveforms, the artifacts, are any recorded electrical potentials not originated in
brain. There are four main sources of artifacts emission:
1. EEG equipment.
2. Electrical interference external to the subject and recording system.
3. The leads and the electrodes.
4. The subject her/himself: normal electrical activity from the heart, eyeblinking, eyes movement,
and muscles in general.
In case of visual inspections, the artifacts can be quite easily detected by EEGexperts. However,
during the automated analysis these signal patterns often cause seriousmisclassifications thus
reducing the clinical usability of the automated analyzing systems.Recognition and elimination of
the artifacts in real – time EEG recordings is a complextask, but essential to the development of
practical systems.Previous works have shown that the most severe of the artifacts are due to
eyeblinks and eyeball movements. A movement of the eyeball and the eyelids causes achange in
the potential field because of the existing potential difference of about 100mVbetween the cornea
and the retina [4.1]. This change affects mainly the signals from themost frontal electrodes (Fp1
and Fp2 and also other frontal electrodes: F3, F4, F7 andF8), and induces in them many high and
low frequencies, depending upon its durationand amplitude
18Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
EEG SIGNAL CLASSIFICATION.
Oscillatory states are the most remarkable features of EEG activity, because they
reflect not only the synchronization of massive numbers of neurons but also a temporally
ordered rhythmicity of activation [5.6]. Different oscillatory patterns may be indicative of
different information processing states, and it has been proposed that the oscillatory
patterns play an active role in these states [5.6], [5.7]. According to this view, the
rhythmic synchronization during oscillatory states can serve to enhance perception,
learning, and the transmission of neuronal signals between different regions of the brain
Classifier Design:
The design of the classifier is heavily based on Christin Schäfer's design used for the Dataset III
of the BCI Competition II. Instead of using a Gaussian multivariate Bayesian classifier, here we
use a neural net to obtain the classification for each time instant t. Those outputs are then
integrated in time using a weighted sum. The idea is simple: outputs with low confusion should
have higher weights
EXTERNAL DEVICE
The classifier‘s output is the input for the device control. The device control simply transforms
the classification to a particular action. The action can be, e.g., an up or down movement of a
cursor on the feedback screen or a selection of a letter in a writing application. However, if the
classification was ―nothing‖ or ―reject‖, no action is performed, although the user may be
informed about the rejection. It is the device that subject produce and control motion. Examples
are robotic arm, thought controlled wheel chair etc
FEEDBACK
Real-time feedback can dramatically improve the performance of a brain–machine interface.
Feedback is needed for learning and for control. Real-time feedback can dramatically improve the
performance of a brain–machine interface. In the brain, feedback normally allows for two
corrective mechanisms. One is the ‗online‘ control and correction of errors during the execution
of a movement. The other is learning: the gradual adaptation of motor commands, which takes
place after the execution of one or more movements.
In the BMIs based on the operant conditioning approach, feedback training is essential for the
user to acquire the control of his or her EEG response. The BMIs based on the pattern recognition
approach and using mental tasks do not definitely require feedback training. However, feedback
can speed up the learning process and improve performance. Cursor control has been the most
popular type of feedback in BMIs. Feedback can have many different effects, some of them
19Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
beneficial and some harmful. Feedback used in BMIs has similarities with biofeedback,
especially EEG biofeedback.
20Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
BCI APPROACHES
Neuropsychological signals used in BCI applications.
Interfaces based on brain signals require on-line detection of mental states fromspontaneous
activity: different cortical areas are activated while thinking different things(i.e. a mathematical
computation, an imagined arm movement, a music composition,etc...). The information of these
"mental states" can be recorded with different methods.Neuropsychological signals can be
generated by one or more of the followingthree: implanted methods, evoked potentials (also
known as event related potentials), andoperant conditioning. Both evoked potential and operant
conditioning methods arenormally externally-based BCIs as the electrodes are located on the
scalp. Table 3.1describes the different signals in common use. It may be noted that some of the
describedsignals fit into multiple categories. As an example, single neural recordings may
useoperant conditioning in order to train neurons for control or may accept the naturaloccurring
signals for control. Where this occurs, the signal is described under thecategory that best
distinguishes it.
21Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
22Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
P300-Interface
ERS/ERD-Interface
The event related synchronization / desynchronization is a signal type that can be measured
whenthe subject imagines a hand or foot movement. ERS / ERD was researched by a team at
thetechnical university of Graz in Austria [CA_EEG]. They also developed an interface which
allows ovement of a cursor in a two dimensional space by combining hand and foot movement.
After wo training sessions three of the five test persons achieved a success rate between 89 and
100percent, the two other persons only 51and 60 percent.This could have the cause in different
imaginations of hand movement each person has.After a 62 training sessions with 160 trials a 25-
year old paraplegic patient could move the cursorpractically error-free. Regarding his immense
handicap the accomplishment of 0.95 letters perminute is definitely respectable.Another
23Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
experiment was done at the technical university Graz. They combined the ERS/ERDInterfacewith
a FES (functional electrical stimulation) which is the stimulation of muscles using
surface electrodes. A patient who had lost his ability to grasp with his hand is now no longer
Ipeded.
24Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
BCI RESEARCH: EXISTING SYSTEMS.
Different research groups work on communication channels between the brain and
the computer. The leading groups are presented in alphabetical order in Table 3. 2. These
experimental interfaces include the hardware used in the BCI, the underlying BCI
backend software, and the user application. In assessing current systems, several factors
must be considered, including five mentioned by Ben Schneiderman [3.19]:
1. What is the time to learn the system?
2. What is the speed of performance?
3. How many and what kinds of errors do users make?
4. How well do users maintain their knowledge after an hour, a day, or a week?
What is their retention?
Berlin Brain-Computer-Interface (BBCI)
The Berlin Brain-Computer-Interface is a joint venture of several german research organisations.
Members are:
– The Institute Computer Architecture and Software Technology of the Fraunhofer Society
– The research group Intelligent Data Analysis (IDA)
– The Neurophysics Research Group
– The department of Neurology at the Campus Benjamin Franklin of the Charité University
Medicine
– The Technical University Berlin
The BBCI project is sponsored by the Ministry for Education and Research of the German
government.
The goal of the project is the development of an EEG based BCI system. The applications of this
system are on the one hand computer supported workplaces, to control a cursor via brain waves
and
on the other hand tools for paralyzed or paraplegic people.
The BBCI project aims to shift the main learning effort to the computer. Therefore robust
artificial
learning and signal processing algorithms need to be developed to classify and interpret the brain
waves correctly. [BBCI]
BCI Classification Competitions
The first Brain-Computer-Interface Competition took place at the Laboratory for Intelligent
Imaging and Neural Computing of the Columbia University in 2002.
25Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
It was initalized to „forster development of machine learning techniques and evaluate different
algorithms for BCI“.
The competition focused on classification and signal processing algorithms. Data sets for several
different BCI task were provided to be analyzed by the participants.
The rules were pretty simple:
1. All data sets may only be evaluated single-trial
2. The statistics/metrics outlined in the description of each dataset should be reported
3. By using the data sets the participant agrees to refer their origin
4. Cheating is uncool!
[BCICC_2002]
The competition was a great success. Therefore it was repeated at the University of Graz, Austria
in
2003 and at the The Institute Computer Architecture and Software Technology of the Fraunhofer
Society in 2005. [BBCI]
Brain
1) The Brain Response Interface (Smith-Kettlewell Institute of Visual
Sciences in San Francisco).
Sutter's Brain Response Interface (BRI) [3.8] uses visually evoked potentials (VEP's)produced in
response to brief visual stimuli. These EP's are then used to give a discretecommand to pick a
certain part of a computer screen. Word processing output approaches10-12 words/min. and
accuracy approaches 90% with the use of epidural electrodes. This the only system mentioned
that uses implanted electrodes to obtain a stronger, less
contaminated signal.
A BRI user watches a computer screen with a grid of 64 symbols (some of whichlead to other
pages of symbols) and concentrates a given symbol. A specific subgroup ofthese symbols
undergoes a equiluminant red/green fine check or plain color patternalteration in a simultaneous
stimulator scheme at the monitor vertical refresh rate (40-70frames/s). Sutter considered the
usability of the system over time and since coloralteration between red and green was almost as
effective as having the monitor flicker, hechose to use the color alteration because it was shown
to be much less fatiguing for users.This system is basically the EEG version of an eye movement
recognition systemand contains similar problems because it assumes that the subject is always
looking at acommand on the computer screen.
2) P3 Character Recognition (University of Illinois, USA).
26Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
In a related approach, Farwell and Donchin use the P3 evoked potential [3.15]. A 6x6grid
containing letters from the alphabet is displayed on the computer monitor and usersare asked to
select the letters in a word by counting the number of times that a row orcolumn containing the
letter flashes. Flashes occur at about 10 Hz and the desired letterflashes twice in every set of
twelve flashes. The average response to each row andcolumn is computed and the P3 amplitude is
measured. Response amplitude is reliablylarger for the row and column containing the desired
letter. After two training sessions,users are able to communicate at a rate of 2.3 characters/min,
with accuracy rates of 95%.This system is currently only used in a research setting.
3) ERS/ERD Cursor Control (University of Technology Graz, Austria)
Pfurtscheller and his colleagues take a different approach Using multiple electrodes placed over
sensorimotor cortex they monitor
event-related synchronization/ desynchronization (ERS/ERD) [64]. In all sessions,epochs with
eye and muscle artifact are automatically rejected. This rejection can slowdown subject
performance. As this is a research system, the user application is a simplescreen that allows
control of a cursor in either the left or right direction. In anotherexperiment, for a single trial the
screen first appears blank, then a target box is shown onone side of the screen. A cross hair
appears to let the user know that he/she must begintrying to move the cursor towards the box.
Feedback may be Delayed or immediate anddifferent experiments have slightly different displays
and protocols. After two trainingsessions, three out of five student subjects were able to move a
cursor right or left withaccuracy rates from 89-100%. Unfortunately, the other two students
performed at 60%and 51%. When a third category was added for classification, performance
dropped to a
low of 60% in the best case [3.23].
4) A Steady State Visual Evoked Potential BCI (Wright-Patterson Air Force
Base, The Air Force Research Laboratory, USA).
Middendorf and colleagues use operant conditioning methods in order to trainvolunteers to
control the amplitude of the steady-state visual evoked potential (SSVEP) toflorescent tubes
flashing at 13.25 Hz [3.11][3.10][3.9]. This method of control may beconsidered as continuous as
the amplitude may change in a continuous fashion. Either ahorizontal light bar or audio feedback
is provided when electrodes located over theoccipital cortex measure changes in signal amplitude.
If the VEP amplitude is below orabove a specified threshold for a specific time period, discrete
control outputs aregenerated. After around 6 hours of training, users may have an accuracy rate of
27Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
greaterthan 80% in commanding a flight simulator to roll left or right.Recognizing that the
SSVEP may also be used as a natural response,
5) Mu Wave Cursor Control (Wadsworth Center, Albany, USA).
Wolpaw and his colleagues free their subjects from being tied to a flashing florescenttube by
training subjects to modify their mu wave [3.24][3.1]. This method of control iscontinuous as the
mu wave may be altered in a continuous manner. It can be attenuatedby movement and tactile
stimulation as well as by imagined movement. A subject's maintask is to move a cursor up or
down on a computer screen. While not all subjects are ableto learn this type of biofeedback
control, the subjects that do, perform with accuracygreater than or equal to 90%. These
experiments have also been extended to twodimensionalcursor movement, but the accuracy of
this is reported as having “not reachedthis level of accuracy” when compared to the one-
dimensional control [3.11].
.
6 An Implanted BCI (Georgia State University, USA).
The implanted brain-computer interface system devised by Kennedy andcolleagues has been
implanted into two patients [3.25][3.7]. These patients are trained to
control a cursor with their implant and the velocity of the cursor is determined by the rateof
neural firing. The neural waveshapes are converted to pulses and three pulses are aninput to the
computer mouse. The first and second pulses control X and Y position of thecursor and a third
pulse as a mouse click or enter signal.
The patients are trained using software that contains a row of icons representingcommon phrases
(Talk Assist developed at Georgia Tech). There are two paradigmsusing this software program
and a third one using the visual keyboard. In the firstparadigm, the cursor moves across the
screen using one group of neural signals and downthe screen using another group of larger
amplitude signals. Starting in the top left corner,the patient enters the leftmost icon. He remains
over the icon for two seconds so that thespeech synthesizer is activated and phrases are produced.
In the second paradigm, thepatient is expected to move the cursor across the screen from one icon
to the other. Thepatient is encouraged to be as accurate as possible, and then to speed up the
cursormovement while attempting to remain accurate. In the third paradigm, a visual keyboardis
shown and the patient is encouraged to spell his name as accurately and quickly aspossible and
then to spell anything else he wishes. Unfortunately, the maximumcommunication rate with this
BCI has been around 3 characters per minute.
28Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
3 ALS (Amyotrophic Lateral Sclerosis) is a fatal neuromuscular disease characterized
byprogressive muscle weakness resulting in paralysis.
8) The Flexible Brain Computer Interface (University of Rochester, USA).
Bayliss and colleagues [3.26] have performed an environmental controlapplication in a virtual
apartment that enables a subject to turn on/off a light, television
set, and radio or say Hi/Bye to a virtual person. This system uses the P3 evoked potentialin an
immersive and dynamic Virtual Reality world. The main drawback of P3-basedBCI's is their
slowness. Single trial analysis may speed up recognition, but often at thecost of accuracy.
A single trial accuracy average of 85% was obtained in an environment of virtualdriving.
Subjects were instructed to drive in a virtual town and stop at red stop lightswhile ignoring both
green and yellow lights. The subjects used a virtual reality helmet,and a go cart with brake,
accelerator, and steering output to control the virtual car. Whilethis choice could have caused
more artifacts in the signal collection (due to turning thesteering wheel and braking), most of the
artifact discovered and preprocessed was due toeye movement
APPLICATIONS
Introduction
Apart from being a non-conventional input device for a computer we have found three main
application fields for BCIs and BCI related devices which are more or less controversial:
– Medical applications
– Human enhancement
– Human manipulation
Medical applications
BCIs provide a new and possibly only communication channel for people suffering from severe
physical disabilities but having intact cognitive functions. For example these devices could help
in
treating (or rather overcoming) paraplegia or amyotrophia.
Somewhat related to this topic is the field of Neuroprosthetics which deals with constructing and
surgically implanting devices used for replacing damaged areas of the brain and more generally
for
neural damages of any kind.
For example, the most widespread neuroprosthetic (approx. 85,000 people worldwide 2005
29Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
[WIKI_NP]) is the cochlear implant or bionic ear. This device can help people with impaired
hearing. In contrast to conventional hearing aids this device is not a sound amplifier but directly
stimulates any appropriate functioning nerves.
Human enhancement
Definition
“Human enhancement describes any attempt (whether temporary or permanent) to overcome the
current limitations of human cognitive and physical abilities, whether through natural or artificial
means.” [WIKI_HE]
Having this definition in mind one can think of many applications of the BCI in this field. For
example BCIs could help facilitate communication systems in Cybernetic Organisms, Brainwave
Synchronization, or even speculative things such as the Exocortex, among others.
Brain Computer Interfaces - 13 - Behm / Kollotzek / Hüske
Cybernetic Organism describes the enhancement of an organism by means of technology. For
example a BCI could enable the attachment of robotic limbs without the use of the organism’s
original nervous system (as long as the brain is intact).
Brainwave Synchronization
“Brainwave synchronization is the practice to entrain one's brainwaves to a desired frequency, by
means of a periodic stimulus with corresponding frequency. The stimulus can be aural as in the
case
of binaural beats, or visual, as with a Dreamachine.” [WIKI_BS]
A BCI could be used to detect and classify the current state of mind of an individual and actively
adjust the frequency of the Brainwave Synchronization to achieve a certain state of mind.
Excortex
“An exocortex (speculative) is an external information processing system that augments, in a
subtle
and seamless fashion via a brain-computer interface, the brain's biological high-level cognitive
processes.” [WIKI_EC]
Human manipulation
The notion that a BCI could allow a two-way communication between a human and a computer
gives rise to more controversial potential uses of such a device. Using such a communication
mechanism one could imagine directly influencing an individual’s thoughts, decisions, emotions
or
thinking. Of course, the mere “reading” of the mind could be put to criminal use, e.g. unwanted
30Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
reading of passwords, locations, etc.
While this may sound like science-fiction, methods researched in social-psychology such as
advertisement, media manipulation, propaganda, group dynamics or peer pressure have been
proven
to be successful in altering an individual’s behavior and their effectiveness is undisputed. So, in
principle manipulation is possible.
Experiments to explore possibilities of mind-control are by no means theoretical and include the
use
of drugs and electronic signals to alter brain functioning. For example the project codenamed
MKULTRA conducted by the CIA dates back to the 1950’s and was aimed at researching
mindcontrol
[WIKI_MK].
Brain-computer-interfaces present a new level of technology that could be used to actively
manipulate an individual.
The BMI technologies of today can be broken into three major areas:
1. Auditory and visual prosthesis
- Cochlear implants
- Brainstem implants
- Synthetic vision
- Artificial silicon retina
2. Functional-neuromuscular stimulation (FNS)
FNS systems are in experimental use in cases where spinal cord damage or a stroke has severed
the link between brain and the peripheral nervous system. They can use brain to control their own
limbs by this system
3. Prosthetic limb control
Thought controlled motorized wheel chair.
31Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Thought controlled prosthetic arm for amputee.
Various neuroprosthetic devices
Other various applications are Mental Mouse Applications in technology products, e.g., a mobile
phone attachment that allows a physically challenged user to dial a phone number without
touching it or speaking into it. System lets you speak without saying a word in effective 16
`construction of unmanned systems, in space missions, defense areas etc. NASA and DARPA
have used this technology effectively. Communication over internet can be modified.
The Mental Typewriter:
March 14, 2006 Scientists demonstrated a brain-computer interface that translates brain signals
into computer control signals this week at CeBIT in Berlin. The initial project demonstrates how
a paralysed patient could communicate by using a mental typewriter alone – without touching the
keyboard. In the case of serious accident or illness, a patient’s limbs can be paralyzed, severely
restricting communication with the outside world. The interface is already showing how it can
help these patients to write texts and thus communicate with their environment It will be some
years, though, before the mental typewriter can be used in everyday applications. Further research
is needed, in particular to refine the EEG sensors.
BCI offers paralyzed patients improved quality of life:
Tuebingen, Germany. A brain–computer interface installed early enough in patients with neuron-
destroying diseases can enable them to be taught to communicate through an electronic device
and slow destruction of the nervous system The research focuses on a condition called the
completely locked-in state (CLIS, a total lack of muscle control). In a CLIS situation, intentional
thoughts and imagery can rarely be acted upon physically and, therefore, are rarely followed by a
stimulus. The research suggests that as the disease progresses and the probability for an external
event to function as a link between response and consequence becomes progressively smaller it
may eventually vanish altogether.
Researchers have found that by implementing a brain-computer –interface before the completely
locked-in state occurs, a patient can be taught to communicate through an electronic device with
great regularity. The continued interaction between thought, response and consequence is
believed to slow the destruction of the nervous system.
The findings are also raising a number of new questions about the quality of life amongst
paralysis sufferers. Patients surveyed were found to be much healthier mentally than
psychiatrically depressed patients without any life-threatening bodily disease. Only 9% of ALS
32Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
patients showed long episodes of depression and most were during the period following diagnosis
and a period of weeks after tracheotomy
Adaptive BCI for Augmented Cognition and
Action :
The goal of this project is to demonstrate improved human/computer performance for specific
tasks through detection of task-relevant cognitive events with real-time EEG (Fig. 1). For
example, in tasks for which there is a direct tradeoff between reaction time and error rate, (such
as typing or visual search) it may be beneficial to correct a user’s errors without interrupting the
pace of the primary task. Such a user interface is possible through the direct detection of EEG
signatures associated with the perception of a error, often referred to as Error Related Negativity.
In general such signatures may be used to dynamically adjust the behavior of human-computer
interfaces and information displays.
This project advances signal analysis techniques for high density EEG to detect discrete events
associated with cognitive processing. Corresponding real-time adaptive interfaces with sub-
second latency are being designed to evaluate this concept of an adaptive brain-computer
interface in three specific applications (1) Error and conflict perception:
Error related negativity (ERN) in EEG has been linked to perceived response errors and conflicts
in decision-making. In this project we have developed single trial ERN detection to predict task-
related errors. The system can be used as an automated real-time decision checker for time-
sensitive control tasks. In the first phase of this project we demonstrated improved
human/computer performance at a rapid forced choice discrimination task with an average 23%
reduction of human errors (results on one subject are shown in Fig. 2). This open-loop error
correction paradigm represents the first application of real-time cognitive event detection and
33Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
demonstrates the utility of real-time EEG brain monitoring within the Augmented Cognition
program. We will evaluate video game scenarios with closed-loop feedback at latencies of less
than 150 ms where detected errors are corrected or application parameters such as speed are
varied according to the measured or "gauged" conflict perception.
(2) Working memory encoding.
Transient modulation of oscillations in the theta (4-8 Hz) and gamma (20-30 Hz) bands, recorded
using EEG and magnetoencephalography (MEG), have been implicated in the encoding and
retrieval of semantic information in working memory. In this project we will exploit these neural
correlates of semantic processing to detect problems with semantic information processing. This
memory gauge could be used to detect memory recall deficits, and repeat or enhance the
presented information and thus better prime memory recall.
(3) Rapid visual recognition:
We are exploring the signals elicited by visual target detection, which were recently observed in
rapid sequential visual presentation (RSVP) experiments. We have demonstrated that the
detection of these signals on a single trial basis can be used to replace the slow manual response
of a human operator, thereby significantly increasing the throughput of image search tasks (Fig
3). This paradigm has the potential to improve the performance of Image Analysts who need to
routinely survey large volumes of aerial imagery within short periods of time. In addition, the
approach looks to measure the "bottleneck" between constant delay perceptual processing and
more variable delay cognitive processing. Thus the detected signatures can be used to "gauge" if
cognitive systems are capable/incapable of assimilating perceptual input for fast decision making.
In the first phase of this project a fully automated real-time signal analysis system and hardware
infrastructure has been developed that can give short latency feedback to the user within 50ms of
the recorded activity. The signal processing system adaptively learns to detect evoked responses
from the real-time streaming EEG signal. The current system, which is used for tasks 1 and 3, can
be
configured for single trial detection for any number of cognitive events such ERN, rapid visuual
recognition, readiness potential, response to oddball stimulus (P300), as well as conventional
visual, auditory, or somato-sensory responses. We are in the progress of applying this system to
event detection in the Warship Commander - a common task set proposed for integration and
evaluation by the Augmented Cognition Program
34Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
35Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Experimental Brain Computer Interface Software for the Modular EEG (The ABI software)
6.1. Introduction:
ABI is a simple software for the Modular EEG that implements an experimental Brain Computer
Interface (BCI). Nowadays, BCI research is an highly active field, but the existing technology is
still immature for its use outside of a lab's settings. The ABI software tries to provide a simple
tool for hobbyists to do experiments on its own with BCI technology.
Work of the software:
The ABI is a BCI based on trials. A trial is a time interval where the user generates brainwaves to
perform an action. The BCI tries to process this signal and to associate it to a given class. The
association is done by feeding a neural net with the preprocessed EEG data. The neural net's
output is then further processed and this final output corresponds to the given class. The neural
net should be trained in order to learn the association.
The classifier's idea is heavily based on Christin Schäfer's design (winner of the BCI Competition
II, Motor Imaginery Trials).
The ABI software allows you to
Do simple Biofeedback. You can display raw EEG channels, narrow band frequency amplitudes
and classes.
Simulate trials.
Record trials for a number of choice of different classes.
Train the interface.
6.3. The classification achieved by this software:
The method has been previously applied to the data provided by the BCI Competition II data
(dataset III, Graz University, Motor Imaginary) and compared against the results obtained by the
contributors. The method has outperformed the results achieved by them, obtaining a higher
Mutual Information (which was the criterion used in the competition) of 0.67 bits (the winner of
the competition obtained 0.61 bits).
Of course, it is very important that more people test the software and report its results to improve
the method. Statistical stability can only be guaranteed if more people try it out.
6.4. Instructions:
36Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
By executing ABI, it reads a configuration file called "abi.txt" (which you can edit with a simple
text editor), where the way the BCI should act is specified. ABI tries to load the trial file defined
in the configuration file. The trial file is a text database containing trials for different classes.
ABI has three operating modes: SIMULATION, RECORDING and TRAINING. You can
switch between operating modes by pressing F1, F2 or F3 respectively (the software doesn't
change its mode instantly, because a trial shouldn't be interrupted in the middle).
The operation is quite simple. The user records several trials for the different classes
(RECORDING mode). Each class is associated to a different mental task. After recording a
reasonable amount of trials (more than 50 trials for each class), the user can train the system to
learn a way to discriminate between the different classes (TRAINING mode). This process can
be repeated in order to improve the quality of the recognition. The system can be tested under the
SIMULATION mode.
An explanation of the different modes follows.
6.4.1.SIMULATION and RECORDING
These two modes perform single trials. The SIMULATION mode is used to test the BCI.
RECORDING is the same as SIMULATION, with the difference that the EEG data is recorded
and used as training examples. A trial has the following structure
37Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
As you can see, a trial is composed of three subintervals, whose duration is defined by the
variables TPreparation, TPrerecording and TrialLength, in the configuration file.
. TRAINING
Pressing the F3 key, the system starts to train the neural net with the available data. The training set used for this purpose is the
set of the last TrialBuffer recorded trials' features. Example: Suppose you have recorded 300 trials, and TrialBuffer = 100
Then the system extracts the features of the 100 last recorded trials to form the training set.
Training time depends upon the complexity of the training data and the amount of recorded data. The training data is not always
separable. If the mental task for class 1 is too similar to the mental task for class 2, then the neural net won't be able to do the
separation: this isn't magic :-) .
considerations
BCI technology is still in its infancy, so little is known about which mental tasks are better than
others for BCIs. Also, the electrode placing is important. If your electrode setting isn't appropiate,
then it can happen that they even aren't recording the cortical areas related to the mental task!
38Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Research has discovered the following changes in electrical activity during mental tasks (this list
isn't complete, I hope that the OpenEEG community will discover some more):
Motor Imaginery: Imagination of physical movement produces changes in the sensorymotor
cortex. In example, imagination of left and right middle finger imagination produces changes,
namely (de-)synchronization on electrode positions around C3 and C4. Good features are around
10 and 20 Hz.
Rotation of 3D objects: Literature stated that during imagination of rotation of 3d objects
involves frontal and temporal lobe activity. They seem to sinchronize. Good features are around
10 Hz.
39Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
CONCLUSION
Present and Future.
The practical use of BCI technology depends on an interdisciplinary cooperationbetween
neuroscientists, engineers, computer programmers, psychologists, and
rehabilitation specialists, in order to develop appropriate applications, to identifyappropriate users
groups, and to pay careful attention to the needs and desires of
individual users.The prospects for controlling computers through neural signals are
indeeddifficult to judge because the field of research is still in its infancy. Much progress hasbeen
made in taking advantage of the power of personal computers to perform theoperations needed to
recognize patterns in biological impulses, but the search for new andmore useful signals still
continues.If the advances of the 21st century match the strides of the past few decades,
directneural communication between humans and computers may ultimately mature and
findwidespread use. Perhaps newly purchased computers will one day arrive with
biologicalsignal ensors and thought-recognition software built in, just as keyboard and mouse
arecommonly found on today's units. Cultures may have diverse ethics, but regardless, individual
liberties and human life are always valued over and above machines. What happens when humans
merge with machines? The question is not what will the computer be like in the future, but
instead, what will we be like? What kind of people are we becoming?
BMI‘s will have the ability to give people back their vision and hearing. They will also change
the way a person looks at the world. Someday these devices might be more common than
keyboards. Is someone with a synthetic eye, less a person than someone without? Shall we
process signals like ultraviolet, X-rays, or ultrasounds as robots do? These questions will not be
answered in the near future, but at some time they will have to be answered. What an interesting
day that will be. And further still:
In principle we could do this for all the senses—record not just whatyou see, but also what you
hear, taste, smell, and feel, all at the levelof your brain. Playing back such an experience would be
a little likereliving it. You might even be able to play that kind of sensory recordingback for
someone else, turning the experience you had into a set ofnerve impulses that could be sent into
the other person’s brain, allowinghim or her to experience at least the sensory parts of an event
from
your perspective. . . .
40Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
ANNEXURE 1: BIOMEDICAL CONSIDERATIONS.
The brain contains approximately 100 billion neurons [A1-1]. Each neuron may have
as many as 2000 or more connections with other neurons and may receive as many as
20,000 inputs. Abstractly, a neuron is probably the most diverse, in terms of form and
size, of all cells in the body; however, all neurons have in common the functional
properties of integration, conduction and transmission of nerve impulses. A neuron
consists of three basic parts (Fig. A1-1):
Cell body (or soma). This main part has all of the necessary components of the
cell, such as the nucleus (contains DNA), endoplasmic reticulum and ribosomes
(for building proteins) and mitochondria (for making energy). If the cell body
dies, the neuron dies.
Axon - This long, cable-like projection of the cell carries the electrochemical
message (nerve impulse or action potential) along the length of the cell.
Depending upon the type of neuron, axons can be covered with a thin layer of
myelin, like an insulated electrical wire. Myelin is made of fat, and it helps to
speed transmission of a nerve impulse down a long axon. Myelinated neurons are
typically found in the peripheral nerves (sensory and motor neurons), while nonmyelinated
neurons are found in the brain and spinal cord.
Dendrites or nerve endings - These small, branch-like projections of the cell
make connections to other cells and allow the neuron to talk with other cells or
perceive the environment. Dendrites can be located on one or both ends of the
cell. There are two types of dendrites, apical and basal dendrites.
41Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Figure A1-1. Neuron topology.
Each neuron is in contact through its axon and dentrites with other neurons, so that each neuron is
an
interconnecting segment in the network of the nervous system.
57
Neurons come in many sizes. For example, a single sensory neuron from the
fingertip has an axon that extends the length of the arm, while neurons within the brain
may extend only a few millimeters. Neurons have different shapes depending on what
they do. Motor neurons that control muscle contractions have a cell body on one end, a
long axon in the middle and dendrites on the other end; sensory neurons have dendrites
on both ends, connected by a long axon with a cell body in the middle. Neurons also vary
with respect to their functions:
Sensory neurons carry signals from the outer parts of the body (periphery) into
the central nervous system.
Motor neurons (motoneurons) carry signals from the central nervous system to
the outer parts (muscles, skin, glands) of the body.
Receptors sense the environment (chemicals, light, sound, touch) and encode this
information into electrochemical messages that are transmitted by sensory
neurons.
Interneurons connect various neurons within the brain and spinal cord.
Figure A1-2. Some types of neurons: interneuron, sensory neuron, motoneuron and cortical
pyramidal cell.
The synapse, a specialized site of contact between neurons, is of prime
42Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
significance in the integrative activities of the nervous system, as the information from
one neuron flows to another neuron across of it. The synapse is a small gap separating 2
neurons, and consists of:
58
1. A presynaptic ending that contains neurotransmitters, mitochondria and other cell
organelles.
2. A postsynaptic ending that contains receptor sites for neurotransmitters.
3. The synaptic cleft: a space between the presynaptic and postsynaptic endings.
For communication between neurons, an electrical impulse must travel down an axon
to the synaptic terminal.
Electric potentials are produced at the synaptic junctions, which can be localized over
the axon (axoaxonic synapse), the soma (axosomatic synapse) or the dentrites
(axodendritic synapse) (Fig A1.3), and reflect the communication between neurons.
When a neurotransmitter binds to a receptor on the postsynaptic side of the synapse, it
changes the postsynaptic cell's excitability: it makes the postsynaptic cell either more or
less likely to fire an action potential. If the number of excitatory postsynaptic events are
large enough, they will cause an action potential in the postsynaptic cell and the
continuation of the "message." In addition to The EEG pick up this synchronized
subthreshold dentritic potentials produced by the postsynaptic activity of many neurons
summed [A1-2].
Figure A1-3. Types of synapses. From left two right: axoaxonic synapse, axodendritic synapse,
and
axosomatic synapse.
Not all types of brain activity have identical impact on the EEG. The depth,
43Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
orientation and intrinsic symmetry of connections in the cortex are significant in it. The
primary cell of importance in the neocortex is the pyramidal cell. It's known that its
neurotransmitter is a potent excitatory transmitter. The pyramidal cell receives many
inputs from stellate cells that are also excitatory. The pyramidal cell is different from
other neurons in that it violates one of the fundamental rules of standard
neurophysiology, that only axons produce action potentials which transmit information
from one cell to another, and dendrites produce excitatory and inhibitory slow potentials
that summate at the axon hillock where they establish the action potentials. In the case of
the pyramidal cell, the apical dendrite, which is a long shaft between the basal and apical
region, can actually produce action potentials, and these in turn act to amplify the
powerful action potentials that now project to output systems--sensory, motor,
autonomic, or integrative [A1.3]. For this reason, pyramidal cells are thought to cause the
strongest part of the EEG signal
THE HUMAN BRAIN
All of it happens in the brain. The brain is undoubtedly the most complex organ found among the
carbon-based life forms. So complex it is that we have only vague information about how it
works. The average human brain weights around 1400 grams. The most relevant part of brain
concerning BMI‘s is the cerebral cortex. The cerebral cortex can be divided into two
hemispheres. The hemispheres are connected with each other via corpus callosum. Each
hemisphere can be divided into four lobes. They are called frontal, parietal, occipital and
temporal lobes. Cerebral cortex is responsible for many higher order functions like problem
solving, language comprehension and processing of complex visual information. The cerebral
cortex can be divided into several areas, which are responsible of different functions. This kind of
knowledge has been used when with BCI‘s based on the pattern recognition approach. The
mental tasks are chosen in such a way that they activate different parts of the cerebral cortex.
Cortical Area Function
Auditory Association Area Processing of auditory information
Auditory Cortex Detection of sound quality (loudness, tone)
Speech Center (Broca‘s area) Speech production and articulation
Prefrontal Cortex Problem solving, emotion, complex thought
Motor Association Cortex Coordination of complex movement
Primary Motor Cortex Initiation of voluntary movement
Primary Somatosensory Cortex Receives tactile information from the body
44Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Sensory Association Area Processing of multisensory information
Visual Association Area Complex processing of visual information
Wernicke‘s Area Language comprehension
PRINCIPLES OF ELECTROENCEPHALOGRAPHY.
The Nature of the EEG signals.
The electrical nature of the human nervous system has been recognized for more
than a century. It is well known that the variation of the surface potential distribution on
the scalp reflects functional activities emerging from the underlying brain [2.1]. This
surface potential variation can be recorded by affixing an array of electrodes to the scalp,
and measuring the voltage between pairs of these electrodes, which are then filtered,
amplified, and recorded. The resulting data is called the EEG. Fig. 1-1 shows waveforms
of a 10 second EEG segment containing six recording channels, while the recording sites
are illustrated in Fig. 2-2. In our experiments, we have used the10-20 System of
Electrode Placement, which is based on the relationship between the location of an
electrode and the underlying area of cerebral cortex (the "10" and "20" refer to the 10%
or 20% interelectrode distance) [2.7].
Figure
45Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Figure 2-2. The 10-20 System of Electrode Placement.
Each site has a letter (to identify the lobe) and a number or another letter to identify the
hemisphere
location. The letters F, T, C, P, and O stand for Frontal, Temporal, Central, Parietal and Occipital.
(Note
that there is no "central lobe", but this is just used for identification purposes.) Even numbers
(2,4,6,8) refer
to the right hemisphere and odd numbers (1,3,5,7) refer to the left hemisphere. The z refers to an
electrode
placed on the midline.
Nasion: point between the forehead and nose.
Inion: Bump at back of skull
The EEG is thought to be the synchronized subthreshold dentritic potentialsproduced by the
synaptic activity of many neurons summed [2.2]. In its formation not all
types of brain activity have identical impact. The depth, orientation and intrinsicsymmetry of
connections in the cortex are significant in it. As it is exposed in previous
works [2.2][2.3], pyramidal cells are thought to cause the strongest part of the
EEGsignal1.Nowadays, modern techniques for EEG acquisition collect these underlyingelectrical
patterns from the scalp, and digitalize them for computer storage. Electrodesconduct voltage
potentials as microvolt level signals, and carry them into amplifiers thatmagnify the signals
approximately ten thousand times. The use of this technologydepends strongly on the electrodes
positioning and the electrodes contact. For this reason,electrodes are usually constructed from
conductive materials, such us gold or silverchloride, with an approximative diameter of 1 cm, and
subjects must also use aconductive gel on the scalp to maintain an acceptable signal to noise ratio.
46Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
EEG wave groups.
The analysis of continuous EEG signals or brain waves is complex, due to thelarge amount of
information received from every electrode. As a science in itself, it has tobe completed with its
own set of perplexing nomenclature. Different waves, like so manyradio stations, are categorized
by the frequency of their emanations and, in some cases,by the shape of their waveforms.
Although none of these waves is ever emitted alone, thestate of consciousness of the individuals
may make one frequency range morepronounced than others. Five types are particularly
important:
BETA. The rate of change lies between 13 and 30 Hz, and usually has a low voltage
between 5-30 V (Fig. 2-6). Beta is the brain wave usually associated with active
thinking, active attention, focus on the outside world or solving concrete problems. It can
reach frequencies near 50 hertz during intense mental activity.
Figure 2-4. Alpha (left) and Beta (right) waves.
ALPHA. The rate of change lies between 8 and 13 Hz, with 30-50 V amplitude (Fig 2-
4). Alpha waves have been thought to indicate both a relaxed awareness and alsoinattention. They
are strongest over the occipital (back of the head) cortex and also overfrontal cortex. Alpha is the
most prominent wave in the whole realm of brain activity andpossibly covers a greater range than
has been previously thought of. It is frequent to see apeak in the beta range as high as 20 Hz,
which has the characteristics of an alpha staterather than a beta, and the setting in which such a
response appears also leads to the sameconclusion. Alpha alone seems to indicate an empty mind
rather than a relaxed one, amindless state rather than a passive one, and can be reduced or
eliminated by opening theeyes, by hearing unfamiliar sounds, or by anxiety or mental
concentration.
THETA. Theta waves lie within the range of 4 to 7 Hz, with an amplitude usually greaterthan 20
V. Theta arises from emotional stress, especially frustration or disappointment.Theta has been
47Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
also associated with access to unconscious material, creative inspirationand deep meditation. The
large dominant peak of the theta waves is around 7 Hz.
DELTA. Delta waves lie within the range of 0.5 to 4 Hz, with variable amplitude. Deltawaves
are primarily associated with deep sleep, and in the waking state, were thought toindicate
physical defects in the brain. It is very easy to confuse artifact signals caused bythe large muscles
of the neck and jaw with the genuine delta responses. This is becausethe muscles are near the
surface of the skin and produce large signals whereas the signalwhich is of interest originates
deep in the brain and is severely attenuated in passingthrough the skull. Nevertheless, with an
instant analysis EEG, it is easy to see when theresponse is caused by excessive movement.
Figure 2-6. Delta wave.
GAMMA. Gamma waves lie within the range of 35Hz and up. It is thought that this bandreflects
the mechanism of consciousness - the binding together of distinct modular brainfunctions into
coherent percepts capable of behaving in a re-entrant fashion (feeding backon themselves over
time to create a sense of stream-of-consciousness).
MU. It is an 8-12 Hz spontaneous EEG wave associated with motor activities andmaximally
recorded over motor cortex (Fig. 2-8). They diminish with movement or thentention to move. Mu
wave is in the same frequency band as in the alpha wave (Fig. 2-7), but this last one is recorded
over occipital cortex.
Figure 2-7. Mu (left) and alpha (right) waves.
48Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Most attempts to control a computer with continuous EEG measurements work bymonitoring
alpha or mu waves, because people can learn to change the amplitude of thesetwo waves by
making the appropriate mental effort. A person might accomplish thisresult, for instance, by
recalling some strongly stimulating image or by raising his or herlevel of attention.
49Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
REFERENCES:
E. R. Kandel, J. H. Schwartz, and T. M. Jessell, Principles of Neural Science, 3rd
ed. New York: Elsevier/North-Holland, 1991
Adam Keiper ,The Age of Neuroelectronics
Alexander Behm, Markus A. Kollotzek and Fabian Hüske. Controlling computers
by thoughts. http://spectrum.ieee.org/biomedical/devices/the-brainmachine-
interface-unplugged/0
B.W.Jervis, E.C.Ifeachor, and E.M.Allen. The removal o ocular artifacts from the
BCI Competition, http://liinc.bme.columbia.edu/competition.htm
Berger http://en.wikipedia.org/wiki/Hans_Berger
Berlin Brain-Computer-Interface, http://bbci.de/
Brain-Computer Interface Technology: Theory and Practice:First International
Meeting Program and Papers, The Rensselaerville Institute, Rensselaerville, New
York
Brain-Computer Interface, http://en.wikipedia.org/wiki/Brain_computer_interface
Dr Larry Abbott . Electrical properties of neurons
electroencephalogram :a review . http://newton.bme.columbia.edu/wolpaw.html
Electroencephalography, pages 15-26
Exocortex , http://en.wikipedia.org/wiki/Exocortex
F. H. Lopes da Silva and A. van Rotterdam. Biophysical aspects of EEG and
MEG generation. In E. Niedermeyer and F. H. Lopes da Silva, editors,
G. Gratton, M.G.H. Coles and E. Donchin. A New Method for off – line removal
of Ocular artifacts. Electroencephalography and Neurophysiology, Vol 55, pp.
468-484,1983
HansGraz Brain-Computer-Interface, http://bci.tugraz.at/index.html
Hewett, Baecker, Card, Carey, Gasen, Mantei, Perlman, Strong and
Jorge Baztarrica Ochoa,Gary Garcia Molina.Touradj Ebrahimi. EEG Signal
Classification for BrainComputer Interface Applications,
50Dept. of ECE College of Engineering
Brain Computer Interfacing Seminar Report 2011
Middendorf, M.S., McMillan, G., Calhoun, G., und Jones, K.S.,1999,Brain-
computer interfaces based on the steady-state visual-evoked response
Neuroimaging, http://en.wikipedia.org/wiki/Neuroimaging
Schalk, Dennis J. McFarland, and Jonathan R. Wolpaw. BRAIN-COMPUTER
INTERFACES FOR COMMUNICATION AND CONTROL
Verplank. Human-Computer Interaction, http://sigchi.org/cdg/cdg2.html
51Dept. of ECE College of Engineering