Brain Port Vision Technology[1]

8/3/2019 Brain Port Vision Technology[1]

1/14

1 | P a g e

BRAIN PORT VISION TECHNOLOGY

1. The voice for the Blind

1.1 INTRODUCTION:-

Brain Port is a technology where by sensory information can be sent to one's brain via a

signal from the Brain Port (and its associated sensor) that terminates in an electrode array which

sits atop the tongue. It was initially developed by Paul Bach-y-Rita as an aid to people's sense of

balance, particularly of stroke victims. Brain Port technology has been developed for use as a

visual aid. For example, the Brain Port has demonstrated its ability to allow a blind person to see

his surroundings in polygonal and pixel form. In this scenario, a camera picks up the image of

the surrounding; the information is processed by a chip which converts it into impulses which are

sent through an electrode array, via the tongue, to the person's brain. The human brain is able to


2/14

2 | P a g e

interpret these impulses as visual signals and they are then redirected to the visual cortex,

allowing the person to see. Brain Port technology is based on the phenomenon of sensory

substitution. For the vision application, visual information is perceived via the sense of touch on

the human tongue. Well, not exactly through her tongue, but the device in her mouth sent visual

input through her tongue in much the same way that seeing individuals receive visual input

through the eyes. All sensory information sent to the brain is carried by nerve fibers in the form

of patterns of impulses, and the impulses end up in the different sensory centers of the brain for

interpretation. To substitute one sensory input channel for another, you need to correctly encode

the nerve signals for the sensory event and send them to the brain through the alternate channel.

The brain appears to be flexible when it comes to interpreting sensory input. You can train it to

read input from, say, the tactile channel, as visual or balance information, and to act on it

accordingly. In JS On line's "Device may be new pathway to the brain,

What is the Brain Port vision device?


3/14

3 | P a g e

The Brain Port vision device is an investigational non-surgical assistive visual prosthetic

device that translates information from a digital video camera to your tongue, through gentle

electrical stimulation. The Brain Port vision system consists of a postage-stamp-size electrode

array for the top surface of the tongue (the tongue array), a base unit, a digital video camera, and

a hand-held controller for zoom and contrast inversion. Visual information is collected from the

user-adjustable head-mounted camera (FOV range 390 degrees) and sent to the BrainPort base

unit. The base unit translates the visual information into an stimulation pattern that is displayed

on the tongue. The tactile image is created by presenting white pixels from the camera as strong

stimulation, black pixels as no stimulation, and gray levels as medium levels of stimulation, with

the ability to invert contrast when appropriate. Users often report the sensation as pictures that

are painted on the tongue with Champagne bubbles. With the current system (arrays containing

100 to 600+ electrodes), study participants have been able to recognize high-contrast objects,

their location, movement, and some aspects of perspective and depth. Trained blind participants

use information from the tongue display to augment understanding of the environment. Ongoing

research with the Brain Port vision device demonstrates the great potential of tactile vision

augmentation and we believe that these findings warrant further exploration.

Parts of brain port:

Brain Port uses the tongue instead of the fingertips, abdomen or back used by othersystems. The tongue is more sensitive than other skin areas -- the nerve fibers are closer to the

surface, there are more of them and there is no stratum corneum (an outer layer of dead skin

cells) to act as an insulator. It requires less voltage to stimulate nerve fibers in the tongue -- 5 to

15 volts compared to 40 to 500 volts for areas like the fingertips or abdomen. Also, saliva

contains electrolytes, free ions that act as electrical conductors, so it helps maintain the flow of

current between the electrode and the skin tissue. And the area of the cerebral cortex that

interprets touch data from the tongue is larger than the areas serving other body parts, so the

tongue is a natural choice for conveying tactile-based data to the brain.


4/14

4 | P a g e

An accelerometer is a device that measures,

among other things, tilt with respect to the pull

of gravity. The accelerometer on the underside

of the 10-by-10 electrode array transmits data

about head position to the CPU through the

communication circuitry. When the head tilts

right, the CPU receives the "right" data and

sends a signal telling the electrode array to

provide current to the right side of the wearer's

tongue. When the head tilts left, the device

buzzes the left side of the tongue. When the

head is level, Brain Port sends a pulse to the

middle of the tongue. After multiple sessions

with the device, the subject's brain starts to pick

up on the signals as indicating head position --

balance information that normally comes from

the inner ear -- instead of just tactile

information. From the CPU, the signals are

sent to the tongue via a "lollipop," an electrode


5/14

5 | P a g e

array about nine square centimeters that sits directly on the tongue. Each electrode corresponds

to a set of pixels. White pixels yield a strong electrical pulse, whereas black pixels translate into

no signal. Densely packed nerves at the tongue surface receive the incoming electrical signals,

which feel a little like Pop Rocks or champagne bubbles to the user. A dental-retainer-like unit

would house a battery, the electrode array and all of the microelectronics necessary for signal

encoding and transmitting. In the case of the Brain Port vision device, the electronics might be

completely embedded in a pair of glasses along with a tiny camera and radio transmitter, and the

mouthpiece would house a radio receiver to receive encoded signals from the glasses. It's not

exactly a system on a chip.

1.2WORKING OF BRAIN PORT:-

To produce tactile vision, Brain Port uses a camera to capture visual data. The optical

information -- light that would normally hit the retina -- that the camera picks up is in digital

form, and it uses radio signals to send the ones and zeroes to the CPU for encoding. Each set of


6/14

6 | P a g e

pixels in the camera's light sensor corresponds to an electrode in the array. The CPU runs a

program that turns the camera's electrical information into a spatially encoded signal. The

encoded signal represents differences in pixel data as differences in pulse characteristics such as

frequency, amplitude and duration. Multidimensional image information takes the form of

variances in pulse current or voltage, pulse duration, intervals between pulses and the number of

pulses in a burst, among other parameters. To the extent that a trained user may simultaneously

distinguish between multiple of these characteristics of amplitude, width and frequency, the

pulses may convey multidimensional information in much the same way that the eye perceives

color from the independent stimulation of different color receptors. The electrode array receives

the resulting signal via the stimulation circuitry and applies it to the tongue. The brain eventually

learns to interpret and use the information coming from the tongue as if it were coming from the

eyes.

An experience (research) of a blind woman who go to see the world with the help of

this evolutionary device BRAINPORT:

A blind woman sits in a chair holding a video camera focused on a scientist sitting in front of

her. She has a device in her mouth, touching her tongue, and there are wires running from that

device to the video camera. The woman has been blind since birth and doesn't really know what

a rubber ball looks like, but the scientist is holding one, and when he suddenly rolls it in her

direction, she puts out a hand to stop it. The blind woman saw the ball through her tongue. Well,

not exactly through her tongue, but the device in her mouth sent visual input through her tongue


7/14

7 | P a g e

in much the same way that seeing individuals receive visual input through the eyes. In both

cases, the initial sensory input mechanism the tongue or the eyes send the visual data to

the brain, where that data is processed and interpreted to form images.

1.3 CONCEPT OF ELECRTICALSTIMULATION:-

Electro tactile stimulation for sensory augmentation orsubstitution, an area of study that involves

using encoded electric current to represent sensory information that a person cannot receive through the

traditional channel and applying that current to the skin, which sends the information to the brain. The

brain then learns to interpret that sensory information as if it were being sent through the traditional

channel for such data. Most of us are familiar with the augmentation or substitution of one sense for


8/14

8 | P a g e

another. Eyeglasses are a typical example of sensory augmentation. Braille is a typical example of

sensory substitution -- in this case, you're using one sense, touch, to take in information normally intended

for another sense, vision. Electro tactile stimulation is a higher-tech method of receiving somewhat

similar (although more surprising) results, and it's based on the idea that the brain can interpret sensory

information even if it's not provided via the "natural" channel.

The idea is to communicate non-tactile information via electrical stimulation of the sense

of touch. In practice, this typically means that an array of electrodes receiving input from a non-

tactile information source (a camera, for instance) applies small, controlled, painless currents

(some subjects report it feeling something like soda bubbles) to the skin at precise locations

according to an encoded pattern. The encoding of the electrical pattern essentially attempts to

mimic the input that would normally be received by the non-functioning sense. So patterns

of light picked up by a camera to form an image, replacing the perception of the eyes, are

converted into electrical pulses that represent those patterns of light. When the encoded pulses

are applied to the skin, the skin is actually receiving image data those nerve fibers forward their

image-encoded touch signals to the tactile-sensory area of the cerebral cortex, the parietal lobe.

Within this system, arrays of electrodes can be used to communicate non-touch information

through pathways to the brain normally used for touch-related impulses.

The multiple channels that carry sensory information to the brain, from the eyes, ears and

skin, for instance, are set up in a similar manner to perform similar activities. All sensory

information sent to the brain is carried by nerve fibers in the form ofpatterns of impulses, and

the impulses end up in the different sensory centers of the brain for interpretation. To substitute

one sensory input channel for another, you need to correctly encode the nerve signals for the

sensory event and send them to the brain through the alternate channel. The brain appears to be

flexible when it comes to interpreting sensory input. You can train it to read input from, say, the

tactile channel, as visual or balance information, and to act on it accordingly. In JS Online's

"Device may be new pathway to the brain," University of Wisconsin biomedical engineer and

Brain Port technology co-inventor Mitch Tyler states, "It's a great mystery as to how that processtakes place, but the brain can do it if you give it the right information." Action potentials (AP's)

thus recorded had amplitudes from 0.1 to 1.0 mV and a 5 : 1 signal-to-noise ratio (SNR).A

circular electrode surrounding the recording site served as the ground reference. Following pre

amplification and band pass filtering (200-10 000 Hz), a differential amplitude detector

identified AP's, producing an output pulse whenever the recorded signal entered a predefined


9/14

9 | P a g e

amplitude-time window. In the first experiment, electro tactile entrainment currents (iEN) were

determined by adjusting the stimulation current from near zero to the minimal value resulting in

one AP for each stimulation pulse. These currents exceeded the absolute thresholds (the currents

causing occasional AP's) by approximately 5%. The entrainment current was determined twice

for positive- and negative-polarity stimulation pulses of ten different widths: 20, 30, 40, 50, 70,

100, 150, 200, 300, and 500 _s, delivered at a rate of 10 pulses/s. The width sequence was

reversed during the second run on each of the three fibers.

Relative timing between simultaneous mechanical and electrotactile stimulation.

The top trace represents the sinusoidal, 30-Hz, 50-100-_m (0-P) mechanical displacement.

1.4WHAT BRAINPORT USERS SEE:-


10/14

10 | P a g e

With the current prototype (arrays containing 100 to 600+ electrodes), study participants

have recognized the location and movement of high-contrast objects and some aspects of

perspective and depth. In most studies, participants use the device for between two and 10 hours,

often achieving the following milestones:

y Within minutes: users perceive where in space stimulation arises (up, down, left, andright) and the direction of movement

y Within an hour: users can identify and reach for nearby objects, and point to and estimatethe distance of objects out of reach

y Within several hours: users can identify letters and numbers and can recognize landmarkinformation.

The device provides a new sensory language with which users learn to translate the impulse

patterns on the tongue to objects in space. Neuro imaging research suggests that using BrainPort

stimulates the visual regions of the brain in blind individuals.

At present, Brain Port is an investigational prototype and not commercially available. A

number of academic and research institutions have had or will have studies using the BrainPort

Vision Device with specific participation requirements. Contact Wicab for more information.

While Brain Port does not replace vision, it enhances the overall sensory experience and givesusers information on the size, shape, and location of objectsperception that will no doubt help

blind and visually impaired persons move with greater independence and lead fuller lives. In any

case, within 15 minutes of using the device, blind people can begin interpreting spatial

information via the Brain Port, says William Seiple, research director at the nonprofit vision

healthcare and research organization Lighthouse International. The electrodes spatially correlate

with the pixels so that if the camera detects light fixtures in the middle of a dark hallway,

electrical stimulations will occur along the center of the tongue.

"It becomes a task of learning, no different than learning to ride a bike," Arnoldussen says,

adding that the "process is similar to how a baby learns to see. Things may be strange at first,

but over time they become familiar."


11/14

11 | P a g e

1.5 RESOLUTION:-

The resolution of brain port camera varies according to the necessity. The images below

demonstrate how information from the video camera is represented on the tongue. Today's

prototypes have 400 to 600 points of information on a ~3cm x 3cm tongue display, presented at

approximately 30 frames per second, yielding an information rich image stream. Our research

suggests that the tongue is capable of resolving much higher resolution information and we are

currently working to develop the optimal tongue display hardware and software.


12/14

12 | P a g e

1.6 APPLICATIONS:

The Brain Port test results are somewhat astonishing and lead many to wonder about the

scope of applications for the technology

CURRENT APPLICATIONS:-

The current or foreseeable medical applications include:

y providing elements of sight for the visually impairedy providing sensory-motor training for stroke patientsy providing tactile information for a part of the body with nerve damagey alleviating balance problems, posture-stability problems and muscle rigidity in people

with balance disorders and Parkinson's disease

y enhancing the integration and interpretation of sensory information in autistic people

POTENIAL APPLICATIONS:

The Brain Port electrodes would receive input

from a sonar device to provide not only directional

cues but also a visual sense of obstacles and

terrain. Military-navigation applications could

extend to soldiers in the field when radio

communication is dangerous or impossible or

when their eyes, ears and hands are needed to

manage other things -- things that might blow up.

BrainPort may also provide expanded information for military pilots, such as a pulse on the

tongue to indicate approaching aircraft or to indicate that they must take immediate action. With

training, that pulse on their tongue could elicit a faster reaction time than a visual cue from a

light on the dashboard, since the visual cue must be processed by the retina before it's forwarded

to the brain for interpretation. Other potential Brain Port applications include robotic surgeries.

The surgeon would wear electro tactile gloves to receive tactile input from robotic probes inside

someone's chest cavity. In this way, the surgeon could feel what he's doing as he controls the

robotic equipment. Race car drivers might use a version of Brain Port to train their brains for


13/14

13 | P a g e

faster reaction times, and gamers might use electro tactile feedback gloves or controllers

tofeelwhat they're doing in a video game. A gaming Brain Port could also use a tactile-vision

process to let gamers perceive additional information that isn't displayed on the screen.

Brain Port is currently conducting a second round of clinical trials as it works its way

through the FDA approval process for the balance device. The company estimates a commercial

release in late 2006, with a roughly estimated selling price of $10,000 per unit. Brain Port

envisions itself even smaller and less obtrusive in the future. In the case of the balance device, all

of the electronics in the handheld part of the system might fit into a discreet mouthpiece.

1.7 DISADVANTAGES:-

The major disadvantage to the Brain Port is that you cant see and speak at the same time

with the electric array (lollipop) in your mouth. Whatever you do, dont drive and speak on thephone while using this device at the same time.

1.8 CONCLUSION:-

Brain port is indeed one of the finest and useful technologies. This article offers insights and

navigates the action about the pro and cons, of the brain port technology. Technology is a boon

in biomedical and can work for all the field like defense, sports, robotics, spy gadgets, and is able

to change the life of physically and mentally impaired persons. In comparison to biology,

human-machine interface technology is in its early infancy.


14/14

14 | P a g e

REFERENCES:-

Bach-y-Rita, Paul et al. "Form perception with a 49-point electrotactile stimulus array onthe tongue: A technical note." Journal of Rehabilitation Research and Development,

1998.

http://kaz.med.wisc.edu/Publications/1998-BachyRita-JRRD-Tongue.pdf

Blakeslee, Sandra. "New Tools to Help Patients Reclaim Damaged Senses." New YorkTimes, Nov. 23, 2004.

http://www.goupstate.com/apps/pbcs.dll/article?AID=/20041123/ZNYT05/411230391/10

51/NEWS01

Kaczmarek, Kurt, Ph.D. "Tongue Display Technology." University of Wisconsin, Aug.18, 2005.http://kaz.med.wisc.edu/Publicity/Synopsis.html

Kupers, Ron et al. "Activation of visual cortex by electrotactile stimulation of the tonguein early-blind subjects." Human Brain Mapping 2003.

http://208.164.121.55/hbm2003/abstract/abstract1557.htm

Manning, Joe. "Device may be new pathway to the brain." JS Online, Dec. 7, 2004.http://www.jsonline.com/story/index.aspx?id=282145

Phone interview with Kurt Kaczmarek, Ph.D., Senior Scientist, University of WisconsinDepartment of Orthopedics and Rehabilitation Medicine. July 7, 2006.

Ptito, Maurice et al. "Cross-modal plasticity revealed by electrotactile stimulation of thetongue in the congenitally blind." Brain, 2005.

http://brain.oxfordjournals.org/cgi/content/abstract/128/3/606

U.S. Patent #6,430,450. "Tongue placed tactile output device." Wicab, Inc.

http://www.wicab.com/

"Wicab to present BrainPort at Boston conference." WTN News. Oct. 4, 2005.http://wistechnology.com/printarticle.php?id=2319

Brain Port Vision Technology[1]

Documents

Transcript of Brain Port Vision Technology[1]