The Sixth Sense Technology Fri

7/27/2019 The Sixth Sense Technology Fri

1/21

A REPORT ON THE SIXTH

SENSE TECHNOLOGY

Submitted by:

Friday, 15 February 2012

AKHILESH YADAV

ECE-C

1002731008


2/21

THE SIXTH SENSE TECHNOLOGY:

Abstract Sixth Sense is a wearable gestural

interface that augmentsthe physical world around uswith digital information and lets us usenatural handgestures to interact with that information. Steve Mann isconsidered as the father of Sixth Sense Technology whomadewearable computer in 1990. He implemented theSixth SenseTechnology as the neck worn projector witha camera system (whichMann originally referred to asSynthetic Synesthesia of the Sixth

Sense) . He was a media lab student at that time. Thenhis work was carried forward by Pranav Mistry (Ph.Dstudent in the Fluid Interfaces Group at the MIT MediaLab). By using a camera and atiny projector, SixthSensesees what you see and visually augmentsany surfaces

or objects we are interacting with.It projects information onto surfaces, walls andphysical objects around us, andlets us interact with theprojected information through natural handgestures, armmovements, or our interaction with the object itself.This wearable gestural interface attempts to freeinformation from itsconfines by seamlessly integrating it

with reality, and thus makingthe entire world yourcomputer. To a layman, it would be


3/21

somethingsupernatural. Some might just consider it as asuperstition orsomething psychological. But theinvention of Sixth SenseTechnology has completely

shocked the world. Although it is notwidely known as ofnow but the time is not far when this technology willchange our perception of the world.

INTRODUCTION:We've evolved over millions of years to sense the

world around us.When we encounter something,

someone or some place, we use ourfive natural sensesto perceive information about it; that informationhelps usmakedecisions and chose the right actions to take.Butarguably the most useful information that can help usmake the rightdecision is not naturally perceivable withour five senses, namely thedata, information andknowledge that mankind has accumulatedabout

everything and which is increasingly all available online.Although the miniaturization of computing devices allowsus to carrycomputers in our pockets, keeping uscontinually connected to thedigital world, there is no linkbetween our digital devices and ourinteractions with thephysical world. Information is confinedtraditionally onpaper or digitally on a screen.

Sixth Sense bridges this gap, bringing intangible,digital information out into the tangibleworld, andallowing us to interact with this information vianaturalhand gestures. Sixth Sense Technology is thenewest jargon that hasproclaimed its presence in thetechnical arena. This technology hasemerged, whichhas its relation to the power of these six senses.Ourordinary computers will soon be able to sense the

different feelingsaccumulated in the surroundings and it


4/21

is all a gift of the SixthSense Technology newlyintroduced. Sixth Sense will allow us tointeract with ourworld like never before. We can get information

onanything we want from anywhere within a fewmoments! We willnot only be able to interact with thingson a whole new level but alsowith the people! One greatpart of the device is its ability to scanobjects or evenpeople and project out information regarding whatyouare looking at.

Sixth Sense Technology is a perception of augmented

realityconcept. Sixth Sense is in fact,aboutcomprehending informationmore than our availablesenses. And today there is not just thisphysical worldfrom where we get information but also the digitalworldwhich has become a part of our life.

This digital world is nowas important to us as thisphysical world. And with the internet thedigital world canbe expanded many times the physical world. Godhasntgiven us sense to interact with the digital world so wehave created them like smartphones, tablets,computers, laptops, net book sand other gadgets. Thesegadgets enable us to communicate with thedigital worldaround us.But we are humans and our physical bodyisnt meant for digitalworld so we cant interact directly tothe digital world. For instance we press keys to dial a

number; we type text to search it and so on.This meansfor an individual to communicate with the digitalworldhe/she must learn it. We dont communicatedirectly and efficientlyto the digital world as we do withthe real world. The Sixth SenseTechnology is all aboutinteracting to the digital world in mostefficiency anddirect world.


5/21

COMPONENTS:

The devices which are used in Sixth Sense Technologyare:

1. Camera.2. Coloured Marker.3. Mobile Component.

4. Projector.5. Mirror.1.Camera:


6/21

It captures the image of the object in view and tracks

the users handgesture. The camera recognizesindividuals, images, pictures,gestures that user makeswith his hand. The camera then sends thisdata to asmartphone for processing. Basically the camera forms

adigital eye which connects to the world of digitalinformation.

2. Coloured Marker:


7/21

There are colour markers placed at the tip of users

fingers. Markingthe users fingers with red, yellow, greenand blue coloured tapehelps the webcam to recognizethe hand gestures. The movementsand arrangement ofthese markers are interpreted into gestures that act as a

interaction instruction for the projectedapplicationinterfaces.

3. Mobile Component:

The Sixth Sense device consists of a web enabledsmartphone whichprocess the data send by the camera.


8/21

The smartphone searches the web and interprets thehand gestures with the help of the colouredmarkersplaced at the finger tips. Basic processing works

oncomputer vision algorithms where approx. 50,000lines of code areused written in Symbian C++.

4. Projector:

The information that is interpreted through thesmartphone can beprojected into any surface. Theprojector projects the visual information enabling

surfaces and physical objects to be used as interfaces.The projector itself consists of a battery which have3hours of battery life. A Tiny LED projector displays thedata sentfrom the smartphone on any surface in view-object, wall or person.The downward facing projectorprojects the image on to a mirror.

5. Mirror:


9/21

The usage of a mirror is important as the projectordangles pointingdownward from the neck. The mirrorreflects the image on to a desiresurface. Thus finally the

digital image is freed from its confines andplaced in thephysical world.

KINDS OF GESTURES RECOGNIZED:

The software recognizes three kinds of gestures:1.Multi-Touch Gestures:

Like the ones we see in the iphone-where we touch thescreen andmake the map move by pinching anddragging.2. Freehand Gestures:Like when you take a picture or Namaste gesture to starttheprojection on the wall.3. Iconic Gestures:

Drawing an icon in the air. Like, whenever we draw astar, shows usthe weather details. When we draw amagnifying glass, shows us themap.

WORKINGThe Sixth Sense Technology works as follows:


10/21

1. It captures the image of the object in view and trackthe users handgestures.2. There are colour markers placed at the tip of usersfinger. Markingthe users fingers with red, yellow, greenand blue coloured tapehelps the webcam to recognizethe hand gestures. The movements and arrrangement of

these markers are interpreted into gestures that act as ainteraction instruction for the projected applicationinterfaces.3. The smartphone searches the web and interprets thehand gestureswith the help of the coloured markersplaced at the finger tips.4. The information that is interpreted through the

smartphone can beprojected into any surface.


11/21

5. The mirror reflects the image on to a desired surface.

APPLICATIONS:

The Sixth Sense Technology finds a lot ofapplications in the modernworld. The Sixth Sensedevices bridge the gap by bringing thedigital world intothe real world and in the process allowing the userstointeract with the information without the help of anymachineinterfaces. Prototypes of the Sixth Sense devicehave demonstrated

viability, usefulness and flexibility of this new technology.Acording to the words of its developers the extend ofuse of thisnew device is only limited by the imaginationof human beings. Some practical aplications of the SixthSense Technology is given below:


12/21

1. Viewing Map:With the help of a map application the user can call

upon any map ofhis/her choice and navigate through

them by projecting the map on toany surface. By usingthe thumb and index fingers movements theuser canzoom in, zoom out or pan the selected map.

2. Taking Pictures:Another application of Sixth Sense devices is the

implementation ofa gestural camera. This camera takes

the photo of the location user islooking at by detectingthe framing gesture. After taking the desirednumber ofphotos we can project them onto any surfaces and thenusegestures to sort through those photos and organizeand resize them.

3. Drawing Application:The drawing application allows the user you to draw

on any surfaceby tracking the fingertip movements ofthe users index finger. Thepictures that are drawn bythe user can be stored and replaced on anyothersurface. The user can also shuffle through variouspictures anddrawing by using the hand gesturemovements.

4. Making Calls:

We can make calls with the help of Sixth Sensedevice. The SixthSense device is used to protect thekeyboard into your palm andusing that virtual keypad wecan make calls to anyone.

5. Interacting with physical objects:


13/21

The Sixth Sense system also helps to interact withphysical objectswe use in a better way. It augmentsphysical objects by projectingmore information about

these objects projected on them. Forexample, a gestureof drawing a circle on the users wrist projects ananolog watch on the users hand. Similarly a newspapercan showlive video news or dynamic information can beprovided on aregular piece of paper.

6. Getting Information:

Sixth Sense devices can be used for getting variousinformationrelating to our everyday life by getting incontact with objects.(a)Product information:

Sixth Sense Technology uses marker technology orimagerecognition techniques to recognize the objetcswe pick in our handand then provide information realtingto the product.(b) Book Information:

By holding and shuffling through the book pages, theSixth Senseprovides Amazon ratings on that book, otherreviews and otherrelevant things related to the book.(c) Flight Updates:

With the help of the Sixth Sense Technology it is nolonger requiredto log into any sites for checking the

status of the flights. The systemwill recognize yourboarding pass and let you know whether theflight is ontime or not.

7. Sixth Sense also lets the user draw icons or symbolsin the airusing the movement of the index finger andrecognizes thosesymbols as interaction instructions.


14/21

For example, drawing a magnifying glass symboltakes the user tothe map application or drawing an@symbol lets the user check hismail.

ADVANTAGES:

1.Portable:One of the main advantages of the Sixth Sense

devices is its smallsize and portability. It can be easilycarried around without anydifficulty. The prototype of the

Sixth Sense is designed in such a waythat it gives moreimportance to the portability factor. All the devices arelight in weight and the smartphone can easily fit into theuserspocket.

2. Support Multi touch and Multi user interaction:Multi touch and Multi user interaction is another

added feature ofthe Sixth Sense devices. Multi sensingtechnique allows the user tointeract with system withmore than one finger at a time. Sixth Sensedevices alsoin-corporates Multi user functionality. This is typicallyuseful for large interaction scenarios such as interactivetable topsand walls.

3. Cost Effective:The cost incurred for the construction of the Sixth

Sense prototype isquite low. It was made from partscollected together from commondevices. And a typicalSixth Sense device costs upto $300. The SixthSensedevices have not been made in large scale forcommercial purpose. Once that happens its almostcertain that the device willcost much lower than thecurrent price.


15/21

4. Data access directly from the machines in realtime:

With the help of a Sixth Sense device the user caneasily access datafrom any machine at real time speed.The user doesnt require anymachine-human interfaceto access the data. The data access throughrecognitionof hand gestures is much easier and user friendliercompared to the text user interface or graphical userinterface whichrequires keyboard or mouse.

5. Mind map the idea anywhere:With the advent of the Sixth Sense device,

requirement of a platformor a screen to analyze andinterpret the data has become obsolete. Wecan projectthe information onto any surface and can work andmanage the data as per our convenience.

6. Open Source Software:The software that is used to interpret and analysis the

data collectedby the device is going to be made opensource as said by its inventor.This will enable other developers to contribute to thedevelopment ofthe system.

CONCERNS:There is a health issue regarding Sixth Senses

projection technology.When the device is projecting on ahard surface, it is not privateenough just for the user.People around him can see the projection that is verydetailed.

Projection is better in the night time and dark areas

rather than mornings and bright areas. This is an issue


16/21


17/21

gestures can berecognised using different techniques.Neutral network approaches orstatistical templates arethe common techniques used for therecognition

purposes. This technique have an accuracy of morethan95%. Time dependent neural network will also beused for real timerecognition of the gestures.

Augmented reality:The Augmented reality is a visualiztation technique

that allows theuser to experience the virtual experience

added over real world inreal time. With the help ofadvanced AR technology the informationabout thesurrounding real world of the user becomes interactiveanddigitally usable. Artificial information about theenvironment and the objects in it can be stored andretrieved as an information layer ontop of the real worldview.

When we compare the spectrum between virtualreality, which creates immersive, computer-generatedenvironments, and the real world, augmented reality iscloser to thereal world. Augmented reality addsgraphics, sounds, haptic feedbackand smell to thenatural world as it exists. Both video games and cellphones are driving the development of augmentedrealtiy. Theaugmented systems will also superimpose

graphics for everyperspective available and try adjust toevery movement of the usershead and eyes.

The three basic components of an augmentedrealitysystem are the head mounted display, trackingsystem and mobilecomputer for the hardware. The maingoal of this technology is tomerge these threecomponents into a highly portable unit. The

headmounted display used in augmented reality


18/21

systems will enable theuser to view superimposedgraphics and text created by the system.

Another component of augmented reality system is its

tracking andorientation system. This system pinpointsthe users location inreference to his surrooundings andadditionally tracks the users eyeand head movements.

Computer vision:Computer vision is the technology in which machines

are able tointerpret/extract necessary information from

an image. Computervision technology includes variousfields like image processing,image analysis andmachine vision. It includes certain aspect ofartificialintelligence techniques like pattern recognition.

The machines which implement computer visiontechniques requiresimage sensors which detectelectromagnetic radiation which areusually in the formultraviolet rays or light rays. The computer vision finditself applicable in various fields of interest. One suchfield isbio medical image processing. Its also used inautonomous vehicleslike SUVs. The computer visiontechnique basically includes fourprocesses.

1.Recogniton: One of the main task of computer visiontechnique isto determine whether the particular object

contain the useful data ornot.

2. Motion Analysis: Motion analysis includes severaltasks related toestimation of motion where an imagesequence is processedcontinuously to detect thevelocity at each point of the image the 3D scene.


19/21

3. Scene Reconstruction: Computer vision techniqueemploys severalmethods to recreate a 3D image fromthe available images of a scene.

4. Image Restoration: The main aim of this step is toremove noisefrom a given image. The simplest methodinvolves using simplefilters like low pass or medianfilters. In order to get better qualityimages more complexmethods like Inpainting are used.

Radio Frequency Identification:Radio frequency identification systems transmit theidentity of anobject wirelessly, using radio magneticwaves. The main purpose of aradio frequencyidentification system is to enable the transfer of a datavia a poratble device. The portable device is commonlyknownas a tag.

The data send by the tag is received and processedby areader according to the needs of the application.The data send sendby the tag cotains varioousinformations identification or location ofthe information,or specifies aboout the product that has been tagged,forexample price, colour, date of purchase,etc. Thistechnology gained importance because of its ability totrack moving object.

A typical radio frequency tag consists of a microchipattached to a radioantenna which is mounted on asubstrate. To retrieve data from thetag a reader isneeded. A typical radio frequency reader consists oftwoantennas that emit radio waves and at the same timeare capableof accepting the signals from the tag. Thereader then passes theinformation that it has received to


20/21

a computer device in digitalformat. The computer theninterprets this digital data and processes it.

Radio frequency identification techniques are widely

used in thefields like asset tracking, supply chainmanagement, manufacturing, payment systems, etc.One of the major advantages of devices using radiofrequency technology over other similar devices is thatRFIDdevices need not be positioned precisely relative tothe scanner. Buttill then there are certain areas ofconcern for this technology.

Some problem related to this technology is tagcollision and readercollision. Usually the reader collisionoccurs when the signals fromthe signals from two ormore reader overlap, while tag collisionoccurs whenmany tags are present in a small area.

FUTURE ENHANCEMENTS:

1.To get rid of colour markers.2. To incorporate camera and projector inside mobilecomputingdevice.3. To have 3D gesture tracking.4. To make Sixth Sense work as fifth Sense for disabledperson.

CONCLUSION:Information is often confined to paper or computer

screens. SixthSense frees data from these confines andseamlessly integratesinformation and reality. With theminiaturization of computing devices,we are alwaysconnected to the digital world, but there is no linkbetween our interactions with these digital devices and

ourinteractions with the physical world. Sixth Sense


21/21

bridges this gap byaugmenting the physical world withdigital information, bringingintangible information intotangible world.

Sixth Sense recognizes the objects around us,displaying information automatically and letting usaccess it in any way weneed. The Sixth Sense prototypeimplements several applications that Demonstrate theusefulness, viability and flexibilty of the system.Thepotential of becoming the ultimate transparent userinterface foraccessing information about everything

around us is present in thisTechnology.Using this technology Finally one gets a taste of

theworld of Harry Potter. Sixth Sense Technology is thescience oftomorrow with the aim of connecting thephysical world seamlessly, eliminating hardwaredevices.

The Sixth Sense Technology Fri

Documents

Transcript of The Sixth Sense Technology Fri