444 Sixth Sense Technology
-
Upload
ramesh-dev -
Category
Documents
-
view
215 -
download
0
Transcript of 444 Sixth Sense Technology
-
8/2/2019 444 Sixth Sense Technology
1/17
1
1.1 INTRODUCTION
Although miniaturized versions of computers help us to connect to the digital world
even while we are travelling there arent any device as of now which gives a direct link
between the digital world and our physical interaction with the real world. Usually the
informations are stored traditionally on a paper or a digital storage device. Sixth sense
technology helps to bridge this gap between tangible and non-tangible world. Sixth Sense
device is basically a wearable gestural interface that connects the physical world around us
with digital information and lets us use natural hand gestures to interact with this information.The sixth sense technology was developed by PranavMistry, a PhD student in the Fluid
Interfaces Group at the MIT Media Lab. According to him the sixth sense technology has a
Web 4.0 view of human and machine interactions.Sixth Sense integrates digital information
into the physical world and its objects, making the entire world your computer. It can turn
any surface into a touch-screen for computing, controlled by simple hand gestures. It is not a
technology which is aimed at changing human habits but causing computers and other
machines to adapt to human needs. It also supports multi user and multi touch provisions.
Sixth Sense device is a mini-projector coupled with a camera and a cell phonewhich acts as
the computer and your connection to the Cloud, all the information stored on the web. The
current prototype costs around $350. The Sixth Sense prototype is used to implement several
applications that have shown the usefulness, viability and flexibility of the system.
-
8/2/2019 444 Sixth Sense Technology
2/17
2
1.2 SIXTH SENSE DEVICES
'Sixth Sense' is a wearable gestural interface that augments the physical world around
us with digital information and lets us use natural hand gestures to interact with that
information the hardware components are coupled in a pendant like mobile wearable device.
The Sixth Sense prototype is comprised of a pocket projector, a mirror,colored marker and a
camera. The camera, mirror and projector is connected wirelessly to a blue tooth smart phone
device that can easily fit into the users pocket.A software then processes the data that is
collected by the capturing device and produces analysis.The software that is used in sixthsense device is open source type.
Fig. 1.1
1.2.1 Camera
It captures the image of the object in view and track the users hand gesture. The
camera recognizes individuals, images, pictures, gestures that user makes with his hand. The
camera then sends this data to a smart phone for processing. Basically the camera forms a
digital eye which connects to the world of digital information.
-
8/2/2019 444 Sixth Sense Technology
3/17
3
1.2.2 Colored Marker
There are color markers placed at the tip of users finger. Marking the users fingers
with red, yellow green and blue coloured tape helps the webcam to recognize the hand
gestures.The movements and arrangement of these markers are interpreted into gestures that
act as a interaction instruction for the projected application interfaces.
1.2.3 Mobile Component
The SixthSense device consists of a web enabled smart phone which process the data
send by the camera. The smart phone searches the web and interprets the hand gestures with
help of the coloured markers placed at the finger tips.
1.2.4 Projector
The information that is interpreted through the smart phone can be projected into any
surface. The projector projects the visual information enabling surfaces and physical objects
to be used as interfaces. The projector itself consists of a battery which have 3 hours of
battery life .A tiny LED projector displays the data sent from the smart phone on any surface
in view- object, wall or person. The downward facing projector projects the image on to a
mirror.
Using palm for dialling a number
Fig. 1.2
1.2.5 Mirror
The usage of a mirror is important as the projector dangles pointing downwards from
the neck. The mirror reflects the image on to a desire surface.Thus finally the digital image is
freed from its confines and placed in the physical world.
-
8/2/2019 444 Sixth Sense Technology
4/17
4
Fig. 1.3
-
8/2/2019 444 Sixth Sense Technology
5/17
5
1.3 Technologies that are related to Sixth Sense Devices
1.3.1 Augmented Reality
The augmented Reality is a visualization technology that allows the user to experience
the virtual experience added over real world in real time.With the help of advanced AR
technology the information about the surrounding real world of the user
becomes interactive and digitally usable. Artificial information about the environment and the
objects in it can be stored and retrieved as an information layer on top of the real world view.
When we compare the spectrum between virtual reality, which creates immersive, computer-
generated environments, and the real world, augmented reality is closer to the real world.
Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world as it
exists. Both video games and cell phones are driving the development of augmented reality .Theaugmented systems will also superimpose graphics for every perspective available andtry
adjust to every movement of the user's head and eyes.The three basic components of an
augmented reality system are the head-mounted display, tracking system and mobile
computer for the hardware. The main goal of this new technology is to merge these three
components into a highly portable unit much like a combination of a high tech Walkman and
an ordinary pair or eyeglasses.The head-mounted display used in augmented reality systems
will enable the user to view superimposed graphics and text created by the system.Another
component of an augmented reality system is its tracking and orientation system. This system
pinpoints the user's location in reference to his surroundings and additionally tracks the user's
eye and head movements.Augmented reality systems will need highly mobile computers. As
of now many computers arent there to satisfy to provide this option.
1.3.2 Gesture Recognition
It is a technology which is aimed at interpreting human gestures with the help of
mathematical algorithms. Gesture recognition technique basically focuses on the emotion
recognition from the face and hand gesture recognition. Gender recognition technique enables
humans to interact with computers in a more direct way without using any external
interfacing devices. It can provide a much better alternative to text user interfaces and
graphical user interface which requires the need of a keyboard or mouse to interact with the
computer. Interfaces which solely depends on the gestures requires precise hand pose
tracking. In the early versions of gesture recognition process special type of hand gloves
which provide information about hand position orientation and flux of the fingers. In the
SixthSense devices colored bands are used for this purpose. Once hand pose has been
captured the gestures can be recognized using different techniques. Neural network
approaches or statistical templates are the commonly used techniques used for the recognition
purposes. This technique have an high accuracy usually showing accuracy of more than 95%.Time dependent neural network will also be used for real time recognition of the gestures.
-
8/2/2019 444 Sixth Sense Technology
6/17
6
1.3.3 Computer Vision
Computer vision is the technology in which machines are able to interpret/extract
necessary information from an image. Computer vision technology includes various fields
like image processing, image analysis and machine vision. It includes certain aspect of
artificial intelligence techniques like pattern recognition. The machines which implement
computer vision techniques require image sensors which detect electromagnetic radiation
which are usually in the form of ultraviolet rays or light rays. The computer vision find itself
applicable in varies field of interest. One such field is bio medical image processing. Its also
used in autonomous vehicle like SUVs. The computer vision technique basically includes
four processes.
1. Recognition: One of the main task of computer vision technique is to determinewhether the particular object contain the useful data or not.
2. Motion Analysis: Motion analysis includes several tasks related to estimation ofmotion where an image sequence is processed continuously to detect the velocity at
each point of the image or in the 3D scene.
3. Scene Reconstruction: Computer vision technique employs several methods torecreate a 3D image from the available images of a scene.
4. Image Restoration: The main of aim of this step is to remove noise from an givenimage. The simplest method involves using simple filters like low pass or median
filters. In order to get better quality images more complex methods like Inpainting are
used.
1.3.4 Radio Frequency Identification
Radio frequency identification systems transmit the identity of an object wirelessly,
using radio magnetic waves. The main purpose of a radio frequency identification system is
to enable the transfer of a data via a portable device. The portable device is commonly known
as tag. The data send by the tag is received and processed by a reader according to the needsof the application. The data send by the tag contains various informations identification or
location of the information, or specifics about the product that has been tagged, for example
price, color, date of purchase, etc. This technology gained importance because of its ability to
track moving object. A typical radio frequency tag consists of a microchip attached to a radio
antenna which is mounted on a substrate. To retrieve the data from the tag a reader is needed.
A typical radio frequency reader consists of two antennas that emit radio waves and at the
same are capable of accepting the signals from the tag. The reader then passes the
information that it has received to a computer device in digital form. The computer then
interprets this digital data and processes it. Radio frequency identification techniques arewidely used in the fields like asset tracking, supply chain management, manufacturing,
-
8/2/2019 444 Sixth Sense Technology
7/17
7
payment systems etc. One of the major advantages of devices using radio frequency
technology over other similar devices is that RFID devices need not be positioned precisely
relative to the scanner. But till then there are certain areas of concern for this technology.
Some problem related to this technology is tag collision and reader collision. Usually the
reader collision occurs when the signals from two or more readers overlap, while tag collisionoccurs when many tags are present in a small area.
-
8/2/2019 444 Sixth Sense Technology
8/17
8
2.1 ADVANTAGES OF SIXTH SENSE DEVICES
2.1.1 Portable
One of the main advantages of the sixth sense devices is its small size and portability.
It can be easily carried around without any difficulty.The prototype of the sixth sense is
deigned in a such a way that it gives more importance to the portability factor. All the devices
are light in weight and the smart phone can easily fit into the users pocket.
2.1.2 Support multi touch and multi user interaction
Multi touch and multi user interaction is another added feature of the sixth sense
devices. Multi sensing technique allows the user to interact with system with more than one
than one finger at a time.. Sixth sense devices also in-cooperates multi user functionality.
This is typically useful for large interaction scenarios such as interactive table tops and walls.
2.1.3 Cost effective
The cost incurred for the construction of the sixth sense proto type is quiet low. It was
made from parts collected together from common devices. And a typical sixth sense device
cost up to $300. The sixth sense devices have not been made in large scale for commercial
purpose. Once that happens its almost certain that the device will cost much lower than the
current price.
2.1.4 Connectedness between real world and digital world
Forming a connection between the real world and the digital world was the main aim
of the sixth sense technology.
2.1.5 Data access directly from the machines in real time
With help of a sixth sense device the user can easily access data from any machine at
real time speed. The user doesnt require any machine-human interface to access the data.
The data access through recognition of hand gestures is much easier and user friendlier
compared to the text user interface or graphical user interface which requires keyboard or
mouse.
-
8/2/2019 444 Sixth Sense Technology
9/17
9
2.1.6 Mind map the idea anywhere
With the advent of the sixth sense device, requirement of a platform or a screen to
analyse and interpret the data has become obsolete. We can project the information into any
any surface and can work and manage the data as per our convenience.
.
2.1.7 Open source software
The software that is used to interpret and analysis the data collected bythe device is
made open source. This enables other developers to contribute to the development of the
system
-
8/2/2019 444 Sixth Sense Technology
10/17
10
2.2 APPLICATIONThe sixth sense technology finds a lot of application in the modern world. The sixth
sense devices bridge the gap by bringing the digital world into the real world and in that
process allowing the users toiu interact with the information without the help of any machine
interfaces. Prototypes of the sixth sense device have demonstrated viability, usefulness and
flexibility of this new technology. According to the words of its developers the extend of use
of this new device is only limited by the imagination of human beings. Some practical
applications of the sixth sense technology is given below
2.2.1 Map Application
With the help of a map application the user can call upon any map of his/her choice
and navigate through them by projecting the map on to any surface.By using the thumb and
index fingers movements the user can zoom in, zoom out or pan the selected map.
2.2.2 Taking Pictures
Another application of sixth sense devices is the implementation of a gestural camera.
This camera takes the photo of the location user is looking at by detecting the framing
gesture. After taking the desired number of photos we can project them onto any surfaces andthen use gestures to sort through those photos and organize and resize them.
The user taking making a pose gesture The camera takes the photo of the
location
Fig. 2.1. Fig. 2.2
-
8/2/2019 444 Sixth Sense Technology
11/17
11
2.2.3 Drawing Application
The drawing application allows the user you to draw on any surface by tracking the
fingertip movements of the users index finger. The pictures that are drawn by the user can be
stored and replaced on any other surface. The user can also shuffle through various pictures
and drawing by using the hand gesture movements
2.2.4 Making Calls
We can make calls with help of sixth sense device .The sixth sense device is used to
project the keypad into your palm and the using that virtual keypad we can make calls to
anyone.
2.2.5 Interacting with physical objects
The SixthSense system also helps to interact with physical objects we use in a better
way. It augments physical objects by projecting more information about these objects
projected on them. For example, a gesture of drawing a circle on the users wrist projects an
analog watch on the users hand. Similarly a newspaper can show live video news or dynamic
information can be provided on a regular piece of paper
Newspaper showing live feeds an analog watch projected into the wrist
Fig. 2.3
2.2.6 Getting Information
Sixth sense devices can be used for getting various information relating to our
everyday life by getting in contact with objects
2.2.7 Product Information:
Sixth sense technology uses marker technology or image recognition techniques to
recognize the objects we pick in our hand and then provide information relating to that
product.
-
8/2/2019 444 Sixth Sense Technology
12/17
12
2.2.8 Book Information:
By holding and shuffling through the book pages, the sixth sense provides Amazon
ratings on that book, other reviews and other relevant things related to the book.
2.2.9 Flight Updates:
With the help of the sixth sense technology it is no longer required to log into any
sites for checking the status of the flights. The system will recognize your boarding pass and
let you know whether the flight is on time or not.
2.2.10 Information About People:
With help of face recognition techniques the sixth sense devices are able to provideinformation about the people when we meet them. The sensor detects the face and checks the
data base for the relevant information.The system will then project the relevant information
about a person like what they do, where they work,
Boarding pass showing current status The information about a person being
projected
Fig. 2.4 Fig. 2.5
-
8/2/2019 444 Sixth Sense Technology
13/17
13
2.3 WAY FORWARD
Upcoming Projects Making Use of the Sixth Sense Technology
2.3.1 Mouse less:
Mouse less is a concept which is developed in order to avoid the usage of mouse in
computers. It helps to provide a familiarity of the physical mouse without actually requiring a
physical mouse.
Fig. 2.6
Even though the computer hardware technology has been developed to a greater extent, still
there has been no proper substitute for a physical mouse. This new invention aims to remove
the requirement of having a physical mouse altogether but still it provides the feeling of a
physical mouse that users are familiar with.It basically consists of an Infrared laser beam and
an Infrared camera. Both laser and camera are embedded together with the computer. The
laser beam module is modified with a line cap and placed such that it creates a plane of IR
laser just above the surface the computer sits on. The user cups their hand, as if a physical
mouse was present underneath, and the laser beam lights up the hand which is in contact with
the surface. The IR camera detects those bright IR blobs using computer vision. The change
in the position and arrangements of these blobs are interpreted as mouse cursor movement
and mouse clicks.
2.3.2 Third Eye:
Third Eye is a new technique that enables multiple viewers to see different things on
a same display screen at the same time.
-
8/2/2019 444 Sixth Sense Technology
14/17
14
Fig. 2.7
With thirdEye, - We can have a public sign board where a Japanese tourist sees all the
instructions in Japanese and an American in English. - We don't need to have the split screen
in games now. Each player can see his/her personal view of the game on the TV screen. -
Two people watching TV can watch their favourite channel on a single TV screen. - A public
display can show secret messages or patterns. - In the same movie theatre, people can see
different end of a suspense movie.
2.3.3 Inktuitive:
Even though there has been a tremendous development of computer aided
designing tools, still paper and pencil is the preferred by most of the designers,
especially in the earlier stages of designing
Fig. 2.8
The project inktuitive is aimed to combine the process of creation that is inherent in paper
and pencil with the power of computing that the digital design tools provide. It also extends
the natural work-practice of using physical paper by giving the pen the ability to control the
design in physical three-dimensional space, freeing it from its tie to the paper. There always
remains an intuition of pen and paper ,but the lines that have been drawn are be captured and
translated into shapes in the digital world. The physical paper is augmented with overlaid
digital strokes. The platform provides a novel interaction mechanism for drawing and
designing using above the surface pen movements. In other words inktuitive can be
-
8/2/2019 444 Sixth Sense Technology
15/17
15
considered as an intuitive physical design workspace that aims to bridge the gap and bring
together the conventional design tools such as paper and pencil with the power and
convenience of the digital tools for design.
2.3.4 TaPuma:
TaPuma is the short form of Tangible Public Map. It is another attempt to bring the
digital world nearer to the real world by exploring the techniques of sixth sense technology.
Key word based search mechanism has vastly influenced and modified the ways in which we
search informations. The keyword based search allows the user to filter out the required data
from the vast data that are available on search engines such as yahoo and Google. TaPuma
allows people to use their own belongings, the objects they usually carry with them to accessrelevant and just-in-time information and locations of places from a public map. The project
also explores and analyzes the advantages and challenges of this novel interaction
mechanism, where real life objects serve as the interface for information acquirement. The
broad concept behind the project Ta Puma is Object Amelioration, where the functions of
everyday objects can be expanded by using their affordances or functionalities in a variety of
different contexts.
-
8/2/2019 444 Sixth Sense Technology
16/17
16
3.1 CONCLUSION
The sixth sense technology and devices are definitely the way forward. The sixth
sense recognizes the objects around us and displays the information relating to those objects
in a real time environment. The sixth sense technology allows the user to interact the
information through hand gestures. This is a quiet efficient way compared to the text and
graphic based user interface. It has the potential to form the transparent user interface for
accessing the information around us. . Even though the sixth sense devices are currently in
development stages and have not been used in large now, its quiet predictable that this
technology have the potential to revolutionise the way in which we interacts with the digital
world. As MrPranavMistry says imagination of the human brain is the only thing that could
limit the application of the sixth sense technology
-
8/2/2019 444 Sixth Sense Technology
17/17
17
3.2 REFERNCES
http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense_technology.htm
http://www.commoncraft.com/augmented-reality-video http://www.arcane-technologies.com/en/ http://en.wikipedia.org/wiki/Augmented_reality http://www.rfidjournal.com/ http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html http://timesofindia.indiatimes.com/Health__Science/Sixth_sense_gizmo_a_sci-
fi_reality/articleshow/4083978.cms
http://www.gesturetek.com/