Seventh sense by Sahal Hash
-
Upload
sahal-hashim -
Category
Engineering
-
view
93 -
download
0
Transcript of Seventh sense by Sahal Hash
05/02/2023 1
Seventh Sense Technology
Under the guidance of:Mr. Mohammed SaifuddeenAsst. ProfessorPACE Mangaluru
Presented by :Sahal Hashim P.USN: 4PA11EC091Branch : Electronics & Communication
05/02/2023 2
Contents :-INTRODUCTION : - What is Sixth Sense What is Seventh Sense PROBLEM STATEMENT :-• Need for Sixth/Seventh SenseLITERATURE SURVEY :-• Overview of Sixth Sense Technology• Flaws in Sixth SensePROPOSED METHOD :-RESULTCONCLUSION & FUTURE SCOPEREFERENCES
05/02/2023 3
What is Sixth Sense.?• Sixth Sense is a technology that augments the physical world around us with digital
information and lets us use natural hand gestures to interact with that information.
• It was developed by Pranav Mistry, a PhD student in the Fluid Interfaces Group at MIT Media Lab.
• It is so called Wear Your World (WUR) Device that uses Natural hand gestures to interact with the digital world.
• Sixth Sense comprises a mobile device, pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device
05/02/2023 4
What is Seventh Sense.?• Seventh Sense Technology is a technology used for the human
electronic interaction using natural hand gestures• The hand gestures are used to control any micro controller based
devices or robots at distant places using Zigbee or GSM Module or Satellite transceiver• Seventh Sense Technology uses the Sixth Sense Technology and its
advancements to control the autonomous robots or any microcontroller based devices on human will.
05/02/2023 5
Need for Sixth/Seventh Sense.• The bridge gap between the Physical and digital world which needs to
be overcomed.
• Much simpler and cost efficient approach is needed to interact with the digital world.
• The world of Robotics is limited at present and it needs to be explored by making them Smarter, Intelligent and Human friendly.
• Bulky gadgets needs to get smaller yet providing Broad Functions.
05/02/2023 6
Overview of Sixth Sense TechnologySixth Sense is supported by techniques such as..
Gesture recognition
Computer vision
Radio Frequency Identification
Augmented reality
05/02/2023 7
Device used in Sixth Sense TechnologyThe hardware components are coupled in a pendant like mobile wearable device.
Camera
Projector
Mirror
Mobile Component
Colored Markers
05/02/2023 8
05/02/2023 9
Flaws in Sixth Sense • Sixth Sense is human oriented and cannot be implied at a particular
location without his physical presence.• The present Technology cannot control any kind of Robots or
microcontrollers.• The person in charge of sixth sense device can only access the
Information around him.• Gestures cannot be recognized with bare hands and the user needs
the colored markers at all time.• The main flaw of the Sixth Sense algorithm is the movement of
objects with same color of the marker in the background.
05/02/2023 10
Proposed Method • Seventh Sense is focused on forecasting information and making an
alteration somewhere he is not virtually present.
• Seventh Sense technology is built upon Robotics and Advanced Sixth Sense Technology on the basis of Computer Vision.
• It comprises of a projector and a simple camera at the users place and remotely placed Robot or Nano robot or any micro controlled device.
• We can control the activities of the robot or micro controlled devices along with a help of camera attached to the robot or device.
05/02/2023 11
Seventh Sense Flow Diagram
05/02/2023 12
Design of Seventh Sense TechnologyDesign of the seventh sense can be broadly classified into three
components.
1. Seventh sense technology based hardware2. Second component is related to Computer vision and Artificial
neural networks.3. Robotics
05/02/2023 13
HARDWARE COMPONENTS OF SEVENTH SENSE TECHNOLOGY
• Camera• Projector• Mobile Computing Device• Micro controlled Device or ROBOTS with Wireless Camera• ZIGBEE, GSM or Satellite Transceiver Module
05/02/2023 14
Working of an Autonomous Robot
05/02/2023 15
SOFTWARE IMPLENENTATION OF SEVENTH SENSE TECHNOLOGY • To develop an application based on seventh sense technology one can
use:
1. Languages used are Java, OpenCV, JavaCV. 2. Image processing Software used is Matlab 3. Embedded programming in Embedded C
05/02/2023 16
SYSTEM IMPLEMENTATION• Seventh Sense Technology is based upon mainly two types of gesture
recognition
1. Gesture recognition with markers 2. Gesture recognition with bare hand (without markers)
05/02/2023 17
Hand gesture recognition process execution
05/02/2023 18
METHODS OF IMPLEMENTATION Gesture Recognition with Markers/Bare Hand
• Capture every single frame from the video. • Process each frame obtained. • Get the two red/blue channels from the frame by setting threshold. • Subtract the gray scale image from the channel. • Convert the subtracted image to binary image. • Find bounding box of definite height and definite distance from each
other.
05/02/2023 19
For Bare Hand Gesture Recognition• The frame obtained from the video is first captured and converted to
YCC and applied threshold for skin color and then the Image is converted back to RGB and finally to the binary image.
05/02/2023 20
Different stages of background subtraction
A Real time video after Background Subtraction.
05/02/2023 21
Pattern Recognitionwe use back ground subtraction followed by canny edge detection and Scale Invariant Feature Transform (SIFT) process to store each gestural actions
Output obtained after Canny edge detection applied on background subtraction output Hand gesture training process using SIFT
05/02/2023 22
Algorithms Applied• Threshold setting code for
Background Subtraction
threshold=25for i=1:width for j=1:height if Fg(i,j)>threshold Fg(i,j)=255; else Fg(i,j)=0;
• Threshold setting for Canny Edge Detection
I_max=max(max(NVI));I_min=min(min(NVI));level=alfa*(I_max-I_min)+I_min;subplot(3,2,5);Ibw=max(NVI,level.*ones(size(NVI)));imagesc(Ibw);title('After Thresholding');
05/02/2023 23
Result• The Combination of Sixth Sense technology with Robotics were tested
and the autonomous robots were able to act more intelligent and were more human friendly providing all the information needed by its operator with just few hand gestures.
05/02/2023 24
Conclusion and Future Scope• Seventh Sense is more likely installing a digital system (computer) into
our body and making it as seventh sense of our body. • It allows us to interact with the information via natural hand gestures
and move robots or small devices at our will.• This Seventh sense can be extended to Nano robotics and can find
different diseases and cure at will, especially cancer.• This technology can be further developed to work as a Fifth Sense for
disabled person.
05/02/2023 25
References• M. K. Saha and S. Hore, “Sixth Sense Technology: A Brief Literary Survey,” Proceedings of International Journal of Engineering Research
Technology (IJERT), Vol. 2 Issue 12, December 2013. • V. J. Oniga, S. and I. Orha, “Intelligent human-machine interface using hand gestures recognition,” In the proceedings of the IEEE Trans.
On Automation Quality and Testing Robotics (AQTR), pp. 559 - 563, 24-27 May, 2012. • I.-L. Jung, N. Akatyev, and Won-Dong Jang, “Touch less user interface based on marker detection and tracking for real-time mobile
applications,” In the proceedings of the International Journal of Innovative Computing(ICIC), Information and Control, Vol. 9 Issue 2, February- 2013.
• www.cs.cmu.edu/cil/vision.html • R. Z. Khan and N. A. Ibraheem, “Hand gesture recognition: A literature review,” In the proceedings of International Journal of Artificial
Intelligence Applications (IJAIA), Vol.3, No.4, July 2012. • H. Park, “A Method for Controlling Mouse Movement Using a Real –Time Camera,” 2008. • Robertson P., Laddaga R., Van Kleek M., “Virtual mouse vision based interface”, In the Proceedings of the nineth U international
conference on intelligent user interfaces, pp. 177 U 183, January 2004. • H. D. Nasser, “Hand gesture interaction with a 3d virtual environment,” The Research Bulletin of Jorden, ACM , ISSN: 2078 U 7952 , Vol
II(III), Page - 86. • T. Kirishima, K. Sato, and K. chihara, “Real -Time Gesture Recognition by Learning Selective Control of Visual Interest Point,” IEEE Trans.
on Pattern Analysis and Machine Intelligence, Vol. 27, No. 3, pp. 351- 364, March 2005. • M. Fiala and C. Shu, “3d Model creation using self-identifying markers and sift keypoints,” In the proceedings of the IEEE Trans. on
Haptic Audio Visual Environments and their Applications, pp. • 118 - 123, 2006.
05/02/2023 26
• www.learnartificialneuralnetworks.com • Murthy and G.R.S., “Hand gesture recognition using neural networks”, In the Proceedings of IEEE Trans. on Advance Computing
Conference(IACC), pp. 134 - 138, 19-20 • February,2010. • Yubing Dong, Mingjing Li and Jie Li., “Image retrieval based on improved Canny edge detection algorithm”, In the Proceedings of the
International conference on Mechatronic Sciences, Electric Engineering and Computer (MEC), pp. 1453 - 1457, 2013. • Ghosh D.K. and Ari S., “A static hand gesture recognition algorithm using k-mean based radial basis function neural network”, In the
Proceedings of 8th IEEE Trans. on Information, Communications and Signal Processing (ICICS), pp. 1-5, 2011. • Kumar, S.P. and Pandithurai, O., “Sixth sense technology”, In the Proceedings of IEEE Trans. on Information Communication and Embedded
Systems (ICICES), pp. 947 - 953, 2013. • Bhowmick S., Kumar S. and Kumar A., “Hand gesture recognition of English alphabets using artificial neural network”, In the Proceedings of
IEEE Trans. on Recent Trends in Information Systems (ReTIS), pp. 405 - 410, 2015. • Hsien-I Lin, Ming-Hsiang Hsu and Wei-Kai Chen, “Human hand gesture recognition using a convolution neural network”, In the Proceedings of
IEEE Trans. on Automation Science and Engineering (CASE), pp. 1038 - 1043, 2014. • Chang Tan, Xiao and Nanfeng, “Improved RCE neural network and its application in human-robot interaction based on hand gesture
recognition”, In the Proceedings of 2nd IEEE Trans. on Information Science and Engineering (ICISE), pp. 1260 - 1263, 2010. • Jalab, H.A. and Omer, H.K., “Human computer interface using hand gesture recognition based on neural network”, In the Proceedings of 5th
National Symposium on Information Technology: Towards New Smart World (NSITNSW), pp. 1 - 6, 2015. • Canny, John., “A Computational Approach to Edge Detection”, In the Proceedings of IEEE Trans. on Pattern Analysis and Machine Intelligence,
pp. 679 – 698,1986. • https://in.mathworks.com/discovery/edge-detection.html• GitHub profile adjecon (https://github.com/ajdecon/gradschool_matlab/blob/4d1c9d247021019b0a0d229112f8e90c6cf96564/bpass.m)• Li Cheng, M. Gong, D. Schuurmans, and T. Caelli. Real-time Discriminative Background Subtraction. IEEE Trans. Image Processing, 20(5), 1401-
1414, 2011(http://web.bii.a-star.edu.sg/~chengli/BkgSbt.htm)• Wikipedia on YCC (https://en.wikipedia.org/wiki/YCbCr)
05/02/2023 27
Credits:-
Pranav Mistry Sidharth Rajeev