Skinput

Skinput

SKINPUT

Seminar Report submitted in partial fulfilment of the

Requirement for the degree of

Bachelor of Technology

In

Computer science & engineeringUnder the supervision of

Mr. Vipin Rai

Ms. Kritika Goel

By

HIMANSHU SINGH SAJWAN

To

Department of Computer Science & EngineeringIIMT COLLEGE OF ENGINEERING, GR. NOIDA

Utter Pradesh Technical University,

Lucknow

Session: 2014-15

1 IIMT College of Engineering

Skinput

DECLARATION

This is to certify that Report entitled “SKINPUT ” submitted by Himanshu Singh Sajwan

which is submitted by me in partial fulfillment of the requirement for the award of degree

B.Tech. in Computer Sc. & Engineering/Information Technology to Deptt of Computer Sc &

Engg,IIMT College of Engg,Gr Noida, Uttar Pradesh Technical University, Lucknow

comprises only my own work and due acknowledgement has been made in the text to all other material used.

Date: 20-04-2015 Name of Student : Himanshu Singh Sajwan

Approved By : Head Of Department, CSE, IIMT, Gr Noida


Skinput

ACKNOWLEDGEMENT

At the very outset, I take this opportunity to convey my heartfelt gratitude to those persons

whose co-operation, suggestions and support helped me to accomplish the project successfully.

I take immense pleasure to express my sincere thanks and profound gratitude to our respected

Dr. Prabhat Kr. Vishwakarma, H.O.D. and Mr. Vipin Rai, Mrs. Kirtika Goel, Department of

Computer Sciences Engineering, IIMT College of Engg., Gr. Noida for his kind co-operation and

able guidance, valuable suggestions and encouragement he rendered for completing the Seminar

topic.

I express my sincere thanks to all the faculty members of the Department of Computer Sciences

Engineering, for providing the encouragement and environment for the success of my topic.

In the end, I would be failing in my duties if I do not express my heartfelt gratitude to my family

whose constant inspiration and patience have helped me to complete this work. And last but not

the least I would like to thank God for all he has given me till today.


Skinput

CERTIFICATE

This is to certify that Report entitled “Modern Operating System” which is

submitted by Karan Panjwani in partial fulfillment of the requirement for the

award of degree B.Tech. in Computer Sc. & Engineering in IIMT College of

Engg,Gr Noida is a record of the candidate own work carried out by him under

my/our supervision. The matter embodied in this work is original and has not been

submitted for the award of any other degree.

Date: 20-04-2015

Mr. Vipin RaiMs. Kirtika Goel ------------------------------- Dr. Prabhat Kr. Vishwakarma

Seminar Guide H.O.D of CSE Dept.


Skinput

INDEX

1.INTRODUCTION…………………………………………………………….. ….7

2.WHAT IS SKINPUT?..............................................................................................8

2.1 Always available input…………………………………………………….9

2.2 Bio-Sensing………………………………………………………….…….10

2.3 Principles of Skinput……………………………………………………..12

3. TECHNOLOGIES IN SKINPUT……………………………………………… 13

3.1 Pico projector……………………………………………………………..14

3.2 Bluetooth………………………………………………………………….15

3.3 Bio-Acoustics and Sensors………………………………………………..16

4.HOW DOES IT WORK……………………………………………………...........18

4.1 Processing………………………………………………………………….19

5.ADVANTAGES OF SKINPUT………………………………………………......20

6.DISADVANTAGES OF SKINPUT……………………………………………. ..21

7.APPLICATIONS OF SKINPUT………………………………………………… 22

8.FUTURE IMPLICATIONS…………………………………………………….. .23

9.CONCLUSION…………………………………………………………………… 24

10.REFERENCES………………………………………………………………….. 25


Skinput

ABSTRACT

Skinput is an input technology that uses bio-acoustic sensing to localize finger taps on the skin. When augmented with a Pico-projector, the device can provide a direct manipulation, graphical user interface on the body. The technology was developed by Chris Harrison, Desney Tan, and Dan Morris, at Microsoft Research's Computational User Experiences Group. Skinput represents one way to decouple input from electronic devices with the aim of allowing devices to become smaller without simultaneously shrinking the surface area on which input can be performed. While other systems, like Sixth Sense have attempted this with computer vision, Skinput employs acoustics, which take advantage of the human body's natural sound conductive properties (e.g., bone conduction). This allows the body to be annexed as an input surface without the need for the skin to be invasively instrumented with sensors, tracking markers, or other items.


Skinput

1.INTRODUCTION

Devices with significant computational power and capabilities can now be easily carried on our bodies. However, their small size typically leads to limited interaction space (e.g. diminutive screens, buttons, and jog wheels) and consequently diminishes their usability and functionality. Since it cannot simply make buttons and screens larger without losing the primary benefit of small size, consider alternative approaches that enhance interactions with small mobile systems. One option is to opportunistically appropriate surface area from the environment for interactive purposes. There is one surface that has been previous overlooked as an input canvas, and one that happens to always travel with us: our skin. Appropriating the human body as an input device is appealing not only because we have roughly two square meters of external surface area, but also because much of it is easily accessible by our hands (e.g., arms, upper legs, torso). Furthermore, proprioception – our sense of how our body is configured in three-dimensional space – allows us to accurately interact with our bodies in an eyes-free manner. For example, we can readily flick each of our fingers, touch the tip of our nose, and clap our hands together without visual assistance. Few external input devices can claim this accurate, eyes-free input characteristic and provide such a large interaction area. In this paper, we present our work on Skinput – a method that allows the body to be appropriated for finger input using a novel, non-invasive, wearable bio-acoustic sensor.


Skinput

2.What is Skinput?

The Microsoft company have developed Skinput , a technology that appropriates the human body for acoustic transmission, allowing the skin to be used as an input surface. In particular, we resolve the location of finger taps on the arm and hand by analyzing mechanical vibrations that propagate through the body. We collect these signals using a novel array of sensors worn as an armband. This approach provides an always available, naturally portable, and on-body finger input system. We assess the capabilities, accuracy and limitations of our technique through a two-part, twenty-participant user study. To further illustrate the utility of our approach, we conclude with several proof-of-concept applications we developed


Skinput

2.1 Always-Available Input

The primary goal of Skinput is to provide an always available mobile input system – that is, an input system that does not require a user to carry or pick up a device. A number of alternative approaches have been proposed that operate in this space. Techniques based on computer vision are popular . These, however, are computationally expensive and error prone in mobile scenarios (where, e.g., non-input optical flow is prevalent). Speech input is a logical choice for always-available input, but is limited in its precision in unpredictable acoustic environments, and suffers from privacy and scalability issues in shared environments. Other approaches have taken the form of wearable computing. This typically involves a physical input device built in a form considered to be part of one’s clothing. For example, glove-based input systems allow users to retain most of their natural hand movements, but are cumbersome, uncomfortable, and disruptive to tactile sensation. A “smart fabric” system that embeds sensors and conductors into a brick, but taking this approach to always-available input necessitates embedding technology in all clothing, which would be prohibitively complex and expensive. The Sixth-Sense project proposes a mobile, always available input/output capability by combining projected information with a colormarker-based vision tracking system. This approach is feasible, but suffers from serious occlusion and accuracy limitations. For example, determining whether, e.g., a finger has tapped a button, or is merely hovering above it, is extraordinarily difficult. In the present work, we briefly explore the combination of on-body sensing with on-body projection.


Skinput

2.2 Bio-Sensing

Skinput leverages the natural acoustic conduction properties of the human body to provide an input system, and is thus related to previous work in the use of biological signals for computer input. Signals traditionally used for diagnostic medicine, such as heart rate and skin resistance, have been appropriated for assessing a user's emotional state. These features are generally subconsciouslydriven and cannot be controlled with sufficient precision for direct input. Similarly, brain sensing technologies such as electroencephalography (EEG) & functional near-infrared spectroscopy (fNIR) have been used by HCI researchers to assess cognitive and emotional state; this work also primarily looked at involuntary signals. In contrast, brain signals have been harnessed as a direct input for use by paralyzed patients, but direct brain computer interfaces (BCIs) still lack the bandwidth requiredfor everyday computing tasks, and require levels of focus, training, and concentration that are incompatible with typical computer interaction.

There has been less work relating to the intersection of finger input and biological signals. Researchers have harnessed the electrical signals generated by muscle activation during normal hand movement through electromyography (EMG). At present, however, this approach typically requires expensive amplification systems and the application of conductive gel for effective signal acquisition, which would limit the acceptability of this approach for most users. The input technology most related to our own is that of Amento et al who placed contact microphones on a user's wrist to assess finger movement. However, this work was never formally evaluated, as is constrained to finger motions in one hand.

The Hambone system employs a similar setup, and through an HMM, yields classification accuracies around 90% for four gestures (e.g., raise heels, snap fingers). Performance of false positive rejection remains untested in both systems at present. Moreover, both techniques required the placement of sensors near the area of


Skinput

interaction (e.g., the wrist), increasing the degree of invasiveness and visibility. Finally, bone conduction microphones and headphones - now common consumer technologies - represent an additional bio-sensing technology that is relevant to the present work. These leverage the fact that sound frequencies relevant to human speech propagate well through bone.

Bone conduction microphones are typically worn near the ear, where they can sense vibrations propagating from the mouth and larynx during speech. Bone conduction headphones send sound through the bones of the skull and jaw directly to the inner ear, bypassing transmission of sound through the air and outer ear, leaving an unobstructed path for environmental sounds


Skinput

2.3Principles of Skinput

It listens to vibrations in your body.

Skinput also responds to the various hand gestures.

The arm is an instrument.


Skinput

3.Technologies in Skinput

There are Three technologies used for Skinput.1. Pico-Projector 2. Bluetooth 3. Bio-Acoustics and Sensors


Skinput

3.1Pico-Projector

Pico-Projector is employed as Output device that show menu. It’s employed in mobile and camera to show the project. Pico-projectors are small, but they can show large displays (sometimes up to 100"). While great for mobility and content sharing, pico-projectors offer low brightness and resolution compared to larger projectors. It is a new innovation, but pico-projectors are already selling at a rate of about a million units a year (in 2010), and the market is expected to continue growing quickly.

pico projector in mobile phone

How do pico projectors work?There are several companies developing and producing pico projectors, and there are 3 major technologies: DLP, LCoS and Laser-Beam-Steering (LBS).

DLP and LCoS use a white light source, and some sort of filtering technique to create a different brightness and color on each pixel:

DLP (Digital Light Processing) the idea behind DLP is to use tiny mirrors on a chip that direct the light. Each mirror controls the amount of light each pixel on the target picture gets (the mirror has two states, on and off. It refreshes many times in a second - and if 50% of the times it is on, then the pixel appears at 50% the brightness). Color is achieved by a using a color wheel between the light source and the mirrors - this splits the light in red/green/blue, and each mirror controls all thee light beams for its designated pixel.

LCoS (Liquid Crystal on Silicon): an LCoS projector uses a small liquid-crystal display (LCD) to control how much light each pixel gets. There are two basic designs to get color: Color-Filter (CF-LCoS) which uses 3 subpixels, each with its own color (RGB) and a Field-Sequential-Color (FSC) which uses a faster LCD with a color filter - so you split the image for the 3 main colors (RGB) sequentially


Skinput

and you refresh the LCD 3 times (once for each color). The light source for the LCoS can be LED or diffused laser.

Laser-Beam-Steering (LBS) projectors are different, creating the image one pixel at a time, using a directed laser beam. You start with 3 different lasers (Red/Green/Blue), each at its required brightness, which are combined using optics, and guided using a mirror (or two mirrors in some designs). If you scan the image fast enough (usually at over 60Hz), you do not notice this pixel-by-pixel design.

3.2BluetoothIt’s used to connect the Bio-Acoustic sensing element for mobile in order so that information will be transferred to many being controlled devices like mobile, iPod or laptopBluetooth is a wireless technology standard for exchanging data over short distances (using short-wavelength UHF radio waves in the ISM band from 2.4 to 2.485 GHz from fixed and mobile devices, and building personal area networks (PANs). Invented by telecom vendor Ericsson in 1994, it was originally conceived as a wireless alternative to RS-232 data cables. It can connect several devices, overcoming problems of synchronization.


Skinput

3.3Bio-Acoustics and Sensors:

When a finger taps the skin, several distinct forms of acoustic energy are produced. Some energy is radiated into the air as sound waves; this energy is not captured by the Skinput system. Among the acoustic energy transmitted through the arm, the most readily visible are transverse waves, created by the displacement of the skin from a finger impact (Figure 1).

Figure 1

When shot with a high-speed camera, these appear as ripples, which propagate outward from the point of contact. The amplitude of these ripples is correlated to both the tapping force and to the volume and compliance of soft tissues under the impact area. In general, tapping on soft regions of the arm creates higher amplitude transverse waves than tapping on boney areas (e.g., wrist, palm, fingers), which have negligible compliance. In addition to the energy that propagates on the surface of the arm, some energy is transmitted inward, toward the skeleton. These longitudinal (compressive) waves(Figure 2). travel through the soft tissues of the arm, exciting the bone, which is much less deformable then the soft tissue but can respond to mechanical excitation by rotating and translating as a rigid body.


Skinput

Figure 2

Bio-Acoustics: Sensing

These signals need to be sensed and worked upon.

This is done by wearing the wave sensor arm band.


Skinput

4.HOW DOES IT WORK

The operating of this Skinput Technology depends on the show and detects principle that uses all 3 parts to produce the result.

Step 1. Once a user faucets on skin surface then Armband Bio-Acoustics sensing element detects the activated or touched part of the skin surface by measure the sound frequency variations in body density, size, mass and impact of sentimental tissues and joints. These variations are then reborn into a digital signal kind.

Step 2. Currently a wireless property technology Bluetooth is employed to attach the Armband Bio-Acoustics sensing element to Mobile, iPod and computers in order that the data/command will be transmitted to those devices that are being controlled. For this software system that matches the sound frequencies of a particular skin surface location is employed. Different corresponding operations are distributed within the device to produce the result.

Step 3. The final step involves the purpose of the show. A Pico-projector is employed during this step as output shows devices operating as a projector to show the menu. This sort of the projectors is employed in mobiles and cameras to show the project.


Skinput

4.1.Processing


Skinput

5.ADVANTAGES OF SKINPUT

No need to interact with the gadget directly. Easy to access when your phone is not available

Don’t have to worry about keypad.

People with larger fingers get trouble in navigating tiny buttons and keypads

on mobile phones. With skinput this problem disappears.

The projected interface can appear much larger than it ever could on a

device’s screen. One can also bring his arm closer to the face (or vice versa)

to see the display close up. Dimming the lights creates an even greater color

contrast if skin and the text are too similar in color during daylight.

Allows users to interact more personally with their device


Skinput

6.DISADVANTAGES OF SKINPUT

If the user has more than a 30% Body Mass Index Skinput is reduced to 80% accuracy

The arm band is currently bulky The visibility of the projection of the buttons on the skin can be reduced if

the user has a tattoo located on their arm


Skinput

7.APPLICATIONS OF SKINPUT

Mobile Gaming I-pods

An aid to paralyzed persons.


Skinput

8.Future Implications

With small sized Pico-projectors, Skinput oriented systems, are an emerging trend.

Research is carried out for smaller wrist watch sized sensor arm band.


Skinput

9.CONCLUSION

Through Skinput, a technological approach to use human body as an input surface is achieved. The wearable bioacoustic sensor array used in the skinput plays a fine role here. The Skinput approach is proved to be useful and better for different gestures when the body is in motion. As a future work, many features like taps with different parts of the finger, single handed gestures and differentiating between objects and materials are being explored and researched with Skinput. Last but not the least, the different applications of Skinput helps us to give a clear idea at what extent we can use this technology effectively. Likewise, Sixth sense also projects information on varied surfaces thus extending the limits of projection from screen to the physical world.


Skinput

10.REFERENCES

"Skinput:Appropriating the Body as an Input Surface". Microsoft Research Computational User Experiences Group

Skinput “Wikipedia” "Skinput: Appropriating the Body as an Input Surface". www.Youtube.com www.google.com


Skinput

Technology

Transcript of Skinput