Silent sound technology
-
Upload
nixytl -
Category
Engineering
-
view
175 -
download
4
Transcript of Silent sound technology
Silent Sound Technology (SST)
OVERVIEW
Introduction
Methods
Applications
Conclusion
Silent Sound………?
“talking without talking”
What is SST?
It is a technology that helps to transmit information without using our vocal cords.
Aims to observe our silent speech and transform it into text/audio output.
The software can be installed in wrist tag/ display, mobile or PC.
“What happens if we don’t communicate? Suddenly welose our voice during an accident……”
Helps those who had lost their voice but wish to speak.
Output can be routed to communication networks.
People can speak over phone without disturbing others.
Also can speak in noisy environment.
Why Needed........?
Idea was popularized in the 1968 Stanley Kubrick’s science fiction film ‘‘2001 – A Space Odyssey ” (Using Electronic signals)
US space agency Nasa has investigated the technique for communicating in noisy environments such as the Space Station.
SST was demonstrated in the year 2010 at CeBIT’s “futurepark”, one of the largest trade fair.
This technology is being developed at Karlsruhe Institute ofTechnology ( KIT ), Germany.
Wand and Tanja Shultz
Origin
METHODS
ELECTROMYOGRAPHY
IMAGE PROCESSING
ELECTROMYOGRAPHY(EMG)
A technique for evaluating and recording the electricalactivity produced by skeletal muscles.
It detects the electrical potential generated by musclecells, when these cells are electrically or neurologicallyactivated.
Performed using instrument called an electromyograph,to produce a record called an electromyogram.
signals can be analyzed to detect medical abnormalities.
How can We Speak….?
When we generally speak aloud, air passes through
larynx or vocal cord & the tongue.
Words are produced using articulator muscle in the
mouth & jaw region.
EMG in SST
Process….
monitor tiny muscular movements that occur when we speak.
Monitored signals are converted into electrical pulses that can then be turned into speech, without a sound uttered.
Fig: Electromyography activity
DRAWBACKS
Device presently needs nine leads to be attached to our face which is quite impractical to make it usable.
It’s little painful.
Translation to Chinese language is a bit difficult.
Not portable
Image processing In SST
A device oriented package to design and implement for the purpose of lip reading.
It works based on our silent speech.
It can recognize words, single sentence or even continuous sentences of people of different region.
Device consider our non-speech accent and pronunciation by observing every movement of our lip and facial Expression
Terms………
Region of Interest(ROI)
Skin segmentation
Face detection
Lip detection
Lip contour
Key points
Facial features
Lip tracking
Fig1:Key points
Fig2:Lip contour with key points
Face Detection
Perform Lighting Compensation on image.
Extract skin region and remove all the noisy data.
Check for face criterions.
Skin colour blocks are identified.
Height and width ratio (1.5 and 0.8) computed and Minimal face dimension constrained is implemented.
Crop the current region.
Skin Segmentation
One of the important steps in face feature extraction.
Colour segmentation of human face depends on the colour space that is selected.
Skin colours of different people are closely grouped in normalized RG colour plane ( by Yang and Waibel).
Search for the pixels which are close enough to this spread .
Normalized RG colour plane
Active Shape Models
a)Original image
d)Active shape of face
Used to detect face in the captured video.Shape model is formed from a set of manually annotated shape of faces:•Align all shapes of the learning data to an arbitrary reference by geometric transformation.•Calculate average shape .Model positioned on the face.Iteratively deformed until it sticks to the face in respective bounding boxesMouth region Localization.
Face Detection
VideoFileReader('path') Reads video frame by frameCascadeObjectDetector('FrontalFaceLBP')Creates a detector for faceactivecontour(A,mask,method)Detect active contour inside face region .Here active contour is lip (i.e.. major difference region).
centroidColumn(X), centroidRow(Y) – centroid pointMiddlerow,middlecolumn– minor and major axis lines of lip contour
Contour fitting point location
Key points
topRowY = find(middleColumn, 1, 'first');centroidColumn, topRowY -this gives top
bottomRowY = find(middleColumn, 1, 'last');centroidColumn, bottomRowY -this gives bottom
leftColumnX = find(middleRow, 1, 'first');leftColumnX, centroidRow -this gives left
rightColumnX= find(middleRow, 1, 'last');rightColumnX, centroidRow -this gives right
1.Live video 2.ROI video
3.Facial features detected live video
4.Lip during motion with perimeter contour and key points
5.Multi Image montage(28 frames)6.Threshold Analysis
Applications
People can communicate in different languages by translating the output of SST. Helps to Analyse and understand the people who have lost voice to speak or stuttering problem.Silent Sound Techniques is applied in Military for communicating secret/confidential matters to others. Helps people to make silent calls during meetings/ in mass crowded places.User can tell PIN no., credit card no., password and other personals without bothering some eavesdroppers.Software can be installed in wrist watch, wrist tag or display/Mobile/Pc and etc.
Conclusion
The software is being trained based on the lip structure, complexion and features of the lip area.
Provide easier mode of communication for people with speech disabilities by converting the identified lip movements directly to speech.
Software can be integrated onto mobile oriented or hand-held devices.
Lip read for Chinese language Mandarin is highly personalized.
Systems are still preliminary need improvement.
REFERERENCES
Pradeep B.S. And Zhang Jingang , “Silent Sound Technology for Mandarin”.
Sasikumar Gurumurthy and B.K.Tripathy , “Design and Implementation of Face Recognition System in Matlab Using the Features of Lips”.
Evangelos Skodras and Nikolaos Fakotakis , “An Unconstrained Method for Lip Detection in Color Images”.
Priya Jethani and Bharat Choudhari , “Silent Sound Technology: A Solution to Noisy Communication”.
Queries?