[email protected] EE Dept., IIT Bombay IEEE Workshop on Intelligent Computing, IIIT Allahabad,...

Slide 1

IEEE Workshop on Intelligent Computing, IIIT Allahabad, 13-15 Oct. 2014

Signal processing for improving speech perception by persons with sensorineural hearing loss: Challenges and some solutions

P. C. Pandey

IIT [email protected] Dept., IIT BombayOutline

A. Speech & Hearing

B. Sliding-band Dynamic Range Compression(N. Tiwari & P. C. Pandey, NCC 2014)

C. Automated modification of consonant-vowel ratio of stops(A. R. Jayan & P. C Pandey, Int. J. Speech Technology, 2014)

#/[email protected] Dept., IIT Bombay2Part A

Speech & HearingP. C. Pandey, "Signal processing for improving speech perception by persons with sensorineural hearing loss: Challenges and some solutions", IEEE Workshop on Intelligent Computing, IIIT Allahabad, 13-15 Oct. 2014

#/[email protected] Dept., IIT BombaySpeech Production Mechanism

Excitation source & filter model Excitation: voiced/unvoiced glottal, frication Filtering: vocal tract filter #/[email protected] Dept., IIT BombaySpeech segments Words Syllables Phonemes Sub-phonemic segments

Phonemes: basic speech units Vowels: Pure vowels, Diphthongs Consonants: Semivowels, Stops, Fricatives, Affricates, Nasals/aba/

/apa/

/aga/

/ada/#/[email protected] Dept., IIT BombayPhonemic features Modes of excitationGlottalUnvoiced (constriction at the glottis), Voiced (glottal vibration) FricationUnvoiced (constriction in vocal tract), Voiced (constriction in v.t., glottal vibration) Movement of articulators Continuant (steady-state v.t. configuration): vowels, nasal stops, fricatives Non-continuant (changing v.t.): diphthongs, semivowels, oral stops (plosives)Place of articulation (place of maximum constriction in v.t.)Bilabial, Labio-dental, Linguo-dental, Alveolar, Palatal, Velar, GluttoralChanges in voicing frequency (Fo)

Supra-segmental features: Intonation, Rhythm #/[email protected] Dept., IIT BombayHearing MechanismPeripheral auditory systemExternal ear: sound collection Pinna Auditory canalMiddle ear: impedance matching Ear drum Middle ear bonesInner ear (cochlea): analysis & transductionAuditory nerve: transmission of neural impulsesCentral auditory system Information processing & interpretation#/[email protected] Dept., IIT Bombay

Tonotopic map of cochleaAuditory system

#/[email protected] Dept., IIT BombayHearing impairment

Types of hearing losses Conductive Sensorineural Central FunctionalSensorineural hearing loss: abnormalities in the cochlear hair cells or the auditory nerve Aging Excessive exposure to noise Infection Adverse effect of medicines Congenital #/[email protected] Dept., IIT BombayEffects of sensorineural hearing lossElevated hearing thresholds: inaudibility of low-level soundsReduced dynamic range & loudness recruitment (abnormal loudness growth): distortion of loudness relationship among speech componentsIncreased temporal masking: poor detection of acoustic landmarksIncreased spectral masking (widening of auditory filters): reduced ability to sense spectral shapes

>> Poor intelligibility and degraded perception of speech,particularly in noisy environment. #/[email protected] Dept., IIT BombaySignal processing in hearing aids

Currently available techniquesFrequency selective amplification: improves audibility but not necessarily intelligibilityAutomatic volume control: not effective in improving intelligibilityMultichannel dynamic range compression (with settable attack & release times, compression ratios): effectiveness reduced due to processing artifacts #/[email protected] Dept., IIT BombayTechniques under developmentNoise suppression Distortion-free dynamic range compressionTechniques for reducing the effects of increased spectral maskingBinaural dichotic presentationSpectral contrast enhancementMulti-band frequency compressionImprovement of consonant-to-vowel ratio (CVR): for reducing the effects of increased temporal masking

#/[email protected] Dept., IIT BombayAnalog Hearing AidsPre-amp AVC Selectable Freq. Response Amp.Digital Hearing AidsPre-amp & AVC ADC Multi-band Amplitude Compr. & Freq. Resp. DAC & Amp.Existing ProblemsNoisy environment & reverberationDistortions due to multiband amplitude compressionPoor speech perception due to increased spectral & temporal maskingVisit to audiologist for change of settings

#/[email protected] Dept., IIT BombayProposed Hearing AidsDistortion-free dynamic range compression & adjustable frequency response Noise suppression & de-reverberation Processing for reducing the effects of increased spectral maskingProcessing for reducing the effects of increased temporal maskingUser selectable settingsImplementation using a low-power DSP chip with acceptable signal delay (< 60 ms)

#/[email protected] Dept., IIT BombaySome Solutions for improving speech perception by listeners with moderate-to-severe sensorineural loss Sliding-band dynamic range compression as a solution to the problem posed by loudness recruitmentAutomated modification of consonant-vowel ratio of stop consonants as a solution to the problem posed by increased intraspeech spectral and temporal masking. Implementation using a 16-bit fixed-point DSP processor & testing for satisfactory operation.#/[email protected] Dept., IIT BombayThank YouTo be continued to Part [email protected] Dept., IIT BombayWorkshop: IEEE Workshop on Intelligent Computing, Allahabad, October 13-15, 2014, organized jointly by CSIR-CEERI Pilani and IIIT Allahabad.Speaker: Prof. P. C. Pandey, EE Dept, IIT BombayTopic: Signal processing for improving speech perception by persons with sensorineural hearing loss: Challenges and some solutionsAbstractSensorineural hearing loss is caused by abnormalities in the cochlear hair cells or the auditory nerve. It occurs due to aging, excessive exposure to noise, infection, or abnormalities at the time of birth. It is generally associated with elevated hearing thresholds, reduced dynamic range, and increased temporal and spectral masking, leading to degraded perception of speech, particular in noisy environment. Several signal processing techniques have been investigated and reported to address these problems. However, most of these are not suited for use in hearing aids due to distortions caused by processing related artifacts or due to constraints of size, power, and acceptable signal delay. As some of the possible solutions, two signal processing techniques have been investigated: (i) a sliding-band dynamic range compression as a solution to the problem posed by loudness recruitment, and (ii) automated modification of consonant-vowel ratio of stop consonants as a solution to the problem posed by increased intraspeech spectral and temporal masking. Both techniques have been implemented using a 16-bit fixed-point DSP processor and tested for satisfactory operation.Persons with sensorineural loss generally have a highly reduced dynamic range of hearing, with a significant frequency-dependent elevation of hearing threshold levels without corresponding increase in the upper comfortable listening levels. Signal processing for dynamic range compression is used to present the sounds comfortably within the limited dynamic range of the listener. Analog hearing aids generally use single-band compression with the gain being dependent on the time-varying signal level. As the power is mostly contributed by the low-frequency components, the amplification of the high-frequency components depends on the energy in the low-frequency components. Thus the high frequency components may become inaudible and distortions in temporal envelope may get introduced. In multiband compression available in most digital hearing aids, the spectral components of the input signal are divided in multiple bands and the gain for each band is calculated on the basis of signal power in that band. This type of processing can introduce spurious spectral distortions and use of a large number of bands reduces spectral contrasts and the modulation depth of speech, resulting in an adverse effect on the perception of certain speech cues. Further, the frequency response of a multiband compression system has a time-varying magnitude response without corresponding variation in the phase response, which can cause audible distortions, particularly for non-speech audio. These distortions may partly offset the advantages of dynamic range compression for the hearing-impaired listener. In order to significantly reduce the temporal and spectral distortions associated with the currently used single-band and multiband compressions in hearing aids, a "sliding-band compression" has been developed. It involves calculating a frequency-dependent gain function, in which the gain for each spectral sample is determined by the short-time power in an auditory critical band centered at it. The gain calculation takes into account the specified hearing thresholds, compression ratios, and attack and release times. Unlike single-band compression, it does not result in any significant temporal distortions because the effect of short-time energy of a spectral component on other spectral components is limited to those located within a critical bandwidth. Due to use of sliding critical bands for calculating the power spectrum, formant transitions do not result in discontinuities in the processed output. The technique is realized using an FFT-based analysis-synthesis method which masks phase related discontinuities and can be integrated with other FFT-based signal processing in hearing aids. The technique is implemented and tested for satisfactory real-time operation on a 16-bit fixed-point DSP [email protected] Dept., IIT BombayIncreasing the level of the consonant segments relative to the nearby vowel segments, known as consonant-vowel ratio (CVR) modification, is reported to be effective in improving speech intelligibility for listeners in noisy backgrounds and for hearing impaired listeners. A technique for real-time CVR modification of stops using the rate of change of spectral centroid for detection of spectral transitions is presented. Its effectiveness in improving the recognition of consonants in the presence of speech spectrum shaped noise is evaluated by conducting listening tests on normal-hearing subjects. At lower values of SNR, there was an increase of 7 - 21% in recognition scores and an equivalent SNR advantage of 3 dB. The technique is implemented on a DSP board based on a 16-bit fixed point processor with on-chip FFT hardware and tested for satisfactory real-time operation.References[1] N. Tiwari and P. C. Pandey, A sliding-band dynamic range compression for use in hearing aids, Proc. National Conference on Communications 2014 (NCC 2014), Kanpur, Feb. 28 - Mar. 2, 2014, paper no. 1569847357.[2] A. R. Jayan & P. C. Pandey, "Automated modification of consonant-vowel ratio of stops for improving speech intelligibility", Int. J. Speech Technology, 2014 (accepted for publication).

Dr. Prem C. Pandey

Dr. Pandey is a Professor in Electrical Engineering at IIT Bombay. He is currently also the Associate Dean of Academic Programmes.He received B.Tech. in electronics engineering from Banaras Hindu University in 1979, M.Tech. in electrical engineering from IIT Kanpur in 1981, and Ph.D. in electrical & biomedical engineering from the University of Toronto (Canada) in 1987. In 1987, he joined the University of Wyoming (USA) as an assistant professor and later joined IIT Bombay in 1989.His research interests include speech & signal processing; biomedical signal processing & instrumentation; electronic instrumentation & embedded system design. The focus of his R&D efforts has been in the areas of impedance cardiography and aids for persons with speech and hearing impairment. [email protected] Dept., IIT Bombay

[email protected] EE Dept., IIT Bombay IEEE Workshop on Intelligent Computing, IIIT Allahabad,...

Documents

Transcript of [email protected] EE Dept., IIT Bombay IEEE Workshop on Intelligent Computing, IIIT Allahabad,...