Chapter 2. Speech Processing Research Program

Chapter 2. Speech Processing Research Program

Academic and Research Staff

Professor Jae S. Lim, Giampiero Sciutto

Graduate Students

Michael S. Brandstein, Shiufun Cheung, John C. Hardwick, Katherine S. Wang, Chang Dong Yoo

Technical and Support Staff

Debra L. Harring, Cynthia LeBlanc

2.1 IntroductionThe objective of this research program is todevelop methods for solving important speechcommunication problems. Current research topicsin progress include development of a new speechmodel and algorithms to (1) enhance speechdegraded by background noise and (2) modify thetime scale of speech. We are also investigatingmethods for displaying spectrograms more effi-ciently.

2.2 Development of a 1.5 KbpsSpeech Vocoder

Sponsor

National Science Foundation Fellowship

Project Staff

Michael S. Brandstein, Professor Jae S. Lim

The recently developed Multi-Band ExcitationSpeech Model has been shown to accuratelyreproduce a wide range of speech signals withoutmany of the limitations inherent in existing speechmodel based systems.' The robustness of thismodel makes it particularly applicable to low bitrate, high quality speech vocoders. Griffin and Limfirst described a 9.6 Kbps speech coder based onthis model. 2 Later work resulted in a 4.8 Kbpsspeech coding system. 3 Both of these systems are

capable of high quality speech reproduction inboth low and high SNR conditions.

The purpose of this research was to exploremethods for using the new speech model at the1.5 Kbps rate. Results indicated that a substantialamount of redundancy exists between the modelparameters. Research focused on exploitingredundancies to quantize these parameters moreefficiently. Attempts were also made to simplifythe existing model without significantly reducingspeech quality.

This research was completed in June 1990.

2.3 A New Method forRepresenting SpeechSpectrograms

Sponsors

National Science FoundationGrant MIP 87-14969

U.S. Navy - Office of Naval ResearchContract N00014-89-J-1489

Project Staff

Shiufun Cheung, Professor Jae S. Lim

The spectrogram, which is a two-dimensionaltime-frequency display of a one-dimensionalsignal, is used extensively in speech research.Existing spectrograms are generally divided into

1 D.W. Griffin and J.S. Lim, "A New Model-Based Speech Analysis/Synthesis System," IEEE International Confer-ence on Acoustic, Speech and Signal Processing, Tampa, Florida, March 26-29, 1985, pp. 513-516.

2 D.W. Griffin and J.S. Lim, "A High Quality 9.6 Kbps Speech Coding System," IEEE International Conference onAcoustic, Speech and Signal Processing, Tokyo, Japan, April 8-11, 1986.

3 J.C. Hardwick, "A 4.8 Kbps Multi-Band Excitation Speech Coder," S.M. thesis, Dept. of Electr. Eng. and Comput.Sci., MIT, 1988.

243

Chapter 2. Speech Processing Research Program

two types, wideband spectrograms and narrow-band spectrograms, according to the bandwidth ofthe analysis filters used to generate them. Due tothe different characteristics of the two types ofspectrograms, they are employed for different pur-poses. The wideband spectrogram is valued for itsquick temporal response and is used for wordboundary location and formant tracking. On theother hand, the narrowband spectrogram, with itshigh frequency resolution, is primarily used formeasuring the pitch frequency.

Various attempts have been made to improve thespectrographic display. Past efforts include thedevelopment of neural spectrograms which usecritical bandwidth analysis filters in the imitation ofthe human auditory system and the developmentof better time-frequency distributions such as theWigner distribution.

In this research, we propose a different approach.The spectrogram is viewed as a two-dimensionaldigital image instead of a transformed one-dimensional speech signal. Image processingtechniques are used to create an improved spec-trogram which preserves the desirable visual fea-tures of the wideband and narrowbandspectrograms. This transforms a speech pro-cessing problem into an image processingproblem.

At this point, we have developed a simple buteffective method for combining the two types ofspectrograms. In this method, each pixel of thecombined spectrogram is the geometric mean ofcorresponding pixel values of the two originalspectrograms. Apart from the geometric-meanmerge, we are also experimenting with the use ofcolor and other merge algorithms. Initial resultsare promising.

2.4 A Dual Excitation SpeechModel

Sponsors

U.S. Air Force - Electronic Systems DivisionContract F19628-89-K-0041

U.S. Navy - Office of Naval ResearchContract N00014-89-J-1489

Project Staff

John C. Hardwick, Professor Jae S. Lim

One class of speech analysis/synthesis system(vocoder) which has been extensively studied andused in practice is based on an underlying model

of speech. Even though traditional vocoders havebeen quite successful in synthesizing intelligiblespeech, they have not been successful in synthe-sizing high quality speech. The Multi-BandExcitation (MBE) speech model, introduced byGriffin, improves the quality of vocoder speechthrough the use of a series of frequency dependentvoiced/unvoiced decisions. The MBE speechmodel, however, still results in a loss of qualitywhen compared with the original speech. Thisdegradation is caused in part by thevoiced/unvoiced decision process. A large numberof frequency regions contain a substantial amountof both voiced and unvoiced energy. If a region ofthis type is declared voiced, then a tonal or hollowquality is added to the synthesized speech. Simi-larly, if the region is declared unvoiced, then addi-tional noise occurs in the synthesized speech. Asthe signal-to-noise ratio decreases, the classifica-tion of speech as either voiced or unvoicedbecomes more difficult, and, consequently, thedegradation is increased.

A new speech model has been proposed inresponse to the aforementioned problems. Thismodel is referred to as the Dual Excitation (DE)speech model, due to its dual excitation and filterstructure. The DE speech model is a generalizationof most previous speech models, and, with theproper selection of the model parameters, itreduces to either the MBE speech model or to avariety of more traditional speech models.

Current research is examining the use of thisspeech model for speech enhancement, time scalemodification and bandwidth compression. Addi-tional areas of study include further refinements tothe model and improvements in the estimationalgorithms.

2.5 Speech EnhancementTechniques for the DualExcitation Vocoder Model

Sponsors

National Science Foundation FellowshipU.S. Air Force - Electronic Systems Division

Contract F19628-89-K-0041U.S. Navy - Office of Naval Research

Contract N00014-89-J-1489

Project Staff

Katherine S. Wang, Professor Jae S. Lim

We explored some conventional methods forspeech enhancement in the presence of additive

244 RLE Progress Report Number 133

white noise4 in a new framework where the voicedestimation provided by the MBE model 5 allowedus to perform noise reduction separately on voicedand unvoiced components.

Conventional methods which take advantage ofthe periodic structure of voiced speech includecomb filtering and adaptive noise cancellation. Atechnique based on short-time spectral amplitudeestimation obtains the minimum mean-square-error, linear estimator of the speech signal by non-causal Wiener filtering, which we couldapproximate by an adaptive Wiener filtering tech-nique. Speech enhancement can also be modelbased, for example, by using classical estimationtheory applied to an all pole model of speech. Wedrew from some of these conventional techniquesto create a speech enhancement system custom-ized to the traits of the Multi-Band Excitation(MBE) Vocoder and, subsequently, to the DualExcitation Model. 6 The noiselike characteristics ofunvoiced speech and the harmonic structure ofvoiced speech suggest that noise reduction canmost effectively be done on speech that has beenseparated into the two components, rather thanattempting to categorize the frequency band aspurely voiced and unvoiced.

The recently developed Multi-Band Excitation(MBE) Speech Model has been shown to accu-rately reproduce a wide range of speech signalswithout many of the limitations inherent in existingspeech model based systems.

This research was completed in August 1990.

Chapter 2. Speech Processing Research Program

2.6 Nonlinear and StatisticalApproach to Speech Synthesis

Sponsor

U.S. Navy - Office of Naval ResearchContract N00014-89-J-1489

Project Staff

Chang Dong Yoo, Professor Jae S. Lim

Numerous speech models have been proposed,and most of these models have been incorporatedwith some variation into the basic speech pro-duction model where a linear time invariant systemis excited by periodic or random pulses dependingon voiced/unvoiced decision. But studies haveindicated that speech production is a nonlinearprocess and that for some sounds such as thevoiced fricatives, it is difficult to make a hardvoiced/unvoiced decision. Discrepancy betweenactual speech production and production from thelinear model system might stem from the Teagerparadigm, considering the vocal tract operationfrom the air flow point of view.

For higher quality speech synthesis, we need toderive either an enhanced speech productionmodel incorporating more of the nonlinear effectsof speech production or a speech synthesismethod totally divorced from the mechanism ofspeech production such as the hidden Mardovmodeling of speech production.

In this research, various nonlinear and statisticalaspects of speech characteristics are being studiedto derive a more efficient method of speech anal-ysis/synthesis.

4 J.S. Lim and A.V. Oppenheim, "Enhancement and Bandwidth Compression of Noisy Speech," IEEE Proc. 67 (12):1586-1604 (1979); J.S. Lim, ed. Speech Enhancement (Englewood Cliffs, N.J.: Prentice Hall, 1983).

5 D.W. Griffin and J.S. Lim, "A New Model-Based Speech Analysis/Synthesis System," IEEE International Confer-ence on Acoustics, Speech and Signal Processing, Tampa, Florida, March 26-29, 1985, pp. 513-516.

6 John Hardwick, Ph.D. research, MIT.

245

Professor Jae S. Lim (standing) and graduate student Matthew M. Bace demonstrate a new method that

was recently developed to reduce channel degradation in the National Television Standards Committee

system.

246 RLE Progress Report Number 133