MICROPHONE ARRAY PROCESSING FOR DISTANT SPEECH RECOGNITION: FROMCLOSE-TALKING MICROPHONES TO FAR-FIELD SENSORS

Kenichi Kumatani, John McDonough, Bhiksha Raj

Distant speech recognition (DSR) holds the promise of the mostnatural human computer interface because it enables man-machineinteractions through speech, without the necessity of donning in-trusive body- or head-mounted microphones. Recognizing distantspeech robustly, however, remains a challenge. This contributionprovides a tutorial overview of DSR systems based on microphonearrays. In particular, we present recent work on acoustic beamform-ing for DSR, along with experimental results verifying the effective-ness of the various algorithms described here; beginning from a worderror rate (WER) of 14.3% with a single microphone of a linear ar-ray, our state-of-the-art DSR system achieved a WER of 5.3%, whichwas comparable to that of 4.2% obtained with a lapel microphone.Moreover, we present an emerging technology in the area of far-fieldaudio and speech processing based on spherical microphone arrays.Performance comparisons of spherical and linear arrays reveal thata spherical array with a diameter of 8.4 cm can provide recognitionaccuracy comparable or better than that obtained with a large lineararray with an aperture length of 126 cm.

1. INTRODUCTION

When the signals from the individual sensors of a microphone arraywith a known geometry are suitably combined, the array functionsas a spatial filter capable of suppressing noise, reverberation, andcompeting speech. Such beamforming techniques have received agreat deal of attention within the acoustic array processing commu-nity in the recent past [1, 2, 3, 4, 5, 6, 7]. Despite this effort, how-ever, such techniques have often been ignored within the mainstreamcommunity working on distant speech recognition. As pointed outin [6, 7], this could be due to the fact that the disparate research com-munities for acoustic array processing and automatic speech recog-nition have failed to adopt each other’s best practices. For instance,the array processing community tends to ignore speaker adaptationtechniques, which can compensate for mismatches between acous-tic conditions during training and testing. Moreover, this commu-nity has largely preferred to work on controlled, synthetic record-ings, obtained by convolving noise- and reverberation-free speechwith measured, static room impulse responses, with subsequent ar-tificial addition of noise, as in the recent PASCAL CHiME SpeechSeparation Challenge [8, 9, 10, 11]. A notable exception was thePASCAL Speech Separation Challenge 2 [5, 12] which featured ac-tual array recordings of real speakers; this task, however, has fallenout of favor, to the extent that it is currently not even mentionedon the PASCAL CHiME Challenge website, nor in any of the con-comitant publications. This is unfortunate because improvements ob-tained with novel speech enhancement techniques tend to diminish—or even disappear—after speaker adaptation; similarly, techniquesthat work well on artificially convolved data with artificially addednoise tend to fail on data captured in real acoustic environments withreal human speakers. Mainstream speech recognition researchers,on the other hand, are often unaware of advanced signal and arrayprocessing techniques. They are equally unaware of the dramatic re-ductions in error rate that such techniques can provide in DSR tasks.

The primary goal of this contribution is to provide a tutorial inthe application of acoustic array processing to distant speech recog-

Speaker Tracker Beamformer Post-filter Speech Recognizer

Microphone Array

Multi-channel Data

Fig. 1. Block diagram of a typical DSR system.

nition that is intelligible to anyone with a general signal processingbackground, while still maintaining the interest of experts in the field.Our secondary goal is to bridge the gaps between the current acous-tic array processing and speech recognition communities. A thirdand overarching goal is to provide a concise report on the state-of-the-art in DSR. Towards this end, we present two empirical studies:the first is a comparison of several beamforming algorithms for theireffectiveness in a DSR task with real speakers in a real acoustic envi-ronment. These are conducted with a conventional linear array. Thesecond performance comparison is between a conventional linear ar-ray and a much more compact spherical array. The latter is gainingimportance as the emphasis in acoustic array processing moves fromlarge static fixtures to smaller mobile devices such as robots.

The remainder of this article is organized as follows. In Sec-tion 2, we provide an overview of a complete DSR system, whichincludes the fundamentals of array processing, speaker tracking andconventional statistical beamforming techniques. In Section 3, weconsider several recently introduced techniques for beamformingwith higher order statistics. This section concludes with our firstset of experimental results comparing conventional beamformingtechniques with those based on higher order statistics. We take upour discussion of array geometry and spherical arrays in particularin Section 4. This section also concludes with a set of experimentalstudies, namely that comparing the performance of a conventionallinear array with a spherical array in a DSR task. In the final sectionof this work, we present our conclusions and plans for future work.

2. OVERVIEW OF DSR

Figure 1 shows a block diagram of a DSR system with a microphonearray. The microphone array module typically consists of a speakertracker, beamformer and post-filter. The speaker tracker estimates aspeaker’s position. Given that position estimate, the beamformer em-phasizes sound waves coming from the direction of interest or “lookdirection”. The beamformed signal can be further enhanced withpost-filtering. The final output is then fed into a speech recognizer.We note that this framework can readily incorporate other informa-tion sources such as a mouth locator based on video data [13].

2.1. Fundamental Issues in Microphone Array Processing

As shown in Figure 2, the array processing components of a DSRsystem are prone to several errors. Firstly, there are errors in speakertracking which cause the beam to be “steered” in the wrong direc-tion [14]; such errors can in turn cause signal cancellation. Secondly,the individual microphones in the array can have different amplitudeand phase responses even if they are of the same type [15, §5.5]. Fi-

nally, the placement of the sensors can deviate from their nominalpositions. All of these factors degrade beamforming performance.

2.2. Speaker Tracking

The speaker tracking problem is generally distinguished from thespeaker localization problem. Speaker localization methods estimatea speaker’s position at a single instant in time without relying on pastinformation. On the other hand, speaker tracking algorithms considera trajectory of instantaneous position estimates.

Speaker localization techniques could be categorized into threeapproaches: seeking a position which provides the maximum steeredresponse power (SRP) of a beamformer [16, §8.2.1], localizing asource based on the application of high-resolution spectral estima-tion techniques such as subspace algorithms [17, §9.3], and estimat-ing sources’ positions from time delays of arrival (TDOA) at the mi-crophones. Due to computational efficiency as well as robustnessagainst mismatches of signal models and microphone errors, TDOA-based speaker localization approaches are perhaps the most popularin DSR. Here, we briefly introduce speaker tracking methods basedon the TDOA.

Shown in Figure 3a is a sound wave propagating from a pointx to each microphone located at ms for all s = 0, · · · , S − 1where S is the total number of sensors. Assuming that the posi-tion of each microphone is specified in Cartesian coordinates, de-note the distance between the point source and each microphone asDs , ‖x −ms‖ ∀ s = 0, · · · , S − 1. Then, the TDOA betweenmicrophones m and n can be expressed as

τm,n(x) , (Dm −Dn) /c, (1)

where c is the speed of sound. Notice that equation (1) implies thatthe wavefront—a surface comprised of the locus of all points on thesame phase—is spherical.

In the case that the array is located far from the speaker, thewavefront can be assumed to be planar, which is called the far-fieldassumption. Figure 3b illustrates a plane wave propagating from thefar-field to the microphones. Under the far-field assumption, theTDOA becomes a function of the angle θ between the direction ofarrival (DOA) and the line connecting two sensors’ positions, andequation (1) can be simplified as

τm,n(θ) , dm,n cos θ/c, (2)

where dm,n is the distance between the microphones m and n.Various techniques have been developed for estimation of the

TDOAs. A comprehensive overview of those algorithms is providedby [18] and comparative studies on real data can be found in [19].From the TDOA between the microphone pairs, the speaker’s po-sition can be computed using classical methods, namely, sphericalintersection, spherical interpolation or linear intersection [2, §10.1].These methods can readily be extended to track a moving speaker byapplying a Kalman filter (KF) to smooth the time series of the in-stantaneous estimates as in [16, §10]. Klee et al. [20] demonstrated,

Target sound source

Phase error

Amplitude error

Microphone position error

Steering error

Microphone errors

Localization error

Direction of arrival

Fig. 2. Representative errors in microphone array processing.

ms

mS-1

ms+1

m0

x

DS-1

Ds+1

D0

Ds

Target sound sourceMicrophone array

Spherical wavefront

z

a)

ms

mS-1

ms+1

m0

Direction of arrival

Planar wavefront

z

ds,s+1

ds,s+1cos

k

b)

Fig. 3. Propagation of a) the spherical wave and b) plane wave.

however, that instead of smoothing a series of instantaneous posi-tion estimates, better tracking could be performed by simply usingthe TDOAs as a sequence of observations for an extended Kalmanfilter (EKF) and estimating the speaker’s position directly from thestandard EKF state estimate update formulae. Klee’s algorithm wasextended to incorporate video features in [21], and to track multiplesimultaneous speakers [22].

2.3. Conventional Beamforming Techniques

In the case of the spherical wavefront depicted in Figure 3a, let us de-fine the propagation delay as τs , Ds/c. In the far-field case shownin Figure 3b, let us define the wavenumber k as a vector perpendic-ular to the planar wavefront pointing in the direction of propagationwith magnitude ω/c = 2π/λ. Then, the propagation delay withrespect to the origin of the coordinate system for microphone s is de-termined through ωτs = kTms. The simplest model of wave prop-agation assumes that a signal f(t), carried on a plane wave, reachesall sensors in an array, but not at the same time. Hence, let us formthe vector

f(t) = [f(t− τ0) f(t− τ1) · · · f(t− τS−1)]T

of the time delayed signals reaching each sensor. In the frequencydomain, the comparable vector of phase-delayed signals is F(ω) =F (ω)v(k, ω) where F (ω) is the transform of f(t) and

v(k, ω) ,[e−iωτ0 e−iωτ1 · · · e−iωτS−1

]T (3)

is the array manifold vector. The latter is manifestly a vector ofphase delays for a plane wave with wavenumber k. To a first order,the array manifold vector is a complete description of the interactionof a propagating wave and an array of sensors.

If X(ω) denotes the vector of frequency domain signals for allsensors, the so-called snapshot vector, and Y (ω) the frequency do-main output of the array, then the operation of a beamformer can berepresented as

Y (ω) = wH(ω) X(ω), (4)where w(ω) is a vector of frequency-dependent sensor weights. Thedifferences between various beamformer designs are completely de-termined by the specification of the weight vector w(ω). The sim-plest beamforming algorithm, the delay-and-sum (DS) beamformer,time aligns the signals for a plane wave arriving from the look direc-tion by setting

wDS , v(k, ω)/S. (5)

Substituting X(ω) = F(ω) = F (ω)v(k, ω) into (4) provides

Y (ω) = wHDS(ω) v(k, ω)F (ω) = F (ω);

−1−0.8 −0.4 0 0.4 0.8 1−50−45−40−35−30−25−20−15−10−50

u

Res

pons

e Fu

nctio

n (d

B) Mainlobe

Sidelobes Sidelobes

Look direction

a)

−1 −0.8 −0.4 0 0.4 0.8 1−50−45−40−35−30−25−20−15−10−50

u

Res

pons

e Fu

nctio

n (d

B)

DOA of Interference

b)

Fig. 4. Beampatterns of a) the delay-and-sum beamformer and b)MVDR beamformer as a function of u = cos θ for the linear array;the noise covariance matrix of the MVDR beamformer is computedwith the interference plane waves propagating from u = ±1/3.

i.e., the output of the array is equivalent to the original signal in theabsence of any interference or distortion. In general, this will be truefor any weight vector achieving

wH(ω) v(k, ω) = 1. (6)

Hereafter we will say that any weight vector w(ω) achieving (6) sat-isfies the distortionless constraint, which implies that any wave im-pinging from the look direction is neither amplified nor attenuated.

Figure 4a shows the beampattern of the DS beamformer, whichindicates the sensitivity of the beamformer in decibels to plane wavesimpinging from various directions. The beampatterns are plotted asa function of u = cos θ where θ is the angle between the DOA andthe axis of the linear array. The beampatterns in Figure 4 were com-puted for a linear array of 20 uniformly-spaced microphones with anintersensor spacing of d = λ/2, where λ is the wavelength of theimpinging plane waves; the look direction is u = 0. The lobe aroundthe look direction is the mainlobe, while the other lobes are side-lobes. The large sidelobes indicate that the suppression of noise andinterference off the look direction is poor; in the case of DS beam-forming, the first sidelobe is only 13 dB below the mainlobe.

To improve upon noise suppression performance provided bythe DS beamformer, it is possible to adaptively suppress spatially-correlated noise and interference N(ω), which can be achieved byadjusting the weights of a beamformer so as to minimize the varianceof the noise and interference at the output subject to the distortionlessconstraint (6). More concretely, we seek w(ω) achieving

argminw wH(ω) ΣN(ω) w(ω), (7)

subject to (6), where ΣN , EN(ω)NH(ω) and E· is the ex-pectation operator. In practice, ΣN is computed by averaging orrecursively updates the noise covariance matrix [17, §7]. The weightvectors obtained under these conditions correspond to the minimumvariance distortionless response (MVDR) beamformer, which hasthe well-known solution [2, §13.3.1]

wHMVDR(ω) =

vH(k, ω) Σ−1N (ω)

vH(k, ω) Σ−1N (ω) v(k, ω)

. (8)

If N(ω) consists of a single plane interferer with wavenumber kIand spectrum N(ω), then N(ω) = N(ω)v(kI) and ΣN(ω) =ΣN (ω)v(kI)v

H(kI), where ΣN (ω) = E|N(ω)|2.Figure 4b shows the beampattern of the MVDR beamformer

for the case of two plane wave interferers arriving from directionsu = ±1/3. It is apparent from the figure that such a beamformercan place deep nulls on the interference signals while maintainingunity gain in the look direction. In the case of ΣN = I, which in-dicates that the noise field is spatially-uncorrelated, the MVDR andDS beamformers are equivalent.

Depending on the acoustic environment, adapting the sensorweights w(ω) to suppress discrete sources of interference can leadto excessively large sidelobes, resulting in poor system robustness. Asimple technique for avoiding this is to impose a quadratic constraint‖w‖2 ≤ γ, for some γ > 0, in addition to the distortionless con-straint (6), when estimating the sensor weights. The MVDR solutionwill then take the form [2, §13.3.7]

wHDL =

vH(ΣN + σ2

d I)−1

vH (ΣN + σ2d I)−1 v

, (9)

which is referred to as diagonal loading where σ2d is the loading

level; the dependence on ω in (9) has been suppressed for conve-nience. While (9) is straightforward to implement, there is no directrelationship between γ and σ2

d ; hence the latter is typically set eitherbased on experimentation or through an iterative procedure. Increas-ing σ2

d decreases ‖wDL‖, which implies that the white noise gain(WNG) also increases [23]; WNG is a measure of the robustness ofthe system to the types of errors shown in Figure 2.

A theoretical model of diffuse noise that works well in practiceis the spherically isotropic field, wherein spatially separated micro-phones receive equal energy and random phase noise signals fromall directions simultaneously [16, §4]. The MVDR beamformer withthe diffuse noise model is called the super-directive beamformer [2,§13.3.4]. The super-directive beamforming design is obtained by re-placing the noise covariance matrix ΣN(ω) with the coherence ma-trix Γ(ω) whose (m,n)-th component is given by

Γm,n(ω) = sinc(ωdm,nc

), (10)

where dm,n is the distance between the mth and nth elements of thearray, and sinc x , sinx/x. Notice that the weight of the super-directive beamformer is determined solely based on the distance be-tween the sensors dm,n and is thus data-independent. In the mostgeneral case, the acoustic environment will consist both of diffusenoise as well as one or more sources of discrete interference, such asin

ΣN(ω) = ΣN (ω)v(kI)vH(kI) + σ2

SIΓ(ω), (11)

where σ2SI is the power spectral density of the diffuse noise.

The MVDR beamformer is of particular interest because it formsthe preprocessing component of two other important beamformingstructures. Firstly, the MVDR beamformer followed by a suitablepost-filter yields the maximum signal-to-noise ratio beamformer [17,§6.2.3]. Secondly, and more importantly, by placing a Wiener fil-ter [24, §2.2] on the output of the MVDR beamformer, the minimummean-square error (MMSE) beamformer is obtained [17, §6.2.2].Such post-filters are important because it has been shown that theycan yield significant reductions in error rate [25, 5]. Of the severalpost-filtering methods proposed in the literature [26], the Zelinskipost-filtering [27] technique is arguably the simplest practical imple-mentation of a Wiener filter. Wiener filters in their pure form areunrealizable because they assume that the spectrum of the desiredsignal is available. The Zelinski post-filtering method uses the auto-and cross-power spectra of the multi-channel input signals to esti-mate the target signal and noise power spectra effectively under theassumption of zero cross-correlation between the noises at differentsensors. We have employed the Zelinski post-filter for the experi-ments described in Sections 3.4 and 4.3.

The MVDR beamformer can be implemented in generalizedsidelobe canceller (GSC) configuration [17, §6.7.3] as shown in Fig-ure 5. For the input snapshot vector X(t) at a frame t, the output ofa GSC beamformer can be expressed as

Y (t) = [wq(t)−B(t)wa(t)]H X(t), (12)

where wq is the quiescent weight vector, B is the blocking matrix,and wa is the active weight vector. In keeping with the GSC for-malism, wq is chosen to satisfy the distortionless constraint (6) [2,

+

waHBH

wHqX(ω) Y(ω)

+

-Z( )

Yb( )

Yc( )

ωω

ω

Fig. 5. Generalized sidelobe cancellation beamformer in the fre-quency domain.

§13.6]. The blocking matrix B is chosen to be orthogonal to wq,such that BH wq = 0. This orthogonality implies that the distor-tionless constraint will be satisfied for any choice of wa.

The MVDR beamformer and its variants can effectively suppresssources of interference. They can also potentially cancel the targetsignal, however, in cases wherein signals correlated with the tar-get signal arrive from directions other than the look direction. Thisis precisely what happens in all real acoustic environments due toreflections from hard surfaces such as tables, walls and floors. Abrief overview of techniques for preventing signal cancellation canbe found in [28].

For the empirical studies reported in Section 3.4 and 4.3, sub-band analysis and synthesis were performed with a uniform DFT fil-ter bank based on the modulation of a single prototype impulse re-sponse [2, §11][29]. Subband adaptive filtering can reduce the com-putational complexity associated with time domain adaptive filtersand improve convergence rate in estimating filter coefficients. Thecomplete processing chain—including subband analysis, beamform-ing, and subband synthesis—is shown in Figure 6; briefly it com-prises the steps of subband filtering as indicated by the blocks labeledH(ωm) followed by decimation. Thereafter the decimated subbandsamples X(ωm) are weighted and summed during the beamformingstage. Finally, the beamformed samples are expanded and processedby a synthesis filter G(ωm) to obtain a time-domain signal. In orderto alleviate the unwanted aliasing effect caused by arbitrary mag-nitude scaling and phase shifts in adaptive processing, our analysisand synthesis filter prototype [29] is designed to minimize individualaliasing terms separately instead of maintaining the perfect recon-struction property.

Working in the discrete time and discrete frequency domains re-quires that the definition (3) of the array manifold vector be modifiedas

v(k, ωm) ,[e−iωmτ0fs e−iωmτ1fs · · · e−iωmτS−1fs

],

where fs is the digital sampling frequency, and ωm = 2πm/M forall m = 0, . . . ,M − 1 are the subband center frequencies.

Fig. 6. Schematic view for beamforming in the subband domain.

3. BEAMFORMING WITH HIGHER-ORDER STATISTICS

The conventional beamforming algorithms estimate their weightsbased on the covariance matrix of the snapshot vectors. In otherwords, the conventional beamformer’s weights are determined solelyby second-order statistics (SOS). Beamforming with higher-orderstatistics (HOS) has recently been proposed in the literature [28]; ithas been demonstrated that such HOS beamformers do not sufferfrom signal cancellation.

In this section, we introduce a concept of non-Gaussianity anddescribe how the fine the structure of a non-Gaussian probabilitydensity function (pdf) can be characterized by measures such as kur-tosis and negentropy. Moreover, we present empirical evidence thatspeech signals are highly non-Gaussian; thereafter we discuss beam-forming algorithms based on maximizing non-Gaussian optimizationcriteria.

3.1. Motivation for Maximizing non-Gaussianity

The central limit theorem [30] states that the pdf of the sum of in-dependent random variables (RVs) will approach Gaussian in thelimit as more and more components are added, regardless of thepdfs of the individual components. Hence, a desired signal corruptedwith statistically independent noise will clearly be closer to Gaussianthan the original clean signal. When a non-stationary signal such asspeech is corrupted with the reverberation, portions of the speechthat are essentially independent—given that the room reverberationtime (300-500 ms) is typically much longer than the duration of anyphone (100 ms)—segments of an utterance that are essentially in-dependent will be added together. This implies that the reverberantspeech must similarly be closer to Gaussian than the original “dry”signal. Hence, by attempting to restore the original super-Gaussianstatistical characteristics of speech, we can expect to ameliorate thedeleterious effects of both noise and reverberation.

There are several popular criteria for measuring a degree of non-Gaussianity. Here, we review kurtosis and negentropy [30, §8].

Kurtosis Among several definitions of kurtosis for an RV Y withzero mean, the kurtosis measure we adopt here is

kurt(Y ) , E|Y |4 − βK(E|Y |2)2. (13)

where βK is a positive constant, which is typically set to βK = 3for kurtosis of real-valued RVs in order to ensure that the Gaussianhas zero kurtosis; pdfs with positive kurtosis are super-Gaussian, andthose with negative kurtosis are sub-Gaussian. An empirical estimateof kurtosis can be computed given some samples from the output ofa beamformer by replacing the expectation operator of (13) with atime average.

Negentropy Entropy is the basic measure for information in infor-mation theory [30]. The differential entropy for continuous complex-valued RVs Y with the pdf pY (·) is defined as

H(Y ) , −∫pY (v) log pY (v)dv = −E log pY (v) . (14)

Another criterion for measuring the degree of super-Gaussianity isnegentropy, which is defined as

neg(Y ) , H(Ygauss)−H(Y ), (15)

where Ygauss is a Gaussian variable with the same variance σ2Y as Y .

For complex-valued RVs, the entropy of Ygauss can be expressed as

H(Ygauss) = log∣∣σ2Y

∣∣+ (1 + log π) . (16)

Note that negentropy is non-negative, and zero if and only if Y hasa Gaussian distribution. Clearly, it can measure how far the desireddistribution is from the Gaussian pdf. Computing the entropy H(Y )

−1 0 10

0.5

1

1.5

0.5-0.5

HistogramGG pdf

Gamma

K0

LaplaceGaussian

a)−1 0 1

00.4

0.8

1.2

1.62

Clean speech

0.5-0.5

Noise corrupted speech

b)−1 0 1

00.4

0.8

1.2

1.62

0.5-0.5

Clean speechReverberated speech

c)Fig. 7. Histograms of real parts of subband frequency componentsof clean speech and a) pdfs, b) noise-corrupted speech and c) rever-berated speech.

of a super-Gaussian variable Y requires knowledge of its specificpdf. Thus, it is important to find a family of pdfs capable of closelymodeling the distributions of actual speech signals. The generalizedGaussian pdf (GG-pdf) is frequently encountered in the field of inde-pendent component analysis (ICA). Accordingly, we used the GG-pdf for the DSR experiments described in Section 3.4. The formof the GG-pdf and entropy is described in [31]. As with kurtosis,an empirical version of entropy can be calculated by replacing theensemble expectation with a time average over samples of the beam-former’s output.

As indicated in (13), the kurtosis measure considers not only thevariance but also the fourth moment, a higher-order statistic. Hence,empirical estimates of kurtosis can be strongly influenced by a fewsamples with a low observation probability, or outliers. Empiricalestimates of negentropy are generally more robust in the presence ofoutliers than those for kurtosis [30, §8].

Distribution of Speech Samples Figure 7a shows a histogram ofthe real parts of subband samples at frequency 800 Hz computedfrom clean speech. Figure 7a also shows the Gaussian distributionand several super-Gaussian pdfs: Laplace, K0, Gamma and GG-pdftrained with the actual samples. As shown in the figure, the super-Gaussian pdfs are characterized by a spikey peak at the mean andheavy tails in regions well-removed from the mean; it is clear thatthe pdf of the subband samples of clean speech is super-Gaussian.It is also clear from Figures 7b and 7c that the distributions of thesubband samples corrupted with noise and reverberation get closer tothe Gaussian pdf. These results suggest that the effects of noise andreverberation can be suppressed by adjusting beamformer’s weightsso as to make the distribution of its outputs closer to that of cleanspeech, that is, the super-Gaussian pdf.

3.2. Beamforming with the maximum super-Gaussian criterion

Given the GSC beamformer’s output Y , we can obtain a measureof its super-Gaussianity with (13) or (15). Then, we can adjust theactive weight vector so as to achieve the maximum kurtosis or ne-gentropy while maintaining the distortionless constraint (6) under theformalism of the GSC. In order to avoid a large active weight vec-tor, a regularization term is added, which has the same function asdiagonal loading in conventional beamforming. We denote the costfunction as J (Y ) = J(Y )− α‖wa‖2 (17)where J(Y ) is the kurtosis or negentropy, and α > 0 is a con-stant. A stricter constraint on the active weight vector can also beimposed as ‖wa‖2 ≤ γ for some real γ > 0. Due to the absence of aclosed-form solution for that wa maximizing (17), we must resort toa numerical optimization algorithm; details can be found in [28, 31]for maximum negentropy (MN) beamforming and [32] for maximumkurtosis (MK) beamforming.

As shown in [28], beamforming algorithms based on the maxi-mum super-Gaussian criteria attempt to strengthen the reflected waveof a desired source so as to enhance speech. Of course, any reflected

+

waHBH

wHqX(ω) Y(ω)

+

-Z( )

Yb( )

Yc( )

UH ωω

ω

Fig. 8. Maximum kurtosis beamformer with the subspace filter.

signal would be delayed with respect to the direct path signal. Sucha delay would, however, manifest itself as a phase shift in the fre-quency domain if the delay is shorter than the length of the analysisfilter, and could thus be removed through a suitable choice of wabased on the maximum super-Gaussian criteria. Hence, the MN andMK beamformers offer the possibility of suppressing the reverbera-tion effect by compensating the delays of the reflected signals.

In real acoustic environments, the desired signal will arrive frommany directions in addition to the direct path. Therefore, it is notfeasible for conventional adaptive beamformers to avoid the signalcancellation effect, as demonstrated in experiments described later.On the other hand, MN or MK beamforming can estimate the activeweight vector to enhance target speech without signal cancellationsolely based on the super-Gaussian criterion.

3.3. Online Implementation with Subspace Filtering

Adaptive beamforming algorithms require a certain amount of datafor stable estimation of the active weight vector. In the case of HOS-based beamforming, this problem can become acute because the opti-mization surfaces encountered in HOS beamforming are less regularthan those in conventional beamforming. In order to achieve effi-cient estimation, an eigen- or subspace filter [17, §6.8] can be usedas a pre-processing step for estimation of the active weight vector.In this section, we review MK beamforming with subspace filtering,which was originally proposed in [33].

Figure 8 shows configuration of the MK beamformer with thesubspace filter. The beamformer’s output can be expressed as

Y (t) = [wq(t)−B(t)U(t)wa(t)]H X(t). (18)

The difference between (12) and (18) is the subspace filter betweenthe blocking matrix and active weight vector. The motivations be-hind this idea are to 1) reduce the dimensionality of the active weightvector, and 2) improve speech enhancement performance based ondecomposition of the outputs of the blocking matrix into spatially-correlated and ambient signal components. Such a decompositioncan be achieved by performing an eigendecomposition on the covari-ance matrix of the output of the blocking matrix. Then, we select theeigenvectors corresponding to the largest eigenvalues as the domi-nant modes [17, §6.8.3]. The dominant modes are associated withthe spatially-correlated signals and the other modes are averaged asa signal model of ambient noise. In doing so, we can readily sub-tract the averaged ambient noise component from the beamformer’soutput. Moreover, the dimensionality reduction of the active weightvector leads to computationally efficient and reliable estimation.

Figure 9 illustrates actual eigenvalues sorted in descending orderover frequencies. In order to generate the plots of the figure, wecomputed the eigenvalues from the outputs of the blocking matrixon the real data described in [34]. As shown in Figure 9, there is adistinct difference between the small and large eigenvalues at eachfrequency bin. Thus, it is relatively easy to determine the number ofthe dominant eigenvalues D especially in the case where the numberof the microphones is much larger than the number of the spatially-correlated signals.

Based on equation (13) and (18), the kurtosis of the outputs iscomputed from an incoming block of input subband samples insteadof using the entire utterance. We incrementally update the dominantmodes and active weight vector at each block of samples. Again,

Fig. 9. Eigenvalues sorted in descending order over frequencies.

must to resort to a gradient-based optimization algorithm for estima-tion of the active weight vector. The gradient is iteratively calculatedwith a block of subband samples until the kurtosis value of the beam-former’s outputs converges. This block-wise method is able to tracka non-stationary sound source, and provides a more accurate gradientestimate than sample-by-sample gradient estimation algorithms.

3.4. Evaluation of Beamforming Algorithms

In this section, we compare the SOS-based beamforming methods tothe HOS-based algorithms. The results of DSR experiments reportedhere were obtained on speech material from the Multi-Channel WallStreet Journal Audio Visual Corpus (MC-WSJ-AV); see [4] for de-tails of the data collection apparatus. The size of the recordingroom was 650 × 490 × 325 cm and the reverberation time T60 wasapproximately 380 ms. In addition to reverberation, some record-ings include significant amounts of background noise produced bycomputer fans and air conditioning. The far-field speech data wasrecorded with two circular, equi-spaced eight-channel microphonearrays with diameters of 20 cm, although we used only one of thesearrays for our experiments. Additionally, each speaker was equippedwith a close talking microphone (CTM) to provide the best possiblereference signal for speech recognition. The sampling rate of therecordings was 16 kHz. For the experiments, we used a portion ofdata from the single speaker stationary scenario where a speakerwas asked to read sentences from six fixed positions. The test dataset contains recordings of 10 speakers where each speaker readsapproximately 40 sentences taken from the 5,000 word vocabularyWSJ task. This provided a total 39.2 minutes of speech.

Prior to beamforming, we first estimated the speaker’s positionwith the tracking system described in [22]. Based on an averagespeaker position estimated for each utterance, active weight vectorswa were estimated for the source on a per utterance basis.

Four decoding passes were performed on waveforms obtainedwith various beamforming algorithms. The details of the featureextraction component of our ASR system are given in [28]. Eachpass of decoding used a different acoustic model or speaker adap-tation scheme. The speaker adaptation parameters were estimatedusing the word lattices generated during the prior pass. A descrip-tion of the four decoding passes follows: (1) decode with the un-adapted, conventional maximum likelihood (ML) acoustic model; (2)estimate vocal tract length normalization (VTLN) [2, §9] and con-strained maximum likelihood linear regression (CMLLR) parame-ters [2, §9] for each speaker, then redecode with the conventionalML acoustic model; (3) estimate VTLN, CMLLR and maximum like-lihood linear regression (MLLR) [2, §9] parameters, then redecodewith the conventional model; and (4) estimate VTLN, CMLLR andMLLR parameters for each speaker, then redecode with the ML-SATmodel [2, §8.1]. The standard WSJ trigram language model was usedin all passes of decoding.

Table 1 shows the word error rates (WERs) for each beamform-ing algorithm. As references, WERs are also reported for the CTM

Beamforming (BF) Algorithm Pass (%WER)1 2 3 4

Single array channel (SAC) 87.0 57.1 32.8 28.0Delay-and-sum (D&S) BF 79.0 38.1 20.2 16.5Super-directive (SD) BF 71.4 31.9 16.6 14.1

MVDR BF 78.6 35.4 18.8 14.8Generalized eigenvector (GEV) BF 78.7 35.5 18.6 14.5

Maximum kurtosis (MK) BF 75.7 32.8 17.3 13.7Maximum negentropy (MN) BF 75.1 32.7 16.5 13.2

SD MN BF 75.3 30.9 15.5 12.2Close talking microphone (CTM) 52.9 21.5 9.8 6.7

Table 1. Word error rates for each beamforming algorithm after ev-ery decoding pass.

and single array channel (SAC). It is clear from Table 1 that dra-matic improvements in recognition performance are achieved by thespeaker adaptation techniques which are also able to adapt the acous-tic models to the noisy acoustic environment. Although the use ofspeaker adaptation techniques can greatly reduce WER, they oftenalso reduce the improvement provided by signal enhancement tech-niques. As speaker adaptation is integral to the state-of-the-art, itis essential to report WER all such techniques having been applied;unfortunately this is rarely done in the acoustic array processing lit-erature. It is also clear from these results that the maximum kur-tosis beamforming (MK BF) and maximum negentropy beamform-ing (MN BF) methods can provide better recognition performancethan the SOS-based beamformers, such as the super-directive beam-former (SD BF) [2, §13.3.4], the MVDR beamformer (MVDR BF)and the generalized eigenvector beamformer (GEV BF) [35]. Thisis because the HOS-based beamformers can use the echos of the de-sired signal to enhance the final output of the beamformer and, asmentioned in Section 3.2, do not suffer from signal cancellation. Un-like the SOS beamformers, the HOS beamformers perform best whenthe active weight vector is adapted while the desired speaker is ac-tive. The SOS-based and HOS-based beamformers can be profitablycombined because they employ different criteria for estimation of theactive weight vector. For example, the super-directive beamformer’sweight can be used as the quiescent weight vector in GSC configu-ration [36]. We observe from Table 1 that the maximum negentropybeamformer with super-directive beamformer (SD MN BF) providedthe best recognition performance in this task.

3.5. Effect of HOS-based Beamforming with Subspace Filtering

In this section, we investigate effects of MK beamforming with sub-space filtering. The Copycat data [34] were used as test material here.The speech material in this corpus was captured with the 64-channellinear microphone array; the sensors were arranged linearly with a2 cm inter-sensor spacing. In order to provide a reference, subjectswere also equipped with lapel microphones with a wireless connec-tion to a preamp input. All the audio data were stored at 44.1 kHzwith a 16 bit resolution. The test set consists of 356 (1,305 words) ut-terances spoken by an adult and 354 phrases (1,297 words) uttered bynine children who aged four to six. The vocabulary size is 147 words.As is typical for children in this age group, pronunciation was quitevariable and the words themselves were sometimes indistinct.

For this task, the acoustic models were trained with two publiclyavailable corpora of children’s speech, the Carnegie Mellon Univer-sity (CMU) Kids’ corpus and the Center for Speech and LanguageUnderstanding (CSLU) Kids’ corpus. The details of the ASR systemare described in [33]. The decoder used here consists of three passes;the first and second passes are the same as the ones described in Sec-tion 3.4 but the third pass includes processing of the third and fourthpasses described in Section 3.4.

Table 2 shows word error rates (WERs) of every decoding passobtained with the single array channel (SAC), super-directive beam-forming (SD BF), conventional maximum kurtosis beamforming

Pass (%WER)1 2 3

Algorithm Exp. Child Exp. Child Exp. ChildSAC 9.2 31.0 3.8 17.8 3.4 14.2

SD BF 5.4 24.4 2.5 9.6 2.2 7.6MK BF 5.4 25.1 2.5 9.0 2.1 6.5

MK BF w SF 6.3 25.4 1.2 7.4 0.6 5.3CTM 3.0 12.5 2.0 5.7 1.9 4.2

Table 2. Word error rates (WERs) for each decoding pass.

Pass (%WER)Algorithm Block size 2 3

(second) Exp. Child Exp. ChildConventional 0.25 4.4 15.8 3.5 12.0

MK BF 0.5 3.4 9.2 3.1 7.31.0 2.4 10.3 2.2 6.92.5 2.5 9.0 2.1 6.5

MK BF w SF 0.25 2.5 14.1 1.5 9.70.5 1.3 8.7 1.0 7.01.0 1.2 7.4 0.6 5.3

Table 3. WERs as a function of amounts of adaptation data.

(MK BF) and maximum kurtosis beamforming with the subspacefilter (MK BF w SF). The WERs obtained with the lapel microphoneare also provided as a reference. It is also clear from Table 2 that themaximum kurtosis beamformer with subspace filtering achieved thebest recognition performance.

Table 3 shows the WERs of the conventional and new MK beam-forming algorithms as a function of amounts of adaptation data ineach block. We can see from Table 3 that MK beamforming withsubspace filtering (MK BF w SF) provides better recognition per-formance with the same amount of the data than conventional MKbeamforming. In the case that little adaptation data is available, theMK beamforming does not always improve the recognition perfor-mance due to the dependency of the initial value and noisy gradientinformation which can significantly change over the blocks. The re-sults in Table 3 suggest that unreliable estimation of the active weightvector can be avoided by constraining the search space with a sub-space filter, as described in Section 3.3. Note that the solution of theeigendecomposition does not depend on the initial value in contrastto the gradient-based numerical optimization algorithm.

4. EFFECTS OF ARRAY GEOMETRY

In the majority of the DSR literature, results obtained with linear orcircular arrays have been reported. On the other hand, in the fieldof acoustic array processing, spherical microphone array techniqueshave recently received a great deal of attention [37, 38, 39, 40]. Theadvantage of spherical arrays is that they can be pointed at a desiredspeaker in any direction with equal effect; the shape of the beampat-tern is invariant to the look direction. The following sections providea review of beamforming methods in the spherical harmonics do-main. Thereafter we provide a comparison of spherical and lineararrays in terms of DSR performance.

4.1. Spherical Microphone Arrays

In this section, we describe how beamforming is performed in thespherical harmonics domain. We will use the spherical coordinatesystem (r, θ, φ) shown in Figure 10 and denote the pair of polar an-gle θ and azimuth φ as Ω = (θ, φ).

Spherical Harmonics Let us begin by defining the spherical har-monic of order n and degree m [37] as

z

y

x

r

z=r cosθ

θ

ϕ x=r sin cosϕθ

y=r sin sinϕθ

x

Fig. 10. Relationship between the Cartesian and spherical coordinatesystems.

Y mn (Ω) ,

√(2n+ 1)

4π

(n−m)!

(n+m)!Pmn (cos θ)eimφ, (19)

where Pmn (·) denotes the associated Legendre function [41, §6.10.1].Figure 11 shows the magnitude for the spherical harmonics, Y0 ,Y 0

0 , Y1 , Y 01 , Y2 , Y 0

2 and Y3 , Y 03 in three-dimensional space.

The spherical harmonics satisfy the orthonormality condition [37,38],

δn,n′ δm,m′ =

∫Ω

Y m′

n′ (Ω)Y mn (Ω)dΩ (20)

=

∫ 2π

0

∫ π

0

Y m′

n′ (θ, φ)Y mn (θ, φ) sin θdθ dφ, (21)

where δm,n is the Kronecker delta function, and Y is the complexconjugate of Y .

Spherical Fourier Transform In Section 2.3, we defined thewavenumber as a vector perpendicular to the front of a plane waveof frequency ω pointing in the direction of propagation with amagnitude of ω/c. Now let us define the wavenumber scalar ask = |k| = ω/c; when no confusion can arise, we will also referto k as simply the wavenumber. Let us assume that a plane waveof wavenumber k with unit power is impinging on a rigid sphere ofradius a from direction Ω0 = (θ0, φ0). The total complex soundpressure on the sphere surface at Ωs can be expressed as

G(ka,Ωs,Ω0) = 4π

∞∑n=0

inbn(ka)

n∑m=−n

Y mn (Ω0)Y mn (Ωs), (22)

where the modal coefficient bn(ka) is defined as [37, 39]

bn(ka) , jn(ka)− j′n(ka)

h′n(ka)hn(ka); (23)

jn and hn are the spherical Bessel function of the first kind and theHankel function of the first kind [42, §10.2], respectively, and a prime

a) b) c) d)

Fig. 11. Magnitude of spherical harmonics, a) Y0, b ) Y1, c) Y2 andd) Y3.

-60

-50

-40

-30

-20

-10

0

100 1

|bn(

ka)|

(dB

)

ka

b0

b2

b3

b6 b7

b5b4

b8

b1

10

Fig. 12. Magnitude of the modal coefficients as a function of ka.

indicates the derivative of a function with respect to its argument.Figure 12 shows the magnitude of the modal coefficients as a func-tion of ka. It is apparent from the figure that the spherical array willhave poor directivity at the lowest frequencies—such as ka = 0.2which corresponds to 260 Hz for a = 4.2 cm—inasmuch as only Y0

is available for beamforming; amplifying the higher order modes atthese frequencies would introduce a great deal of sensor self noiseinto the beamformer output. From Figure 11 a), however, it is clearthat Y0 is completely isotropic; i.e., it has no directional characteris-tics and hence provides no improvement in directivity over a singleomnidirectional microphone.

The sound field G can be decomposed by the spherical Fouriertransform as

Gmn (ka,Ω0) =

∫Ω

G(ka,Ω,Ω0)Y mn (Ω)dΩ (24)

and the inverse transform is defined as

G(ka,Ω,Ω0) =

∞∑n=0

n∑m=−n

Gmn (ka,Ω0)Y mn (Ω). (25)

The transform (24) can be intuitively interpreted as the decompo-sition of the sound field into the spherical harmonics illustrated inFigure 11.

Upon substituting the plane wave (22) into (24), we can representthe plane wave in the spherical harmonics domain as

Gmn (ka,Ω0) = 4π in bn(ka) Y mn (Ω0). (26)

In order to understand how beamforming may be performed in thespherical harmonic domain, we need only define the modal arraymanifold vector [43, §5.1.2] as

v(ka,Ω0) ,

G00(ka,Ω0)

G−11 (ka,Ω0)G0

1(ka,Ω0)G1

1(ka,Ω0)G−2

2 (ka,Ω0)G−1

2 (ka,Ω0)G0

2(ka,Ω0)...

G−NN (ka,Ω0)...

GNN (ka,Ω0)

, (27)

which fulfills precisely the same role as (3). It is similarly possible todefine a noise plus interference vector N(ka) in spherical harmonic

z

x

y

a)

z

x

y

b)

Fig. 13. Spherical MVDR beampatterns with a single plane waveinterferer: a) radially symmetric, b) asymmetric.

space. Moreover, Yan et al. [40] demonstrated that the covariancematrix for the spherically isotropic noise field in spherical harmonicspace can be expressed as

Γ(ka) = 4πσ2SI diag|b0(ka)|2,−|b1(ka)|2,−|b1(ka)|2,

− |b1(ka)|2, |b2(ka)|2, · · · , (−1)N |bN (ka)|2, (28)

where σ2SI is the noise power spectral density.

With the changes described above, all of the relations developedin Section 2.3 can be applied; the key intuition is that the physicalmicrophones have been replaced by the spherical harmonics whichhave the very attractive property of the orthonormality as indicatedby (21). In particular, the weights of the delay-and-sum beamformercan be calculated as in (5), the MVDR weights as in (8), and thediagonally loaded MVDR weights as in (9); the spherical harmon-ics super-directive beamformer is obtained by replacing ΣN in (9)with (28).

4.2. Discretization

In practice, it is impossible to construct a continuous, pressure sen-sitive spherical surface; the pressure must be sampled at S discretepoints with microphones. The discrete spherical Fourier transformand the inverse transform can be written as

Gmn (ka) =

S−1∑s=0

αsG(ka,Ωs)Ymn (Ωs), (29)

G(ka,Ωs) =

N∑n=0

n∑m=−n

Gmn (ka)Y mn (Ωs), (30)

where Ωs indicates the position of microphone s and αs is a quadra-ture constant. Typically N is limited such that (N + 1)2 ≤ S toprevent spatial aliasing [37].

Accordingly, the orthonormality condition (21) is approximatedby the weighted summation, which causes orthonormality error [38].In order to alleviate the error caused by discreteness, spatial sam-pling schemes [39] or beamformer’s weights [38] must be carefullydesigned. In this article, we use a spherical microphone array with32 equidistantly spaced sensors and set αs = 4π/S in (29) for theexperiments described later.

Shown in Figure 13 are two spherical three-dimensional MVDRbeampatterns in the presence of a single plane wave interferer, oneconstrained to radial symmetry, and the other with no constraint. Inboth cases the look direction is Ω0 = (0, 0) and the discrete inter-ference is impinging from ΩI =

(π6, −π

4

). As is apparent from the

figures, in order to place a null on the interferer, which is well insidethe main lobe of the DS beamformer, it was necessary to allow largesidelobes.

4.3. Comparison of Linear and Spherical Arrays for DSR

As a spherical microphone array has—to the best of our knowledge—never before been applied to DSR, our first step in investigating its

suitability for such a task was to capture some prerecorded speechplayed into a real room through a loudspeaker, then perform beam-forming and subsequently speech recognition. Figure 14 shows theconfiguration of room used for these recordings. As shown in thefigure, the loudspeaker was placed in two different positions; thelocations of the sensors and loudspeaker were measured with Nat-uralPoint’s motion capture system, OptiTrack. For data capture weused an Eigenmike R© which consists of 32 microphones embeddedin a rigid spherical baffle of radius 4.2 cm; for further details seethe website of mh acoustics, http://www.mhacoustics.com. Eachsensor of the Eigenmike R© is centered on the face of a truncatedicosahedron. As a reference, the speech was also captured with the64-channel linear microphone array described in Section 3.5. Theaperture length of the linear array is 126 cm. The TIMIT data wereused as test material. The test set consisted of 3,241 words utteredby 37 speakers for each recording position. The sampling rate of thedata was 44.1 kHz. In the recording room, the reverberation timeT60 was approximately 525 ms.

We used the same speech recognizer and decoding passes de-scribed in Section 3.4 for the experiments presented here. Tables 4shows word error rates (WERs) for each beamforming algorithmin the case that the incident angles of the target signal to the ar-ray are 28o and 68o, respectively. As a reference, the WERs ob-tained with a single array channel (SAC) and the clean data playedthrough the loudspeaker (Clean data) are also reported. It is clearfrom Tables 4 that every beamforming algorithm can provide the bet-ter recognition performance than the SAC after the adapted passes.It is also clear from the tables that super-directive beamforming withthe small spherical array of radius 4.2 cm (Spherical SD BF) canachieve recognition performance very comparable to that obtainedwith the same beamforming method with the linear array (SD BFwith linear array). In the case where the speaker position is nearly infront of the array, super-directive beamforming with the linear array(SD BF with linear array) can still achieve the best result among allthe algorithms. This is because of the highest directivity index can beachieved with 64 channels, twice as many as the sensors in the spher-ical array. In the other configuration, however, spherical harmonicssuper-directive beamforming (Spherical SD BF) provided better re-sults than the linear array because they can maintain the same beampattern regardless of the incident angle. In these experiments, spher-ical D&S beamforming (Spherical D&S BF) could not improve therecognition performance significantly because of the poor directivity.

5. CONCLUSIONS AND FUTURE DIRECTIONS

This contribution provided a comprehensive overview of representa-tive microphone array methods for DSR. The article also presented

500 cm

400

cm

Room Height 288 cm

Linear array

77 cm 86 cm 96 cm

Position 1

Position 2 28°

68°

128

cm

Fig. 14. Layout of the recording room.

Beamforming (BF) Algorithm Pass (%WER)1 2 3 4

Single array channel (SAC) 47.3 18.9 14.3 13.6D&S BF with linear array 44.7 17.2 11.1 9.8SD BF with linear array 45.5 16.4 10.7 9.3

Spherical D&S BF 47.3 16.8 13.0 12.0Spherical SD BF 42.8 14.5 11.5 10.2

Clean data 16.7 7.5 6.4 5.4a) Incident angle to the array is 28o.

Beamforming (BF) Algorithm Pass (%WER)1 2 3 4

Single array channel (SAC) 57.8 25.1 19.4 16.6D&S BF with linear array 53.6 24.3 16.1 13.3SD BF with linear array 52.6 23.8 16.6 12.8

Spherical D&S BF 57.6 22.7 14.9 13.5Spherical SD BF 44.8 15.5 11.3 9.7

Clean data 16.7 7.5 6.4 5.4b) Incident angle to the array is 68o.

Table 4. WERs for each beamforming algorithm.

recent progress in adaptive beamforming. The undesired effectssuch as signal cancellation and distortion of the target speech canbe avoided by incorporating the fact that the distribution of speechsignals is non-Gaussian into the framework of generalized sidelobecanceller beamforming. It was demonstrated that the state-of-the artDSR system can achieve recognition accuracy very comparable tothat obtained by a close-talking microphone in a small vocabularytask. Finally, we discussed an emerging research topic, namelyspherical microphone arrays. In terms of speech recognition per-formance, the spherical array was compared with the linear array.The results suggested that the compact spherical microphone arraycan achieve recognition performance comparable to a large lineararray. In our view, a key research topic will be how we efficientlyintegrate different information sources such as faces in video dataand turn-taking models into a DSR system.

[1] M. Omologo, M. Matassoni, and P. Svaizer, “Environmen-tal conditions and acoustic transduction in hands-free speechrecognition,” Speech Communication, vol. 25, pp. 75–95, 1998.

[2] Matthias Wolfel and John McDonough, Distant Speech Recog-nition, Wiley, New York, 2009.

[3] Maurizio Omologo, “A prototype of distant-talking interfacefor control of interactive TV,” in Proc. Asilomar Conference onSignals, Systems and Computers (ASILOMAR), Pacific Grove,CA, 2010.

[4] M. Lincoln, I. McCowan, I. Vepa, and H. K. Maganti, “Themulti-channel Wall Street Journal audio visual corpus (MC-WSJ-AV): Specification and initial experiments,” in Proc. IEEEworkshop on Automatic Speech Recognition and Understand-ing (ASRU), 2005, pp. 357–362.

[5] John McDonough, Kenichi Kumatani, Tobias Gehrig, EmilianStoimenov, Uwe Mayer, Stefan Schacht, Matthias Wolfel, andDietrich Klakow, “To separate speech!: A system for recogniz-ing simultaneous speech,” in Proc. MLMI, 2007.

[6] John McDonough and Matthias Wolfel, “Distant speech recog-nition: Bridging the gaps,” in Proc. IEEE Joint Workshopon Hands-free Speech Communication and Microphone Arrays(HSCMA), Trento, Italy, 2008.

[7] Michael Seltzer, “Bridging the gap: Towards a unified frame-work for hands-free speech recognition using microphone ar-rays,” in Proc. HSCMA, Trento, Italy, 2008.

[8] Heidi Christensen, Jon Barker, Ning Ma, and Phil Green, “TheCHiME corpus: A resource and a challenge for computationalhearing in multisource environments,” in Proc. Interspeech,Makuhari, Japan, 2010.

[9] Tomohiro Nakatani, Takuya Yoshioka, Shoko Araki Marc Del-croix, and Masakiyo Fujimoto, “Logmax observation model

with MFCC-based spectral prior for reduction of highly nonsta-tionary ambient noise,” in ICASSP 2012, Kyoto, Japan, 2012.

[10] Felix Weninger, Martin Wollmer, Jurgen Geiger, BjornSchuller, Jort Fe Gemmeke, Antti Hurmalainen, Tuomas Vir-tanen, and Gerhard Rigoll, “Non-negative matrix factorizationfor highly noise-robust asr: To enhance or to recognize?,” inICASSP 2012, Kyoto, Japan, 2012.

[11] Ramon Fernandez Astudillo, Alberto Abad, and Joao Pauloda Silva Neto, “Integration of beamforming and automaticspeech recognition through propagationof the wiener poste-rior,” in ICASSP 2012, Kyoto, Japan, 2012.

[12] Iain McCowan, Ivan Himawan, and Mike Lincoln, “A mi-crophone array beamforming approach to blind speech sepa-ration,” in Proc. MLMI, 2007.

[13] Shankar T. Shivappa, Bhaskar D. Rao, and Mohan M. Trivedi,“Audio-visual fusion and tracking with multilevel iterative de-coding: Framework and experimental evaluation,” J. Sel. Top-ics Signal Processing, vol. 4, no. 5, pp. 882–894, 2010.

[14] Matthias Wolfel, Kai Nickel, and John W. McDonough, “Mi-crophone array driven speech recognition: Influence of local-ization on the word error rate,” in Proc. MLMI, 2005, pp. 320–331.

[15] Ivan Jelev Tashev, Sound Capture and Processing: PracticalApproaches, Wiley, Chichester, UK, 2009.

[16] M. Brandstein and D. Ward, Eds., Microphone Arrays,Springer Verlag, Heidelberg, Germany, 2001.

[17] H. L. Van Trees, Optimum Array Processing, Wiley-Interscience, New York, 2002.

[18] Jingdong Chen, Jacob Benesty, and Yiteng Huang, “Time de-lay estimation in room acoustic environments: An overview,”EURASIP J. Adv. Sig. Proc., 2006.

[19] A. Brutti, M. Omologo, and P. Svaizer, “Comparison betweendifferent sound source localization techniques based on a realdata collection,” in Proc. HSCMA, Trento, Italy, 2008.

[20] Ulrich Klee, Tobias Gehrig, and John McDonough, “Kalmanfilters for time delay of arrival–based source localization,”EURASIP J. Adv. Sig. Proc., 2006.

[21] Tobias Gehrig, Kai Nickel, Hazim K. Ekenel, Ulrich Klee, andJohn McDonough, “Kalman filters for audio–video source lo-calization,” in Proc. IEEE Workshop on Applications of SignalProcessing to Audio and Acoustics (WASPAA), New Paltz, NY,2005.

[22] Tobias Gehrig, Ulrich Klee, John McDonough, Shajith Ikbal,Matthias Wolfel, and Christian Fugen, “Tracking and beam-forming for multiple simultaneous speakers with probabilisticdata association filters,” in Proc. Interspeech, 2006.

[23] H. Cox, R. M. Zeskind, and M. M. Owen, “Robust adaptivebeamforming,” IEEE Trans. Audio, Speech and Language Pro-cessing, vol. ASSP-35, pp. 1365–1376, 1987.

[24] Simon Haykin, Adaptive Filter Theory, Prentice Hall, NewYork, fourth edition, 2002.

[25] Iain A. McCowan and Herve Bourlard, “Microphone arraypost-filter based on noise field coherence,” IEEE Trans. SpeechAudio Processin, vol. 11, pp. 709–716, 2003.

[26] Tobias Wolff and Markus Buck, “A generalized view on mi-crophone array postfilters,” in Proc. International Workshopon Acoustic Signal Enhancement, Tel Aviv, Israel, 2010.

[27] Claude Marro, Yannick Mahieux, and K. Uwe Simmer, “Anal-ysis of noise reduction and dereverberation techniques basedon microphone arrays with postfiltering,” IEEE Trans. SpeechAudio Process., vol. 6, pp. 240–259, 1998.

[28] Kenichi Kumatani, John McDonough, Dietrich Klakow,Philip N. Garner, and Weifeng Li, “Adaptive beamforming witha maximum negentropy criterion,” IEEE Trans. Audio, Speech,and Language Processing, August 2008.

[29] Kenichi Kumatani, John McDonough, Stefan Schacht, Diet-rich Klakow, Philip N. Garner, and Weifeng Li, “Filter bankdesign based on minimization of individual aliasing terms forminimum mutual information subband adaptive beamforming,”in Proc. IEEE International Conference on Acoustics, Speech,and Signal Processing (ICASSP), Las Vegas, Nevada, U.S.A,2008.

[30] Aapo Hyvarinen, Juha Karhunen, , and Erkki Oja, “Indepen-dent component analysis,” Wiley Inter-Science, 2001.

[31] Kenichi Kumatani, John McDonough, Barbara Rauch, and Di-etrich Klakow, “Maximum negentropy beamforming usingcomplex generalized Gaussian distribution model,” in Proc.ASILOMAR, Pacific Grove, CA, 2010.

[32] Kenichi Kumatani, John McDonough, Barbara Rauch,Philip N. Garner, Weifeng Li, and John Dines, “Maximumkurtosis beamforming with the generalized sidelobe canceller,”in Proc. Interspeech, Brisbane, Australia, September 2008.

[33] Kenichi Kumatani, John McDonough, and Bhiksha Raj, “Max-imum kurtosis beamforming with a subspace filter for distantspeech recognition,” in Proc. ASRU, 2011.

[34] Kenichi Kumatani, John McDonough, Jill Lehman, and Bhik-sha Raj, “Channel selection based on multichannel cross-correlation coefficients for distant speech recognition,” in Proc.HSCMA, Edinburgh, UK, 2011.

[35] Ernst Warsitz, Alexander Krueger, and Reinhold Haeb-Umbach, “Speech enhancement with a new generalized eigen-vector blocking matrix for application in a generalized sidelobecanceller,” in Proc. ICASSP, Las Vegas, NV, U.S.A, 2008.

[36] Kenichi Kumatani, Liang Lu, John McDonough, ArnabGhoshal, and Dietrich Klakow, “Maximum negentropy beam-forming with superdirectivity,” in European Signal ProcessingConference (EUSIPCO), Aalborg, Denmark, 2010.

[37] Jens Meyer and Gary W. Elko, “Spherical microphone arraysfor 3D sound recording,” in Audio Signal Processing for Next–Generation Multimedia Communication Systems, pp. 67–90.Kluwer Academic, Boston, MA, 2004.

[38] Zhiyun Li and Ramani Duraiswami, “Flexible and optimal de-sign of spherical microphone arrays for beamforming,” IEEETrans. Speech Audio Process., vol. 15, pp. 2007, 702–714.

[39] Boaz Rafaely, “Analysis and design of spherical microphonearrays,” IEEE Trans. Speech Audio Process., vol. 13, no. 1, pp.135–143, 2005.

[40] Shefeng Yan, Haohai Sun, U. Peter Svensson, Xiaochuan Ma,and J. M. Hovem, “Optimal modal beamforming for sphericalmicrophone arrays,” IEEE Trans. Audio, Speech and LanguageProcessing, vol. 19, no. 2, pp. 361–371, 2011.

[41] Earl G. Williams, Fourier Acoustics, Academic Press, SanDiego, CA, USA, 1999.

[42] Frank W. J. Olver and L. C. Maximon, “Bessel functions,” inNIST Handbook of Mathematical Functions, Frank W. J. Olver,Daniel W. Lozier, Ronald F. Boisvert, and Charles W. Clark,Eds. Cambridge University Press, New York, NY, 2010.

[43] Heinz Teutsch, Modal Array Signal Processing: Principles andApplications of Acoustic Wavefield Decomposition, Springer,Heidelberg, 2007.