1

Reconfigurable Acceleration of Microphone ArrayAlgorithms for Speech Enhancement

Ka Fai Cedric Yiu, Yao Lu, Xiaoxiang ShiThe Hong Kong Polytechnic University

Chun Hok Ho, Wayne LukImperial College London

Nedelko GrbricBlekinge Institute of Technology

3 Jul 2008ASAP 2008

2

Contributions

1. FPGA-based hardware architecture for acoustic beamformer

calibrated signals subband processing

2. Bitwidth analysis to explore suitable bitwidth of the system

3. FPGA at 175MHz: 42 times faster than software at 3.2GHz

3

Handsfree operation in a noisyenvironment

Speech echoes will appear Disturbing noise, such as:

Engine and fan noise Wind and tire friction

4

Speech extraction with microphone arrays (beamforming)

Microphone arrays allows us to use spatial selectivity (directional hearing)

Microphones separated in space can suppress unwanted locations while passing signals from wanted locations

5

Narrowband beamforming principle

Spatial aliasing occurs if the distance

between microphones exceeds half

signal wavelength

6

Drawbacks with narrowband beamformers

Disturbing noise directions are not especially taken into consideration

Microphone placement must be accurate (or carefully calibrated)

Temporal information is not taken into consideration

7

Broadband beamformer structure

FIR-filter

FIR-filter

FIR-filter

FIR-filter

FIR-filter

FIR-filter

Angle of hearing

Angle of hearing

Microphones

Output

By choosing appropriate FIR filters the angle of hearing can be adjusted for

each frequency

8

Adaptive beamforming

In order to track variations in the noise situation and to avoid expensive calibration procedures an adaptive scheme is used

It consists of two phases Calibration mode Operation mode

9

Phase 1: Calibration mode – Select desired location

Speaker says a short sentence in a silent car

Microphones

Estimate covariance matrices

Determine coefficients for each filter

10

Phase 2: Operation mode – Speech extraction while driving

Microphones

Adapting

Beamformer

Output

Speech

11

1. FPGA-based beamformer

Low non-recurring engineering cost Architecture exploration Relatively easy to adapt new algorithms Adjust filter length in different environment Embedded processor to produce a system-

on-a-chip (SoC) solution software speech recogniser

12

Comparison between subband and time-domain implementations

2)]2([time ILILP

22 )]ML(I

MLM[IfreqP

(#channels I= 6)

13

Subband beamformer structure

Angle of hearing

Angle of hearing Mul

ticha

nnel

Subb

and

tran

sfor

mat

ion

Each branch

#I Subband signals

#K Beamformers

Sing

le-c

hann

elSu

bban

dR

econ

stru

ctio

n

#I M icrophones

O utp ut

x1(n )

x2(n )

x3(n )

x4(n )

x I (n )

y(n )

w

w

w

w

w ( K - 1 )

(3 )

( 2 )

(1)

( 0 )

14

FPGA-based architecture

15

DMA interface

16

2. Bitwidth optimisation

Software implementation is based on MATLAB

Floating point implementation is expensive on FPGA

Fixed point implementation Integer width: avoid overflow Saturation arithmetic: overflow protection Fraction width: sufficient accuracy relative to

floating point

17

Optimisation results

Input signal Noise signal

Double precision floating point output Optimised fixed point output

18

3. Implementation results

Target FPGA: Xilinx Virtex 4 SX55 Slice used: 5937 (12%) DSP48 used: 72 (14%) Block RAM used: 8 (2%) Operating frequency: 175MHz Processing rate: 43k sample per second Pentium 4 3.2GHz processing rate: 7.2k sps 6 times faster for 1 instance

19

Multiple instances

Scalable by embedding more beamformer instances Useful in the case where more than one signal

sources in a system 7 instances can achieve 42 times speed up while

running at 175MHz

Instances Frequency (MHz) Slices DSP48 Speed up

1 175 12% 14% 6.0

3 175 36% 42% 17.9

5 175 60% 70% 29.8

7 175 84% 98% 41.7

20

Future work

Implement partial reconfiguration on the filter length based on the change of the environment.

Embed a speech recogniser as a performance indicator to adjust the filter length.

Optimise the power consumption by reducing the number of glitches in the system.

21

Conclusion

Two different beamforming algorithms have represented Narrowband and broadband

Subband processing FPGA architecture

Processing element DMA interface

Bitwidth optimisation 7 instances of beamformer is 42 time faster