Intelligent and Noise- Robust Interfaces for MEMS Acoustic Sensors: Smart Microphone.

Center for Auditoryand Acoustic Research

Institute for Systems ResearchUniversity of Maryland

Intelligent and Noise-Robust Interfaces for

MEMS Acoustic Sensors: Smart Microphone



UNIVERSITY OF MARYLAND

Electrical and Computer Engineering & Psychology Departments

Baras, Horiuchi, Krishnaprasad, Moss, Shamma

THE JOHNS HOPKINS UNIVERSITYElectrical and Computer Engineering Department

Andreou, Cauwenberghs, Etienne-Cummings

UNIVERSITY OF SIDNEY Electrical Engineering Department

van Schaik

SIGNAL SYSTEMS CORPORATIONRiddle, Murray

COLLABORATIONSInstitute for Neuroinformatics, ETH

Army Research Laboratory



PROJECT GOALS AND MISSION

Overall MissionFormulate, design, and implement signal processing systems and

technology that can adapt, control, and utilize the noisy MEMS sensorsignals

Focus Area IIIEmbedding and demonstrating the

functional capability of the integratedMEMS/VLSI sensor and signal

processing arrays in a moving roboticvehicle.

Specific Approach

Focus Area IINoise control in MEMS sensor arrays

through design and fabrication ofversatile analog VLSI MEMS

interface and associated featureextraction and analysis stages

Focus Area ICharacterize and integrate

MEMS sensors withaVLSI circuits that detectand receive the signals.



Specific Objectives

Objective 1 Formulate strategies for

interfacing with acoustic MEMS

Objective 2Develop and implement wind-noise and

platform-noise reduction algorithms in VLSI

Objective 3Implement VLSI cochlear frequency analysis

Objective 4Design and fabricate feature extraction

algorithms

Objective 5Feature synthesis and recognition

Objective 6Technology transfer and demonstrations

log f

log f

log f

log f

log u

u log f

Time(ms)

100 200 300 400 500 600 700 800 900 1000

125

250

500

1000

2000

NSL TOOLBOXCortical Decomposition of Sound

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.



Acoustic and Ultrasonic Transducers(AGA, REC)

Prototype and Evaluate Various Types of MEMS Microphones/Speakers

Custom Microphones/Speakers Commercial Transducers



Integrated MEMS Acoustic and Ultrasonic Arrays(AGA, REC)

Prototype and Evaluate Various Types of MEMS Microphone/Speaker Arrays

2D Piezo ArraysCeramics and Polymers

2D MEMS ArraysCapacitive Micro-Membranes



Vision: • A small, low power microphone interface for acoustic sensors that reduces turbulence and vibration induced noise on military

platforms such as battlefield robotics

Polyurethanefoam

windscreen

Mounting plate

1/2”

1”

PrimaryMicrophone

Port

Connector

Secondary Sensor Port

for Wind Sensing Microphones

PreampsaVLSI noise

reduction circuitry



MEMS/VLSI Integration and Prototyping(REC,AGA)

Develop Integrated Processing Electronics for Transducers

Integrated Transducers and Electronics Transduction Electronics



• Approach:– Utilize multi-channel adaptive filtering modules based on aVLSI biomimetic technology

• Analog filter banks with Independent Component Analysis (ICA) and Least Mean Squares (LMS) adaptation

– Incorporate low noise preamps; acoustic and vibration sensors– Develop specification and prototypes – Demonstrate in acoustic duct and installed on unmanned land vehicle

Multi-Resolution

Adaptive Filter

Low noise acoustic signals

Acoustic Sensors

Wind Noise Sensor

Noise and Vibration Sensors



Cochlear Frequency Analysis

• We will design a new silicon cochlea with the following features:– Increased robustness due to 2D design

– Integrated Inner Hair Cell Model

– Reproducible settings of the parameters

Magnitude Response Phase Response



Stochastic Resonance Exploit stochastic resonance (noise-induced

enhancement of spectral power amplification SPA) in conjunction with auditory-inspired (e.g. cochlear) sensor signal processing architectures

+

ExternalWorld

Band-passfilter

Controllednoise generator

circuit

MEMSheater

K

Thresholddetector



Adaptive Filtering and Blind Source Separation(GC, AGA)

Dynamic ICA Array Processor Adaptive Cell

Static and Dynamic ICA (Independent Component Analysis)

Adaptive Noise and Wind Cancellation without Need for Isolated Reference



Early Auditory Models

S1

11w

S2

A

H1

H2

HN

ICA1

ICA2

ICAN

.

.

.

.

.

.

.

.

X1

X2

X1

X2

X1

X2

X1

X2

12w21w22w

Nw1Nw2

11v12v21v22v

Nv1Nv2

Grouping

And

Competitive

Learning

1w2w w

xˆwys==

-De noisingFiltering

Hair cellCochlea LateralInhibition

-Auditory Based Sound Separation

lo g f

lo g f

lo g f

l gf

u

u

ICAlo g f

lo g f

lo g f

l gf

u

u

S1

S2



Cochlear Feature Extraction(AGA, GC)

co

chle

a15

ch

ann

els

ha

ir c

ells

au

dit

ory

ne

rve

AM

FM

Time

TZC

TZC

BM in

BM out

BM out

Energy

(single channel)

Neuromorphic implementation with asynchronous “spiking” outputs



AnalysisCochlear filters

TransductionHair cells

ReductionLateral inhibition

log f

log f

log f

log f

log u

u log f

eardrum cochlea basilar membranefilters

hair cell stages lateral inhibitorynetwork

Time(ms)

100 200 300 400 500 600 700 800 900 1000

125

250

500

1000

2000

Audit ory Spec t rogram

Early Auditory Processing Stages

500Time (ms)

4000

250

/ r i t a w a y /

500Time (ms)

4000

250



• The second filter models the multiscale processing of the signal that happens in the auditory cortex

• A Ripple Analysis Model, using a ripple filter bank, acts on the output of the inner ear to give multiscale spectra of the sound timbre (Wavelet Transform)

Multiresolution Preprocessor: Auditory Filtering

Upward Moving Downward MovingSlow RateCoarse Scale

Slow RateFine Scale

Fast RateFine Scale

Fast RateCoarse Scale

Fast RateFine Scale

Multiresolution cortical filter outputs

Fast RateCoarse Scale

Slow RateFine Scale

Slow RateCoarse Scale



Auditory frequency

scal

e

20 40 60 80 100 120

70

60

50

40

30

20

10

0

Example of multi-resolution representation from cortical module

Auditory Processing of Vehicle Signals: Cortex



Wavelet TSVQ Applied to Acoustic Vehicle Classification

• Objective: a prototype vehicle acoustic signal classification system with low classification error and short search time

• Biologically motivated feature extraction models: cochlear filter banks and A1-cortical wavelet transform

• Vector Quantization (VQ) based classification algorithm. Including learning VQ (LVQ) and tree structured VQ (TSVQ)

Feature extraction system

Classification Result

Acoustic Recording

Preprocessing

Peripheral auditory processing model

VQ based Classification Algorithm

Cortical processing model

Algorithm Flowchart



Acoustic Transient Time-Frequency Analysis(GC, w/ APL)

Segmenter

audio in

Continuous Wavelet Filterbank

32 (freq) X 64 (time)

Time-FrequencyTemplate Correlator

ATP

... 16 (12 used)

Digital Postprocessor

16 (templ.)

32 (freq)

“Shelf” (class 10) “Tub” (class 11)

Models “Ripple” Dynamics of Cells Recorded in Auditory Cortex (Shamma)



Cochlearfilters

NA VLVp

NM/NL(ITD processing)

(ILD processing)

ICc ICx

Cochlearfilters

NA VLVp

NM/NL(ITD processing)

(ILD processing)

ICc ICx

BINAURAL LOCALIZATION

ABL



Stereausis: A Biologically Plausible Binaural Network.

A binaural sound localization system will be developed using 2 silicon cochleas and an aVLSI implementation for ILD and ITD detection.



C ij = z(t; i).z(t;j)

Z(t;i)

log f

Cochlearfilters

hair cellstages

lateralinhibition

Coincidence Matrix

Z(t;j)

TemporalSharpeningA

r(t; x) z(t; x)y(t; x)log f

log f

x=1

x=128

2

1

.5

.25

.125

2

1

.5

.25

.125

200 200

B

2

1

.5

.25

.125

C



BATS



Steerable Range Gauging and Echolocation(REC)

Develop Ranging Signal Processing Algorithms



Massively Parallel Kernel Learning “Machines”(GC)

Parallel vector quantizer

Kernel “machines” subsume LVQ, RBF and SVM classifiers

Locally adaptive, distributed memory

Scalable and modular

Factor 100-10,000 more efficient than CPU or DSP



Integrate-and-Fire Address-Event VLSI Neural Networks(GC & AGA)

Integrate-and-Fire Array Address-Event Transceiver Chip

Scalable, multi-chip architecture for “neural” computations

Address-event routing circuit provides for arbitrary interconnection topologies

Analog-valued synaptic weights are implemented by probabilistically transmitting address-events

Title:chip2.epsCreator:Xcircuit v2.0Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.



Address-Event AsynchronousCommunication and Computation

(AGA, REC, GC)

The multi-chip modular, scalable approach to system integration



New design: An AER cochlea chip

• Currently in fabrication

• 128 output channels

• Both for sonar and audio

• New silicon process (0.35um minimum feature size versus

2.0um)

•AER makes inter-chip communication possible.

•AER allows manipulation of output such that projectivefields are readily implemented.



1-D Address-Event Transceiver with Diode-Capacitors Integrators (AGA)

1-D Address-Event Transceiver Chip

Address Event (AE) transceiver circuit is a modular element for future multi-dimensional communication between neuromorphic chips.

Input AE data is processed by the sigmoid function of the nonlinear diode-capacitor integrators .

The data is retransmitted using the AE protocol with an arbitrated queueing communication system.



• Impact:– Enable effective acoustic surveillance for Future Combat Systems

– Increase by 20 dB the turbulence induced noise rejection of acoustic sensors relative to passive treatments of the same size, using active noise control

– Increase by 20 dB the platform noise rejection of acoustic sensors over existing methods

Demo III Experimental Unmanned Vehicle



The Robots

Microphones

Sonar sensors

Touch sensors

Wireless camera

Speakers

Intelligent and Noise- Robust Interfaces for MEMS Acoustic Sensors: Smart Microphone.

Documents

Transcript of Intelligent and Noise- Robust Interfaces for MEMS Acoustic Sensors: Smart Microphone.