Computational Spectro-temporal Auditory Model

Taishih ChiJune 29, 2003

Auditory Model

• Overview – two stage processing• Model description and formulation• Examples of representations• Reconstruction from model output

representations• Discussions

SpectralEstimation

Early Auditory

SpectralAnalysis

Primary Cortex (A1)

SoundAuditorySpectrum

CorticalRepresentation

Auditory ModelOverview

• Temporal dynamics reduction

• Monaural model• Two stage functional

model– Early stage

(spectrum estimation)

– Cortical stage (spectrum analysis)

Early stage Mathematical Formulation

Early Stage MATLAB Implementation

Matlab ToolBox Usage:yfinal = wav2aud(s, [frmlen, tc, fac, shft], filt);

s : acoustic input signal

yfinal: auditory spectrogram; N(time) x M(freq.)

CF = 440 * 2 .^ ((-31:97)/24 + shft);

Cortical stage Spectrotemporal Receptive Field

4

0.125

4

0.125

4

0.125

4

0.125

4 4

0.125

CF

requ

ency

(kH

z)F

requ

ency

(kH

z)

Fre

quen

cy (

kHz)

Fre

quen

cy (

kHz)

Fre

quen

cy (

kHz)

Fre

quen

cy (

kHz)

Time (ms) Time (ms) Time (ms)

Time (ms) Time (ms) Time (ms)

250 250 250

250250250

0 0 0

0000.125

D E F

BA

(a)

Time (ms)

Lo

g.

Fre

qu

en

cy

Downward; :1 cyc/oct, :4 Hz

500 1000

0.25 CF

0.5 CF

1 CF

2 CF

4 CF

(b)

-1.25 0 1.25

0

Log. Frequency (octave)

hs

0 1 2 3 4 5

0

Time (sec)

ht

Cortical stage Model Implementation

Cortical stage Mathematical Formulation

where

then the spectrotemporal cortical response:

Cortical stage Mathematical Formulation (cont’d)Consider the complex wavelet transform

where

then

Cortical stage Cortical Representation of Speech

Fre

qu

en

cy (

Hz)

Time (ms)100 200 300 400 500 600 700 800 900 1000

125

250

500

1000

2000

Multiresolution Cortical Filters and Outputs

Upward Downward

Slow RateCoarse Scale

Slow RateFine Scale

Fast RateCoarse Scale

Fast RateFine Scale

Slow RateCoarse Scale

Slow RateFine Scale

Fast RateCoarse Scale

Fast RateFine Scale

Cortical Magnitude Representation of Speech

Fre

quen

cy (

Hz)

Time (ms)

Auditory Spectrogram

100 200 300 400 500 600 700 800 900 1000

125

250

500

1000

2000

Multiresolution Cortical Filters and Outputs

Upward Downward

Slow RateCoarse Scale

Slow RateFine Scale

Fast RateCoarse Scale

Fast RateFine Scale

Slow RateCoarse Scale

Slow RateFine Scale

Fast RateCoarse Scale

Fast RateFine Scale

Cortical Stage MATLAB Implementation

Matlab ToolBox Usage:cr = aud2cor(y, para1, rv, sv, fname, DISP);

cr: 4D cortical representation (scale-rate(up-down)-time-freq.)

y : auditory spectrogram, N(time) x M(freq.)

para1 = [paras FULLT FULLX BP],paras:see WAV2AUD

FULLT (FULLX): fullness of temporal (spectral) margin.

BP: pure bandpass indicator.

rv: rate vector in Hz, e.g., 2.^(1:.5:5).

sv: scale vector in cyc/oct, e.g., 2.^(-2:.5:3).