Direction of Arrival Based Spatial Covariance Model For ...

Direction ofArrival Based

SpatialCovarianceModel For

Blind SourceSeparation[1]

BadriPatro,GokulKrishnan &

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

SignalRepresentation

ConvolutiveMixing Model inSpatialCovarianceDomain

ProposedMethod

Basic Idea

DoA Kernels

Time-difference ofArrival(TDoA)

Superpositionof DoA Kernels

CNMF Modelfor SCMObservations

CNMF ModelAlgo

ReconstructionandClustering

Direction ofWeightsClustering

Reference

Direction of Arrival Based Spatial CovarianceModel For Blind Source Separation[1]

Badri Patro,Gokul Krishnan & Phani Reddy

EE627A: Project Presentation

Guidance: Prof. Rajesh M. HegdeIndian Institute of Technology, Kanpur

April 10, 2016

Badri, Gokul and Phani DOA based SCM for BSS April 10, 2016 1 / 29

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Table of contents

1 IntroductionProblem Statementliterature Survey

Matrix factorization

2 Back Ground WorkSource Mixing ModelsSignal RepresentationConvolutive Mixing Model in Spatial Covariance Domain

3 Proposed MethodBasic IdeaDoA Kernels

Time-difference of Arrival(TDoA)Superposition of DoA Kernels

CNMF Model for SCM ObservationsCNMF Model Algo

4 Reconstruction and ClusteringDirection of Weights Clustering

5 Reference

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Introduction Problem Statement

Problem Statement

This presentation will address problem of sourceseparation from multichannel microphone array.Given a mixture of sound signal received from multiplesource and multiple microphone.

our task is to separate out different sources from mixturesignal.

Ex. Given a mixture of sound signal received from 2source and 2 microphone.

our task is to separate out 2 sources from mixture signal.

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Problem Statement

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

BSS(@Clifford)

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Introduction literature Survey

ICA(Sawada et al.,2004)[5] and (Hyvrinen etal.,2001)[6]*1

frequency-domain BSS involves a permutationproblem:

the permutation ambiguity of ICA in each frequency binshould be aligned so that a separated signal in thetime-domain contains frequency components of the samesource signal.

1These are ref No of My cited paperBadri, Gokul and Phani DOA based SCM for BSS April 10, 2016 5 / 29

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

DOA(Ikram et al.)[7]and(Nesta et al.)[9]*1

Source permutation is usually solved based on timedifference of arrival (TDoA) interpretation of ICAmixing parameters.

Phase differences become ambiguous when thefrequency exceeds the spatial aliasing limit, whichcorresponds to a wavelength greater than half of themicrophone spacing.

As a result, the TDoAs cannot be directly utilized insolving the permutation problem for high frequencies.

TDoA with the help of DUET [10] and binwiseclustering [11].

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

PCA and SVD (@Iyad Batal)

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

What can matrix represent

System of equations

User rating matrix

ImageMatrix structure in graph theory

Adjacent matrixDistance matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

System of equations

User rating matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

System of equations

User rating matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

System of equations

User rating matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

System of equations

User rating matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

System of equations

User rating matrix

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

matrix factorization methods

Matrix factorization methods

LU decomposition

Solving system of equations

Singular Value Decomposition(SVD)Low rank matrix approximationPseudo-inverse

Probabilistic Matrix Factorization(PMF)Recommendation system

Non-negative Matrix Factorization(NMF)Learning the parts of objects

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

LU decomposition

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF (Paatero and Tapper, 1994)

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF (@Author)*1

1Credit Goes to AuthorBadri, Gokul and Phani DOA based SCM for BSS April 10, 2016 11 / 29

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF (@Author)*1

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF (Paatero and Tapper, 1994)

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF ([12,14,15,16,17,20])*1

NMF have been proposed for separation of soundsources both with single and multichannel mixtures.

In the NMF separation framework the spatialproperties of the sources can be modeled using aspatial covariance matrix (SCM) for each source ateach STFT frequency bin [18][22].

Such extensions are hereafter referred to ascomplex-valued NMF (CNMF).

SCM denotes the mixing of the sources by magnitudeand phase differences between the recorded channels,and is not dependent on the absolute phase of thesource signal.

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF ([12,14,15,16,17,20])*1

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF ([12,14,15,16,17,20])*1

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

NMF ([12,14,15,16,17,20])*1

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Back Ground Work Source Mixing Models

Source Mixing Models

Array capture consists of a mixture of sound sourcesconvolved with their spatial responses.

xm (t) =K∑

hmk (τ) sk (t − τ)

Which can be approximated as,

Xil ≈K∑

hiksik =K∑

where ,xil of the capture Xil = [Xil1, . . . .,XilM ]

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

xm (t) =K∑

Xil ≈K∑

hiksik =K∑

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

xm (t) =K∑

Xil ≈K∑

hiksik =K∑

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Back Ground Work Signal Representation

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

numbered lists

SCM calculated at each time-frequency point.

Magntiude-square root version of array capture.

Abs phase of signal transformed to phase differencebetween microphone pair

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

numbered lists

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

numbered lists

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Back Ground Work Convolutive Mixing Model in Spatial Covariance Domain

numbered lists with single pauses

1 Mixing in Spatial Domain.

2 defined by, replacing each term by its covariancecounter-part .

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Proposed Method Basic Idea

Basic Idea

Basic Idea:

BSS via estimation of source SCM of STFT of mixturesignal.

SCM model is combined with a parameter estimationis formulated in a complex-valued non-negative matrixfactorization (CNMF) framework

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Proposed Method Basic Idea

Basic Idea

Block diagram of Proposed Algorithm:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Proposed Method DoA Kernels

Time-difference of Arrival:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Time-difference of Arrival:

Figure: Array geometry consisting of two microphones m and n asseen from above, source azimuth angle given as θ.

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Superposition of DoA Kernels:

DoA kernels for each look direction O and at eachfrequency I is denoted by Wio

source spatial image S = H ∗ S

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Proposed Method CNMF Model for SCM Observations

CNMF Model for SCM Observations

CNMF Model for SCM Observations:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Proposed Method CNMF Model Algo

CNMF Cost Function:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Algorithm Updates for the Non-negativeParameters:

Algorithm Updates for the Non-negative Parameters:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Algorithm Updates for the SCM Model Parameters

Algorithm Updates for the SCM Model Parameters:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Algorithm Implementation:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Reconstruction and Clustering Direction of Weights Clustering

K mean:

Apply k-means clustering on the spatial weights zko ,

No of cluster is equal to the number of sound sources

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

K mean:

Apply k-means clustering on the spatial weights zko ,

No of cluster is equal to the number of sound sources

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

Source Reconstruction:

Phani Reddy

Introduction

ProblemStatement

literatureSurvey

Matrixfactorization

Back GroundWork

Source MixingModels

ProposedMethod

Basic Idea

DoA Kernels

CNMF ModelAlgo

Reference

[1] J. Nikunen., ‘‘Direction of Arrival Based Spatial Covariance Mode forBlind Sound Source Separation”,IEEE/ACM trans.., vol. 22, no. 3, p.727, 2014.

[2] A. Ozerov. and C. Fevotte.,‘‘Multichannel nonnegativematrixfactorization in convolutive mixtures for audio source separation”,IEEETrans. Audio, Speech, Lang. Process., vol. 18, no. 3, pp. 550563, Mar.2010.

[3] H. Sawada, H. Kameoka, S. Araki, and N. Ueda,, ‘‘New formulationsand efficient algorithms for multichannel nmf,”,in Proc. IEEE WorkshopApplicat. Signal Process. Audio Acoust. (WASPAA),pp. 153156, 2011.

[4] H. Sawada, H. Kameoka, S. Araki, and N. Ueda,‘‘Multichannelextensions of non-negative matrix factorization with complex-valueddata.”,IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 5, pp.971982, May 2013.

[5] H Sawada, R Mukai, S Araki, S Makino,‘‘A robust and precise methodfor solving the permutation problem of frequency-domain blind sourceseparation”,Speech and Audio Processing, IEEE Transactions on, 2004.

Direction of Arrival Based Spatial Covariance Model For ...

Documents

Transcript of Direction of Arrival Based Spatial Covariance Model For ...