MUSI-6201 | Computational Music Analysis · MUSI-6201 | Computational Music Analysis Part 9.1:...

overview intro MGC classifiers example summary

MUSI-6201 — Computational Music AnalysisPart 9.1: Genre Classification

alexander lerch

November 4, 2015

temporal analysisoverview

text book

Chapter 8: Musical Genre, Similarity, and Mood (pp. 151–155)

sources: slides (latex) & Matlab

github repository

lecture contentdefinition of musical genretypical features and feature categoriessimple classifiers and basic classifier properties

text book

github repository

text book

github repository

text book

github repository

musical genre classificationintroduction

one of the oldest research topics in MIR

classic machine learning task

related fields:

speech-music classificationinstrument recognitionartist identificationmusic emotion recognition

related fields:

musical genre classificationapplications

large music databases:

annotationsorting, browsing, retrieving

recommendation systems

automatic playlist generation

mashup generation

musical genre classificationgenre: definition

what is musical genre

clusters of musical similarity?

→ hard to answer in general, there are manysystematic problems

1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic

(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)

4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music

1 non-agreement on taxonomies

2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic

1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song

3 ill-defined genre labels: geographic (indian music), historic(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)

4 taxonomy scalability: genres and subgenres evolve over time

5 non-orthogonality: several genres for one piece of music

musical genre classificationgenre: taxonomy examples

Speech

Female

Sports

Country

Hip Hop

Reggae

Classical

Orchestra

String Quartet

Big Band

Fusion

Quartet

Background

Speech

Female

+Background

Classical

Non-Classical

Chamber

Orchestra

Electro/Pop

Jazz/Blues

String Quartet

Symphonic

+Choir

+Soloist

Soft Rock

Hard Rock

Hip Hop

Techno/Dance

musical genre classificationobservations with humans

1 human classification farfrom perfect: 75–90 % forlimited set of classes

2 for many genres, humansneed only a fraction of asecond to classify

⇒ short time timbre featuressufficient?

plots from1,2

1S. Lippens, J.-P. Martens, T. D. Mulder, et al., “A Comparison of Human and Automatic Musical Genre

Classification,” in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing

(ICASSP), Montreal, 2004.2

R. O. Gjerdingen and D. Perrott, “Scanning the Dial: The Rapid Recognition of Music Genres,” Journal of

New Music Research, vol. 37, no. 2, pp. 93–100, Jun. 2008, 00067, issn: 0929-8215.

musical genre classificationobservations with humans

1 human classification farfrom perfect: 75–90 % forlimited set of classes

2 for many genres, humansneed only a fraction of asecond to classify

⇒ short time timbre featuressufficient?

plots from1,2

1S. Lippens, J.-P. Martens, T. D. Mulder, et al., “A Comparison of Human and Automatic Musical Genre

Classification,” in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing

(ICASSP), Montreal, 2004.2

R. O. Gjerdingen and D. Perrott, “Scanning the Dial: The Rapid Recognition of Music Genres,” Journal of

New Music Research, vol. 37, no. 2, pp. 93–100, Jun. 2008, 00067, issn: 0929-8215.

musical genre classificationoverview

AudioSignal

FeatureExtraction

ClassificationGenreLabel

1 feature extractiondimensionality reductionmeaningful representation

2 classificationmap or convert feature to comprehensible domain

musical genre classificationoverview

AudioSignal

FeatureExtraction

ClassificationGenreLabel

1 feature extractiondimensionality reductionmeaningful representation

2 classificationmap or convert feature to comprehensible domain

musical genre classificationfeature categories

high level similarities?melody, hook lines, bass lines, harmony progressionrhythm & tempostructureinstrumentation & timbre. . .

technical feature categoriestonaltechnicaltimbraltemporalintensity

extracted features should beextractable (not: time envelope in polyphonic signals)relevant (not: pitch chroma for instrument ID)non-redundanthave discriminative power(robust to noise)

musical genre classificationinstantaneous features

spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .

pitch features (tonal):pitch chroma distribution/change, . . .

rhythm features (temporal):onset density, beat histogram features, . . .

statistical features (technical):standard deviation, skewness, zero crossings, . . .

intensity features:level variation, number of “pauses”, . . .

musical genre classificationfeature extraction

1 extract instantaneous features

2 compute derived features (derivative, filtered)

3 compute long term features & subfeatures per texturewindow

4 compute subfeatures per file

5 normalize subfeatures

6 (select or) transform subfeatures

7 feature vector → classifier input

1 extract instantaneous features2 compute derived features (derivative, filtered)

mean spectral centroidstdrm

speech

musical genre classificationlong term features 1/2

derived from beat histogram3

3G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” Transactions on Speech and

Audio Processing, vol. 10, no. 5, pp. 293–302, Jul. 2002, issn: 1063-6676. doi: 10.1109/TSA.2002.800560.

musical genre classificationlong term features 2/2

derived from pitch histogram or pitch chroma4

4G. Tzanetakis, A. Ermolinskyi, and P. Cook, “Pitch Histograms in Audio and Symbolic Music Information

Retrieval,” in Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR), Paris,

musical genre classificationadditional feature examples

stereo featuresmid channel energy vs. side channel energyspectral channel differences

features at higher semantic levels:

tempo, structure, harmonic complexity, instrumentation

musical genre classificationadditional feature examples

stereo featuresmid channel energy vs. side channel energyspectral channel differences

features at higher semantic levels:

tempo, structure, harmonic complexity, instrumentation

musical genre classificationclassification: general steps

1 define training set: annotated results

2 normalize training set

3 train classifier

4 evaluate classifier with test set

5 (adjust classifier settings, return to 4.)

3 train classifier

musical genre classificationclassifier: rules of thumb

training settraining set size vs. number of features

training set too small ⇒ overfittingfeature number too large ⇒ overfitting

training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance

classifierpoor classifier ⇒ bad classification performance

different classifier

featurespoor features ⇒ bad classification performance

feature selectionnew, better features

features not normalized ⇒ possibly bad classificationperformance

feature rangefeature meanfeature distribution

musical genre classificationclassifier: evaluation

define test set for evaluation

test set different from training setotherwise, same requirements

example: N-fold cross validation1 split training set into N parts (randomly, but preferably

identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1

parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate

musical genre classificationclassifier: kNN

training: extract reference vectors from training set (keepclass labels)

classification: extract test vector and set class to majority ofk nearest reference vectorsclassifier data: all training vectors

training: extract reference vectors from training set (keepclass labels)classification: extract test vector and set class to majority ofk nearest reference vectors

classifier data: all training vectors

training: extract reference vectors from training set (keepclass labels)

classification: extract test vector and set class to majority ofk nearest reference vectors

musical genre classificationclassifier: GMM

training: build model of each class distribution assuperposition of Gaussian distributions

classification: compute output of each Gaussian and selectclass with highest probability

classifier data: per class per Gaussian: µ and covariance,mixture weight?

training: build model of each class distribution assuperposition of Gaussian distributionsclassification: compute output of each Gaussian and selectclass with highest probability

training: build model of each class distribution assuperposition of Gaussian distributions

classification: compute output of each Gaussian and selectclass with highest probability

musical genre classificationclassifier: SVM

training:

map features to high dimensional space

find separating hyperplane (linear classification) throughmaximum distance of support vectors (data points)

classification: apply feature transform and proceed with’linear’ classification

classifier data: support vectors, kernel, kernel parameters htt

training:

musical genre classificationresults

classification results depend on training set, test set, andnumber of classes

typical ranges: 10 classes ⇒ 50–80%

note: results vary largely between datasets

ill-defined genre boundariesnon-uniformly distributed classesoverfitting through songs from same album or artist. . .

musical genre classificationspeech/music classification baseline example

1 extract features

2 represent each file with its 2-dimensional feature vector

3 kNN to classify unknown audio files

4 evaluate classification performance

musical genre classificationspeech/music classification example: features 1/2

for each audio file1 split input signal into (overlapping) blocks2 compute 2 feature series (spectral centroid, RMS)3 aggregate feature series to one value each

mean of Spectral Centroid

µSC =1

∑∀n

vSC(n)

standard deviation of RMS

σRMS =

∑∀n

(vRMS(n)− µRMS)2

4 represent each file as 2-dimensional vector(µSC, σRMS

µSC =1

∑∀n

vSC(n)

σRMS =

∑∀n

(vRMS(n)− µRMS)2

µSC =1

∑∀n

vSC(n)

σRMS =

∑∀n

(vRMS(n)− µRMS)2

mean spectral centroid

speech

musical genre classificationspeech/music classification example: training set

use dataset annotated as speech and music:requirements

large compared to number of featuresrepresentative for use case (diverse)

110 speech files119 music files

extract the features for the dataset

musical genre classificationspeech/music classification example: results (kNN)

confusion matrix:

speech music # files

speech 93 17 110music 19 100 119

⇒ classification rate:

100 + 93

110 + 119= 84.2%

single feature classification results

Spectral Centroid: 56.7%RMS: 85.1%

confusion matrix:

speech 93 17 110music 19 100 119

100 + 93

110 + 119= 84.2%

confusion matrix:

speech 93 17 110music 19 100 119

100 + 93

110 + 119= 84.2%

summarylecture content

1 name three possible problems in the definition of the groundtruth for genre classification

2 is it possible for genre classifiers to yield better accuracy thanhuman experts

3 list the feature processing steps from audio to the input of theclassifier

MUSI-6201 | Computational Music Analysis · MUSI-6201 | Computational Music Analysis Part 9.1:...

Documents

Transcript of MUSI-6201 | Computational Music Analysis · MUSI-6201 | Computational Music Analysis Part 9.1:...