overview intro MGC classifiers example summary
MUSI-6201 — Computational Music AnalysisPart 9.1: Genre Classification
alexander lerch
November 4, 2015
overview intro MGC classifiers example summary
temporal analysisoverview
text book
Chapter 8: Musical Genre, Similarity, and Mood (pp. 151–155)
sources: slides (latex) & Matlab
github repository
lecture contentdefinition of musical genretypical features and feature categoriessimple classifiers and basic classifier properties
overview intro MGC classifiers example summary
temporal analysisoverview
text book
Chapter 8: Musical Genre, Similarity, and Mood (pp. 151–155)
sources: slides (latex) & Matlab
github repository
lecture contentdefinition of musical genretypical features and feature categoriessimple classifiers and basic classifier properties
overview intro MGC classifiers example summary
temporal analysisoverview
text book
Chapter 8: Musical Genre, Similarity, and Mood (pp. 151–155)
sources: slides (latex) & Matlab
github repository
lecture contentdefinition of musical genretypical features and feature categoriessimple classifiers and basic classifier properties
overview intro MGC classifiers example summary
temporal analysisoverview
text book
Chapter 8: Musical Genre, Similarity, and Mood (pp. 151–155)
sources: slides (latex) & Matlab
github repository
lecture contentdefinition of musical genretypical features and feature categoriessimple classifiers and basic classifier properties
overview intro MGC classifiers example summary
musical genre classificationintroduction
one of the oldest research topics in MIR
classic machine learning task
related fields:
speech-music classificationinstrument recognitionartist identificationmusic emotion recognition
overview intro MGC classifiers example summary
musical genre classificationintroduction
one of the oldest research topics in MIR
classic machine learning task
related fields:
speech-music classificationinstrument recognitionartist identificationmusic emotion recognition
overview intro MGC classifiers example summary
musical genre classificationintroduction
one of the oldest research topics in MIR
classic machine learning task
related fields:
speech-music classificationinstrument recognitionartist identificationmusic emotion recognition
overview intro MGC classifiers example summary
musical genre classificationapplications
large music databases:
annotationsorting, browsing, retrieving
recommendation systems
automatic playlist generation
mashup generation
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies
2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song
3 ill-defined genre labels: geographic (indian music), historic(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time
5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: definition
what is musical genre
clusters of musical similarity?
→ hard to answer in general, there are manysystematic problems
1 non-agreement on taxonomies2 genre label scope: song, album, artist, piece of a song3 ill-defined genre labels: geographic (indian music), historic
(baroque), technical (barbershop), instrumentation(symphonic music), usage (christmas songs)
4 taxonomy scalability: genres and subgenres evolve over time5 non-orthogonality: several genres for one piece of music
overview intro MGC classifiers example summary
musical genre classificationgenre: taxonomy examples
Speech
Music
Male
Female
Sports
Disco
Country
Hip Hop
Rock
Blues
Reggae
Pop
Metal
Classical
Jazz
Choir
Orchestra
Piano
String Quartet
Big Band
Cool
Fusion
Piano
Quartet
Swing
Background
Speech
Music
Male
Female
+Background
Classical
Non-Classical
Chamber
Orchestra
Rock
Electro/Pop
Jazz/Blues
Piano
Solo
String Quartet
Other
Symphonic
+Choir
+Soloist
Soft Rock
Hard Rock
Hip Hop
Techno/Dance
Pop
overview intro MGC classifiers example summary
musical genre classificationobservations with humans
1 human classification farfrom perfect: 75–90 % forlimited set of classes
2 for many genres, humansneed only a fraction of asecond to classify
⇒ short time timbre featuressufficient?
plots from1,2
1S. Lippens, J.-P. Martens, T. D. Mulder, et al., “A Comparison of Human and Automatic Musical Genre
Classification,” in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing
(ICASSP), Montreal, 2004.2
R. O. Gjerdingen and D. Perrott, “Scanning the Dial: The Rapid Recognition of Music Genres,” Journal of
New Music Research, vol. 37, no. 2, pp. 93–100, Jun. 2008, 00067, issn: 0929-8215.
overview intro MGC classifiers example summary
musical genre classificationobservations with humans
1 human classification farfrom perfect: 75–90 % forlimited set of classes
2 for many genres, humansneed only a fraction of asecond to classify
⇒ short time timbre featuressufficient?
plots from1,2
1S. Lippens, J.-P. Martens, T. D. Mulder, et al., “A Comparison of Human and Automatic Musical Genre
Classification,” in Proceedings of the International Conference on Acoustics, Speech, and Signal Processing
(ICASSP), Montreal, 2004.2
R. O. Gjerdingen and D. Perrott, “Scanning the Dial: The Rapid Recognition of Music Genres,” Journal of
New Music Research, vol. 37, no. 2, pp. 93–100, Jun. 2008, 00067, issn: 0929-8215.
overview intro MGC classifiers example summary
musical genre classificationoverview
AudioSignal
FeatureExtraction
ClassificationGenreLabel
1 feature extractiondimensionality reductionmeaningful representation
2 classificationmap or convert feature to comprehensible domain
overview intro MGC classifiers example summary
musical genre classificationoverview
AudioSignal
FeatureExtraction
ClassificationGenreLabel
1 feature extractiondimensionality reductionmeaningful representation
2 classificationmap or convert feature to comprehensible domain
overview intro MGC classifiers example summary
musical genre classificationfeature categories
high level similarities?melody, hook lines, bass lines, harmony progressionrhythm & tempostructureinstrumentation & timbre. . .
technical feature categoriestonaltechnicaltimbraltemporalintensity
extracted features should beextractable (not: time envelope in polyphonic signals)relevant (not: pitch chroma for instrument ID)non-redundanthave discriminative power(robust to noise)
overview intro MGC classifiers example summary
musical genre classificationfeature categories
high level similarities?melody, hook lines, bass lines, harmony progressionrhythm & tempostructureinstrumentation & timbre. . .
technical feature categoriestonaltechnicaltimbraltemporalintensity
extracted features should beextractable (not: time envelope in polyphonic signals)relevant (not: pitch chroma for instrument ID)non-redundanthave discriminative power(robust to noise)
overview intro MGC classifiers example summary
musical genre classificationfeature categories
high level similarities?melody, hook lines, bass lines, harmony progressionrhythm & tempostructureinstrumentation & timbre. . .
technical feature categoriestonaltechnicaltimbraltemporalintensity
extracted features should beextractable (not: time envelope in polyphonic signals)relevant (not: pitch chroma for instrument ID)non-redundanthave discriminative power(robust to noise)
overview intro MGC classifiers example summary
musical genre classificationinstantaneous features
spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .
pitch features (tonal):pitch chroma distribution/change, . . .
rhythm features (temporal):onset density, beat histogram features, . . .
statistical features (technical):standard deviation, skewness, zero crossings, . . .
intensity features:level variation, number of “pauses”, . . .
overview intro MGC classifiers example summary
musical genre classificationinstantaneous features
spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .
pitch features (tonal):pitch chroma distribution/change, . . .
rhythm features (temporal):onset density, beat histogram features, . . .
statistical features (technical):standard deviation, skewness, zero crossings, . . .
intensity features:level variation, number of “pauses”, . . .
overview intro MGC classifiers example summary
musical genre classificationinstantaneous features
spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .
pitch features (tonal):pitch chroma distribution/change, . . .
rhythm features (temporal):onset density, beat histogram features, . . .
statistical features (technical):standard deviation, skewness, zero crossings, . . .
intensity features:level variation, number of “pauses”, . . .
overview intro MGC classifiers example summary
musical genre classificationinstantaneous features
spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .
pitch features (tonal):pitch chroma distribution/change, . . .
rhythm features (temporal):onset density, beat histogram features, . . .
statistical features (technical):standard deviation, skewness, zero crossings, . . .
intensity features:level variation, number of “pauses”, . . .
overview intro MGC classifiers example summary
musical genre classificationinstantaneous features
spectral features (timbre):Spectral Centroid, MFCCs, Spectral Flux, . . .
pitch features (tonal):pitch chroma distribution/change, . . .
rhythm features (temporal):onset density, beat histogram features, . . .
statistical features (technical):standard deviation, skewness, zero crossings, . . .
intensity features:level variation, number of “pauses”, . . .
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features
2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
overview intro MGC classifiers example summary
musical genre classificationfeature extraction
1 extract instantaneous features2 compute derived features (derivative, filtered)
3 compute long term features & subfeatures per texturewindow
4 compute subfeatures per file
5 normalize subfeatures
6 (select or) transform subfeatures
7 feature vector → classifier input
mean spectral centroidstdrm
s
music
speech
overview intro MGC classifiers example summary
musical genre classificationlong term features 1/2
derived from beat histogram3
3G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” Transactions on Speech and
Audio Processing, vol. 10, no. 5, pp. 293–302, Jul. 2002, issn: 1063-6676. doi: 10.1109/TSA.2002.800560.
overview intro MGC classifiers example summary
musical genre classificationlong term features 2/2
derived from pitch histogram or pitch chroma4
4G. Tzanetakis, A. Ermolinskyi, and P. Cook, “Pitch Histograms in Audio and Symbolic Music Information
Retrieval,” in Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR), Paris,
2002.
overview intro MGC classifiers example summary
musical genre classificationadditional feature examples
stereo featuresmid channel energy vs. side channel energyspectral channel differences
features at higher semantic levels:
tempo, structure, harmonic complexity, instrumentation
overview intro MGC classifiers example summary
musical genre classificationadditional feature examples
stereo featuresmid channel energy vs. side channel energyspectral channel differences
features at higher semantic levels:
tempo, structure, harmonic complexity, instrumentation
overview intro MGC classifiers example summary
musical genre classificationclassification: general steps
1 define training set: annotated results
2 normalize training set
3 train classifier
4 evaluate classifier with test set
5 (adjust classifier settings, return to 4.)
overview intro MGC classifiers example summary
musical genre classificationclassification: general steps
1 define training set: annotated results
2 normalize training set
3 train classifier
4 evaluate classifier with test set
5 (adjust classifier settings, return to 4.)
overview intro MGC classifiers example summary
musical genre classificationclassification: general steps
1 define training set: annotated results
2 normalize training set
3 train classifier
4 evaluate classifier with test set
5 (adjust classifier settings, return to 4.)
overview intro MGC classifiers example summary
musical genre classificationclassification: general steps
1 define training set: annotated results
2 normalize training set
3 train classifier
4 evaluate classifier with test set
5 (adjust classifier settings, return to 4.)
overview intro MGC classifiers example summary
musical genre classificationclassification: general steps
1 define training set: annotated results
2 normalize training set
3 train classifier
4 evaluate classifier with test set
5 (adjust classifier settings, return to 4.)
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: rules of thumb
training settraining set size vs. number of features
training set too small ⇒ overfittingfeature number too large ⇒ overfitting
training set too noisy ⇒ underfittingtraining set not representative ⇒ bad classificationperformance
classifierpoor classifier ⇒ bad classification performance
different classifier
featurespoor features ⇒ bad classification performance
feature selectionnew, better features
features not normalized ⇒ possibly bad classificationperformance
feature rangefeature meanfeature distribution
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: evaluation
define test set for evaluation
test set different from training setotherwise, same requirements
example: N-fold cross validation1 split training set into N parts (randomly, but preferably
identical number per class)2 select one part as test set3 train the classifier with all observations from remaining N − 1
parts4 compute the classification rate for the test set5 repeat until all N parts have been tested6 overall result: average classification rate
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)
classification: extract test vector and set class to majority ofk nearest reference vectorsclassifier data: all training vectors
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layK
nn
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)classification: extract test vector and set class to majority ofk nearest reference vectors
classifier data: all training vectors
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layK
nn
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)classification: extract test vector and set class to majority ofk nearest reference vectors
k = 3
classifier data: all training vectors
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layK
nn
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)classification: extract test vector and set class to majority ofk nearest reference vectors
k = 3
k = 5
classifier data: all training vectors
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layK
nn
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)classification: extract test vector and set class to majority ofk nearest reference vectors
k = 3
k = 5
k = 7
classifier data: all training vectors
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layK
nn
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: kNN
training: extract reference vectors from training set (keepclass labels)
classification: extract test vector and set class to majority ofk nearest reference vectors
classifier data: all training vectors
overview intro MGC classifiers example summary
musical genre classificationclassifier: GMM
training: build model of each class distribution assuperposition of Gaussian distributions
classification: compute output of each Gaussian and selectclass with highest probability
classifier data: per class per Gaussian: µ and covariance,mixture weight?
overview intro MGC classifiers example summary
musical genre classificationclassifier: GMM
training: build model of each class distribution assuperposition of Gaussian distributionsclassification: compute output of each Gaussian and selectclass with highest probability
classifier data: per class per Gaussian: µ and covariance,mixture weight?
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layG
mm
.m
overview intro MGC classifiers example summary
musical genre classificationclassifier: GMM
training: build model of each class distribution assuperposition of Gaussian distributions
classification: compute output of each Gaussian and selectclass with highest probability
classifier data: per class per Gaussian: µ and covariance,mixture weight?
overview intro MGC classifiers example summary
musical genre classificationclassifier: SVM
training:
map features to high dimensional space
find separating hyperplane (linear classification) throughmaximum distance of support vectors (data points)
classification: apply feature transform and proceed with’linear’ classification
classifier data: support vectors, kernel, kernel parameters htt
ps:
//
en.w
ikip
edia
.org
/w
iki/
Su
pp
ort
vect
orm
ach
ine
overview intro MGC classifiers example summary
musical genre classificationclassifier: SVM
training:
map features to high dimensional space
find separating hyperplane (linear classification) throughmaximum distance of support vectors (data points)
classification: apply feature transform and proceed with’linear’ classification
classifier data: support vectors, kernel, kernel parameters htt
ps:
//
en.w
ikip
edia
.org
/w
iki/
Su
pp
ort
vect
orm
ach
ine
overview intro MGC classifiers example summary
musical genre classificationclassifier: SVM
training:
map features to high dimensional space
find separating hyperplane (linear classification) throughmaximum distance of support vectors (data points)
classification: apply feature transform and proceed with’linear’ classification
classifier data: support vectors, kernel, kernel parameters htt
ps:
//
en.w
ikip
edia
.org
/w
iki/
Su
pp
ort
vect
orm
ach
ine
overview intro MGC classifiers example summary
musical genre classificationresults
classification results depend on training set, test set, andnumber of classes
typical ranges: 10 classes ⇒ 50–80%
note: results vary largely between datasets
ill-defined genre boundariesnon-uniformly distributed classesoverfitting through songs from same album or artist. . .
overview intro MGC classifiers example summary
musical genre classificationresults
classification results depend on training set, test set, andnumber of classes
typical ranges: 10 classes ⇒ 50–80%
note: results vary largely between datasets
ill-defined genre boundariesnon-uniformly distributed classesoverfitting through songs from same album or artist. . .
overview intro MGC classifiers example summary
musical genre classificationresults
classification results depend on training set, test set, andnumber of classes
typical ranges: 10 classes ⇒ 50–80%
note: results vary largely between datasets
ill-defined genre boundariesnon-uniformly distributed classesoverfitting through songs from same album or artist. . .
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification baseline example
1 extract features
2 represent each file with its 2-dimensional feature vector
3 kNN to classify unknown audio files
4 evaluate classification performance
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: features 1/2
for each audio file1 split input signal into (overlapping) blocks2 compute 2 feature series (spectral centroid, RMS)3 aggregate feature series to one value each
mean of Spectral Centroid
µSC =1
N
∑∀n
vSC(n)
standard deviation of RMS
σRMS =
√1
N
∑∀n
(vRMS(n)− µRMS)2
4 represent each file as 2-dimensional vector(µSC, σRMS
)T
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: features 1/2
for each audio file1 split input signal into (overlapping) blocks2 compute 2 feature series (spectral centroid, RMS)3 aggregate feature series to one value each
mean of Spectral Centroid
µSC =1
N
∑∀n
vSC(n)
standard deviation of RMS
σRMS =
√1
N
∑∀n
(vRMS(n)− µRMS)2
4 represent each file as 2-dimensional vector(µSC, σRMS
)T
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: features 1/2
for each audio file1 split input signal into (overlapping) blocks2 compute 2 feature series (spectral centroid, RMS)3 aggregate feature series to one value each
mean of Spectral Centroid
µSC =1
N
∑∀n
vSC(n)
standard deviation of RMS
σRMS =
√1
N
∑∀n
(vRMS(n)− µRMS)2
4 represent each file as 2-dimensional vector(µSC, σRMS
)T
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: features 2/2
mean spectral centroid
stdrm
s
music
speech
ma
tla
bso
urc
e:m
atl
ab
/d
isp
layS
catt
er.m
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: training set
use dataset annotated as speech and music:requirements
large compared to number of featuresrepresentative for use case (diverse)
here:
110 speech files119 music files
extract the features for the dataset
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: results (kNN)
confusion matrix:
speech music # files
speech 93 17 110music 19 100 119
⇒ classification rate:
100 + 93
110 + 119= 84.2%
single feature classification results
Spectral Centroid: 56.7%RMS: 85.1%
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: results (kNN)
confusion matrix:
speech music # files
speech 93 17 110music 19 100 119
⇒ classification rate:
100 + 93
110 + 119= 84.2%
single feature classification results
Spectral Centroid: 56.7%RMS: 85.1%
overview intro MGC classifiers example summary
musical genre classificationspeech/music classification example: results (kNN)
confusion matrix:
speech music # files
speech 93 17 110music 19 100 119
⇒ classification rate:
100 + 93
110 + 119= 84.2%
single feature classification results
Spectral Centroid: 56.7%RMS: 85.1%
overview intro MGC classifiers example summary
summarylecture content
1 name three possible problems in the definition of the groundtruth for genre classification
2 is it possible for genre classifiers to yield better accuracy thanhuman experts
3 list the feature processing steps from audio to the input of theclassifier
overview intro MGC classifiers example summary
summarylecture content
1 name three possible problems in the definition of the groundtruth for genre classification
2 is it possible for genre classifiers to yield better accuracy thanhuman experts
3 list the feature processing steps from audio to the input of theclassifier
overview intro MGC classifiers example summary
summarylecture content
1 name three possible problems in the definition of the groundtruth for genre classification
2 is it possible for genre classifiers to yield better accuracy thanhuman experts
3 list the feature processing steps from audio to the input of theclassifier
Top Related