MUSI-6201 | Computational Music Analysis...inference meta data feature extraction dimensionality...
Transcript of MUSI-6201 | Computational Music Analysis...inference meta data feature extraction dimensionality...
overview intro content ACA summary course outline
MUSI-6201 — Computational Music AnalysisPart 2: Introduction
alexander lerch
November 4, 2015
overview intro content ACA summary course outline
introductionoverview
text bookChapter 1: Introduction (pp. 1–6)
sources: slides (latex) & Matlab
github repository
lecture contentwhat is audio content analysis?what are typical applications?what is audio content?what are the processing blocks of a typical ACA system?
overview intro content ACA summary course outline
introductionoverview
text bookChapter 1: Introduction (pp. 1–6)
sources: slides (latex) & Matlab
github repository
lecture contentwhat is audio content analysis?what are typical applications?what is audio content?what are the processing blocks of a typical ACA system?
overview intro content ACA summary course outline
introductionoverview
text bookChapter 1: Introduction (pp. 1–6)
sources: slides (latex) & Matlab
github repository
lecture contentwhat is audio content analysis?what are typical applications?what is audio content?what are the processing blocks of a typical ACA system?
overview intro content ACA summary course outline
introductionoverview
text bookChapter 1: Introduction (pp. 1–6)
sources: slides (latex) & Matlab
github repository
lecture contentwhat is audio content analysis?what are typical applications?what is audio content?what are the processing blocks of a typical ACA system?
overview intro content ACA summary course outline
introductionoverview
text bookChapter 1: Introduction (pp. 1–6)
sources: slides (latex) & Matlab
github repository
lecture contentwhat is audio content analysis?what are typical applications?what is audio content?what are the processing blocks of a typical ACA system?
overview intro content ACA summary course outline
introductionaudio content analysis — terminology
goalextract information about the content of audio data
terminologymusic information retrieval (MIR):
analysis and retrieval of music databoth audio and symbolic data
machine listening & computer audition
focus on the recognition and understanding of music
computational auditory scene analysis (CASA)
focus on human perception & cognition, understanding of theauditory scene
overview intro content ACA summary course outline
introductionaudio content analysis — terminology
goalextract information about the content of audio data
terminologymusic information retrieval (MIR):
analysis and retrieval of music databoth audio and symbolic data
machine listening & computer audition
focus on the recognition and understanding of music
computational auditory scene analysis (CASA)
focus on human perception & cognition, understanding of theauditory scene
overview intro content ACA summary course outline
introductionaudio content analysis — research field
interdisciplinarydigital signal processingmachine learning / data miningmusicologypsycho-acoustics. . .
communityISMIR: ismir.net
annual conferencescumulative list of conference papersISMIR-Community mailing listMIREX: MIR Evaluation eXchange
related publicationsconferences: ICASSP, ICME, SMC, DAFx, ACM MM, . . .journals: TASLP, Computer Music, JNMR, JAES, . . .
overview intro content ACA summary course outline
introductionaudio content analysis — research field
interdisciplinarydigital signal processingmachine learning / data miningmusicologypsycho-acoustics. . .
communityISMIR: ismir.net
annual conferencescumulative list of conference papersISMIR-Community mailing listMIREX: MIR Evaluation eXchange
related publicationsconferences: ICASSP, ICME, SMC, DAFx, ACM MM, . . .journals: TASLP, Computer Music, JNMR, JAES, . . .
overview intro content ACA summary course outline
introductionaudio content analysis — research field
interdisciplinarydigital signal processingmachine learning / data miningmusicologypsycho-acoustics. . .
communityISMIR: ismir.net
annual conferencescumulative list of conference papersISMIR-Community mailing listMIREX: MIR Evaluation eXchange
related publicationsconferences: ICASSP, ICME, SMC, DAFx, ACM MM, . . .journals: TASLP, Computer Music, JNMR, JAES, . . .
overview intro content ACA summary course outline
introductionapplications
organization in large databases
search & retrieval, classification, similarity
interfaces to search and retrieval
fingerprinting, query-by-humming systems
music visualizationsymbolic (bars, harmony, score, . . . ), similarity mappings
adaptive processingadaptive effect parametrization or algorithm selection
adaptive interactionplaylist generation, recommendation
overview intro content ACA summary course outline
introductionapplications
organization in large databases
search & retrieval, classification, similarity
interfaces to search and retrieval
fingerprinting, query-by-humming systems
music visualizationsymbolic (bars, harmony, score, . . . ), similarity mappings
adaptive processingadaptive effect parametrization or algorithm selection
adaptive interactionplaylist generation, recommendation
overview intro content ACA summary course outline
introductionapplications
organization in large databases
search & retrieval, classification, similarity
interfaces to search and retrieval
fingerprinting, query-by-humming systems
music visualizationsymbolic (bars, harmony, score, . . . ), similarity mappings
adaptive processingadaptive effect parametrization or algorithm selection
adaptive interactionplaylist generation, recommendation
overview intro content ACA summary course outline
introductionapplications
organization in large databases
search & retrieval, classification, similarity
interfaces to search and retrieval
fingerprinting, query-by-humming systems
music visualizationsymbolic (bars, harmony, score, . . . ), similarity mappings
adaptive processingadaptive effect parametrization or algorithm selection
adaptive interactionplaylist generation, recommendation
overview intro content ACA summary course outline
introductionapplications
organization in large databases
search & retrieval, classification, similarity
interfaces to search and retrieval
fingerprinting, query-by-humming systems
music visualizationsymbolic (bars, harmony, score, . . . ), similarity mappings
adaptive processingadaptive effect parametrization or algorithm selection
adaptive interactionplaylist generation, recommendation
overview intro content ACA summary course outline
introduction(commercial) examples
recommendation, playlist generation
fingerprinting
score following
(multi-) pitch detection
overview intro content ACA summary course outline
introduction(commercial) examples
recommendation, playlist generation
fingerprinting
score following
(multi-) pitch detection
overview intro content ACA summary course outline
introduction(commercial) examples
recommendation, playlist generation
fingerprinting
score following
(multi-) pitch detection
overview intro content ACA summary course outline
introduction(commercial) examples
recommendation, playlist generation
fingerprinting
score following
(multi-) pitch detection
overview intro content ACA summary course outline
audio contentsources
what are the sources of (musical) audio content?
1 score:definition of musical ideas“blue-print” of the musicexamples: melody, key, harmony, rhythmic patterns, . . .
2 performance:unique acoustic renditioninformation in the score is interpreted, modified, added toexamples: (micro-)tempo, dynamics, intonation, . . .
3 production:aesthetic choicesediting & processingexamples: sound quality (EQ, microphone positioning),changes in timing and pitch
overview intro content ACA summary course outline
audio contentsources
what are the sources of (musical) audio content?
1 score:definition of musical ideas“blue-print” of the musicexamples: melody, key, harmony, rhythmic patterns, . . .
2 performance:unique acoustic renditioninformation in the score is interpreted, modified, added toexamples: (micro-)tempo, dynamics, intonation, . . .
3 production:aesthetic choicesediting & processingexamples: sound quality (EQ, microphone positioning),changes in timing and pitch
overview intro content ACA summary course outline
audio contentsources
what are the sources of (musical) audio content?
1 score:definition of musical ideas“blue-print” of the musicexamples: melody, key, harmony, rhythmic patterns, . . .
2 performance:unique acoustic renditioninformation in the score is interpreted, modified, added toexamples: (micro-)tempo, dynamics, intonation, . . .
3 production:aesthetic choicesediting & processingexamples: sound quality (EQ, microphone positioning),changes in timing and pitch
overview intro content ACA summary course outline
audio contentsources
what are the sources of (musical) audio content?
1 score:definition of musical ideas“blue-print” of the musicexamples: melody, key, harmony, rhythmic patterns, . . .
2 performance:unique acoustic renditioninformation in the score is interpreted, modified, added toexamples: (micro-)tempo, dynamics, intonation, . . .
3 production:aesthetic choicesediting & processingexamples: sound quality (EQ, microphone positioning),changes in timing and pitch
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio contenttechnical categories
audio content can be structured into 5 technical fundamentalcategories:
1 timbral: related to sound quality
examples: instrument(ation), playing technique, venue, audioprocessing, . . .
2 intensity-related: related to musical dynamics
examples: accents, loudness, . . .
3 tonal: related to pitch
examples: melody, chords, intonation, vibrato, . . .
4 temporal: related to rhythm and tempo
examples: timing, meter, rhythmic patterns, . . .
5 statistical & technical: related to signal properties
examples: amplitude distribution, number of zero crossings,. . .
overview intro content ACA summary course outline
audio content analysissystem overview
audiosignal
featureextraction
decision,interpretation,classification,
inference
metadata
feature extractiondimensionality reductionmeaningful representation
classificationmap or convert feature tocomprehensible domain
overview intro content ACA summary course outline
audio content analysissystem overview
audiosignal
featureextraction
decision,interpretation,classification,
inference
metadata
feature extractiondimensionality reductionmeaningful representation
classificationmap or convert feature tocomprehensible domain
overview intro content ACA summary course outline
audio content analysissystem overview
audiosignal
featureextraction
decision,interpretation,classification,
inference
metadata
feature extractiondimensionality reductionmeaningful representation
classificationmap or convert feature tocomprehensible domain
overview intro content ACA summary course outline
summarylecture content
what is audio content?
what are the technical categories of interest?
what are the typical processing blocks of an ACA system?
overview intro content ACA summary course outline
summarylecture content
what is audio content?
what are the technical categories of interest?
what are the typical processing blocks of an ACA system?
overview intro content ACA summary course outline
summarylecture content
what is audio content?
what are the technical categories of interest?
what are the typical processing blocks of an ACA system?
overview intro content ACA summary course outline
course outlineoverview 1/2
1 fundamentalsdigital audio signalsconvolution & block based processingFourier transform and filterscorrelation
2 instantaneous featuresaudio pre-processingstatistical and spectral featuresfeature post-processing
3 intensitylevel & loudness
4 tonal analysisfundamental frequencytuning frequencykey and chords
overview intro content ACA summary course outline
course outlineoverview 1/2
1 fundamentalsdigital audio signalsconvolution & block based processingFourier transform and filterscorrelation
2 instantaneous featuresaudio pre-processingstatistical and spectral featuresfeature post-processing
3 intensitylevel & loudness
4 tonal analysisfundamental frequencytuning frequencykey and chords
overview intro content ACA summary course outline
course outlineoverview 1/2
1 fundamentalsdigital audio signalsconvolution & block based processingFourier transform and filterscorrelation
2 instantaneous featuresaudio pre-processingstatistical and spectral featuresfeature post-processing
3 intensitylevel & loudness
4 tonal analysisfundamental frequencytuning frequencykey and chords
overview intro content ACA summary course outline
course outlineoverview 1/2
1 fundamentalsdigital audio signalsconvolution & block based processingFourier transform and filterscorrelation
2 instantaneous featuresaudio pre-processingstatistical and spectral featuresfeature post-processing
3 intensitylevel & loudness
4 tonal analysisfundamental frequencytuning frequencykey and chords
overview intro content ACA summary course outline
course outlineoverview 2/2
5 temporal analysisonset detectiontempo & beatdownbeat & time signature
6 genre, similarity & mood
7 alignmentaudio-to-audioaudio-to-score
8 audio fingerprinting
9 structural segmentation
overview intro content ACA summary course outline
course outlineoverview 2/2
5 temporal analysisonset detectiontempo & beatdownbeat & time signature
6 genre, similarity & mood
7 alignmentaudio-to-audioaudio-to-score
8 audio fingerprinting
9 structural segmentation
overview intro content ACA summary course outline
course outlineoverview 2/2
5 temporal analysisonset detectiontempo & beatdownbeat & time signature
6 genre, similarity & mood
7 alignmentaudio-to-audioaudio-to-score
8 audio fingerprinting
9 structural segmentation
overview intro content ACA summary course outline
course outlineoverview 2/2
5 temporal analysisonset detectiontempo & beatdownbeat & time signature
6 genre, similarity & mood
7 alignmentaudio-to-audioaudio-to-score
8 audio fingerprinting
9 structural segmentation
overview intro content ACA summary course outline
course outlineoverview 2/2
5 temporal analysisonset detectiontempo & beatdownbeat & time signature
6 genre, similarity & mood
7 alignmentaudio-to-audioaudio-to-score
8 audio fingerprinting
9 structural segmentation