Automatic synchronisation between audio and score musical description layers
-
Upload
honorato-nguyen -
Category
Documents
-
view
20 -
download
0
description
Transcript of Automatic synchronisation between audio and score musical description layers
Automatic synchronisation between audio and score musical description layers
Antonello D’Aguanno, Giancarlo VercellesiLaboratorio di Informatica Musicale
Università degli Studi di Milano
Speaker: Antonello D’Aguanno
Introduction
• Contemporary digital music archives consist of huge collections of heterogeneous documents
• The heterogeneity of music information makes retrieval hard to accomplish
• Synchronization is one of the open problems in Music Information Retrieval (MIR)
Automatic synchronisation between audio and score musical description layers 2 /15
Synchronisation
3 /15
Stream Audio
Score
Synchronized score
Synchronization
Synchronisation means that for a given event in a music score we can determine the timing of the corresponding audio events
Automatic synchronisation between audio and score musical description layers
Related Works
• Many algorithms have been proposed in literature that deal with synchronisation
• All the algorithms proposed use MIDI score representation and PCM audio
4 /15Automatic synchronisation between audio and score musical description layers
Related Works
• The unsolved problems of synchronisation are not limited to finding a suitable algorithm
• How can this result generalised to use it in other applications?
• No answers in literature…
• MX can solve the problem
5 /15Automatic synchronisation between audio and score musical description layers
The linking structure in MX
6 /15
Notational Layer
Spine Sub-Layer
Performance Layer
Audio Layer
Automatic synchronisation between audio and score musical description layers
The COMSI algorithm
Three phases:
• Score Analysis
• Audio Analysis
• Recursive Decisional Matching
7 /15Automatic synchronisation between audio and score musical description layers
The three phases
• Score Analysis: The MX score is read in order to extract all relevant musical events
• Audio Analysis: The PCM audio signal is analysed to identify all possible musical notes
• Recursive Decisional Matching: Relates the event at the score level with the same event at the audio level
8 /15Automatic synchronisation between audio and score musical description layers
MX score analysis
1. For each measure, only the notes with strong accent are selected
2. For every strong accent, the verticalisation of the score is computed
9 /15
Observation:
COMSI Algorithm can manage conveniently even MIDI score
Automatic synchronisation between audio and score musical description layers
Audio Analysis
1. The audio signal is filtered with a notch filter centred on the frequency of the note examined
2. The audio signal in time-windows is split at 100ms
3. For each note, a possible attack-time is the audio segment which has an energy value above a threshold, obtained from the average energy of the filtered signal
10 /15Automatic synchronisation between audio and score musical description layers
Execution Events
The execution events are all the audio events having an energy above the threshold
The set of these execution events is named pseudo-score
11 /15Automatic synchronisation between audio and score musical description layers
The Decisional Matching
12 /15Automatic synchronisation between audio and score musical description layers
Measure Synchronisation
Recursive research of attack-time related to the first event of each measure in the score
Musical Event Synchronization
Sequential research of each musical event contained in a measure
Results
13 /15
Track Time Signature
Correct Measure
One Error Measure
Event
Twice Error Measure
Event
Chopin 3/4 66% 24% 10%
Beethoven 4/4 71% 18% 11%
Beethoven Faster 4/4 59% 36% 15%
Fiore di Maggio 4/4 80% 17% 13%
Cavatina 6/8 43% 48% 9%
Aria 6/8 61% 31% 8%
Automatic synchronisation between audio and score musical description layers
Conclusions
In this work we have described algorithm dedicated to score and audio alignment using the MX / IEEE P1599 formatThis algorithm allows alignment of an MX score and its execution, coded in PCM formatIt produces an output for the MX Spine that contains synchronisation between notes and audio signal
14 /15Automatic synchronisation between audio and score musical description layers
Future works
• A new test infrastructure is being developed, which will use the MX capabilities
• More tests in a shorter time!
• Will take into account every musical event
Automatic synchronisation between audio and score musical description layers 15 /15