Enabling Access to Sound Archives through Integration, Enrichment and Retrieval

Report about polyphonic music transcription

Coordinating meeting, Nice, November

Project #033902

What is Automatic Music Transcription

TranscriptionPlay/Synthesis

Coordinating meeting, Nice, November

Project #033902

Key technologies of polyphonic music transcription

Music onset detection

Polyphonic music transcription

Coordinating meeting, Nice, November

Project #033902

A particular time-frequency analysis tool: Resonator Time-frequency Image (RTFI)

Computation-efficient implemented by the first-order complex resonator filter bank development of multi-resolution fast implementation

A uniform Framework of TF analysis for music signal unlike Cohen’s class and Affine class, RTFI is not limited to

either constant-band or constant-Q by simply setting several parameters, the RTFI can implement

different TF analysis such as constant-band, constant-Q and ear-like TF analysis

a frequency-dependent time-frequency analysis

Coordinating meeting, Nice, November

Project #033902

Music Onset Detection

What is music onset detection detection of the instant when a new event begins in acoustical

signal hard Onset (fast transition with big energy change) soft Onset (slow transition with small energy change)

How human detect onset energy change pitch change timbre change

Coordinating meeting, Nice, November

Project #033902

Onset detection method in EASAIEROriginal Audio Signal

RTFI Average Energy Spectrum

Adjusted Energy Spectrum

Pitch Energy Spectrum

Smoothed Pitch Energy Spectrum

Normal Pitch Energy Specturm

Difference Pitch Energy Spectrum

Time -Frequency

Processing

Pitch Change Detection Algorithm Detection

Algorithm

Energy Change Detection

Aloghrithm Detected Onsets

Time-Frequency Processing: incorporating psychoacoustics knowledge about loudness perception making energy-change and pitch-change as clear as possible

Detection Algorithms: detecting onsets by both energy and pitch change clues

Coordinating meeting, Nice, November

Project #033902

Problems in Polyphonic Pitch Estimation

Harmonic components of different music notes may overlap

In-harmonic: some music instrument have inharmonic timbre

Coordinating meeting, Nice, November

Project #033902

Polyphonic Pitch Estimation Method in EASAIER

5 Steps:1) Performing RTFI analysis2) Extracting harmonic

components3) Making preliminary estimation

of possible pitches 4) Cancelling the extra pitches by

checking harmonic components ( simple timbre model)

5) Checking pitch candidates by spectral smoothing principle

Coordinating meeting, Nice, November

Project #033902

Compared with other state-of-art methodsMIREX 2007 Evaluation

Music onset detection According to the overall performance, our method wins this

contest

Polyphonic pitch estimation (Multiple-F0 estimation task) our method performed third best in the submitted 16 methods.

The performance differences between our method and the first and second best method are minor.

but most computationally efficient, about 10 time faster than the first best method, and 100 time faster than the second best method

Coordinating meeting, Nice, November

Project #033902

Future plan

To develop the method for note offset detection To estimate the note duration time

To improve and evaluate the automation music transcription system

To apply the transcription system assisting the other functionalities such as content-based music retrieval