Post on 22-Mar-2022
1/8
Final Project:
Learning to Generate High-Quality Music
With Performance RNN
GCT634
Musical Applications of Machine Learning
Spring 2019
KAIST
Student: Pham Xuan Trung
Kiheum Cho
School of Electrical Engineering
#14
2/8
Introduction
Expressive timing and dynamics are essential parts of music.
Expressive timing
refers to variation in performed durationsamong notes represented in a musical score with a single rhythmic value.
Dynamics
refer to the volume of sound. It relates to the variation of loudness of music over time. In piano, the loudness of a note depends on how hard the pianist hits the key.
Task – Generating piano music with expressive timing and dynamics.
3/8
Why is it an interesting problem?
A sample from sheet music
Two main elements of music:
1. The composition focuses on the musical notes that define a song.2. The performance focuses on how the notes are played by the performer.
Can a machine perform like a human?
Can a machine produce the dynamics for digital sheet music and so, create stylized performances?
Using a RNN to do that task.
s
ot
w
Uxt
o0
U x0
s0
w
o1
U x1
s1
ot
U xt
st
…
4/8
Related work
• We follow the idea from paper: “Learning to Create Piano Performances”, NIPS2017 conference. [1]
[1] “Learning to Create Piano Performances”, Sageev Oore, NIPS 2017; [2] https://magenta.tensorflow.org/performance-rnn
Music generation with score (16s) Music generation with performance
(16s) – Performance RNN tensorflow [2].
All previous work:Generating output is a sequence of notes
(scores).
This paper:Generating the notesGenerating expressive variations in the
timing and musical dynamics (loudness)
5/8
• International Piano-e-Competition, recorded MIDI files
• Example
1) beethoven_opus90_2_format0.mid
2) schub_d960_2_format0.mid
…
N) schubert_d935_3_format0.mid
Dataset
http://www.piano-e-competition.com/midi_2018.asp
6/8
Method
Previous
• Note-based
• Express midi with quantized note data
Performance RNN
• Event-based
• Convert note data to event data considering each note relationship
7/8
Implementations
• Baseline• Our code is based on homework3. We use its network and some configurations.
• Data Preprocessing• Midi -> Note -> Event -> Array (w/ 256 label).
800
850
900
950
1000
1050
1100
0 200 400 600 800 1000 1200 1400
Tra
inin
g l
oss
Epochs
Learning curve
Example code to generate Event
Performance RNN in Pytorch
8/8
Generated music (midi files)
note-based.midevent-based.mid
• Note-based (HW3) - Quantized• Event-based (Ours) - Time Shift