Stravinsqi/De Montfort University at the MediaEval 2014 C@merata Task
-
Upload
multimediaeval -
Category
Software
-
view
147 -
download
2
Transcript of Stravinsqi/De Montfort University at the MediaEval 2014 C@merata Task
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
• Stravinsqi stands for STaff Representation Analysed VIa Natural language String Query Input.
1. Get question string and division value, analyse it and split into compound questions if necessary.
2. Load staff names, time signatures, and point-‐set representations of the piece.
3. Produce time-‐interval answers to elemental question(s). 4. Cross-‐check timings of answers to compound questions. 5. Convert time-‐interval answers to required format.
; Set paths.
(setq
question-‐path&name
(merge-‐pathnames
(make-‐pathname
:directory
'(:relative
"C@merata2014" "training_v1")
:name "training_v1" :type "xml")
*MCStylistic-‐MonthYear-‐data-‐path*))
(setq
notation-‐path
(merge-‐pathnames
(make-‐pathname
:directory
'(:relative
"C@merata2014" "training_v1"))
*MCStylistic-‐MonthYear-‐data-‐path*))
#| Let's begin by looking at Morley's 'April is in my
mistress' face'. |#
(setq notation-‐name "f1")
(setq
notation-‐path&name
(merge-‐pathnames
(make-‐pathname
:name notation-‐name :type "krn") notation-‐path))
(setq question-‐number "001")
#| *********************************
**** 1. ANALYSE QUESTION STRING ****
********************************* |#
; Load question string and division value.
(setq
question&division
(c@merata2014-‐question-‐file2question-‐string
question-‐number question-‐path&name
notation-‐name))
; -‐-‐> ("F# followed two crotchets later by a G" 1)
(setq division (second question&division))
; -‐-‐> 1
; Split up compound questions.
(setq questions
(followed-‐by-‐splitter
(first question&division)))
; -‐-‐> (("F#" 0) ("G" 2))
; Try US version of the question string.
(setq questions
(followed-‐by-‐splitter
"F# followed two quarter notes later by a G"))
; -‐-‐> (("F#" 0) ("G" 2))
; Try junk version of the question string.
(setq questions
(followed-‐by-‐splitter
"F# blah followed two crotchets later by a blah G"))
; -‐-‐> (("F# blah" 0) ("blah G" 2))
; Try reference to bars rather than a specific duration.
(setq questions
(followed-‐by-‐splitter
"F# followed two bars later by a G"))
; -‐-‐> (("F#" 0) ("G" 2 "bars"))
; Try 'consecutive' question.
(setq questions
(followed-‐by-‐splitter
"F# followed by two consecutive crotchets"))
; -‐-‐> (("F#" 0) ("crotchet" 0) ("crotchet" 0))
; Try linguistic variant of compound query.
(setq questions
(followed-‐by-‐splitter
(concatenate
'string "F# then two bars later a G "
"then consecutive fifths in the left hand")))
#|
-‐-‐> (("F#" 0) ("G" 2 "bars")
("fifths in the left hand" 0)
("fifths in the left hand" 0))
|#
; Back to original question.
(setq questions
(followed-‐by-‐splitter
(first question&division)))
; -‐-‐> (("F#" 0) ("G" 2))
#| ***************************************
**** 2. LOAD REPRESENTATIONS OF PIECE ****
*************************************** |#
#| Load staff names and names of the opening clefs on
each staff. |#
(setq
staff&clef-‐names
(staves-‐info2staff&clef-‐names notation-‐path&name))
#|
-‐-‐> (("Bass" "bass clef") ("Tenor" "tenor clef")
("Alto" "treble clef") ("Soprano" "treble clef"))
|#
#| Load time-‐signature information and any changes
across the piece. |#
(setq
ontimes-‐signatures
(append-‐ontimes-‐to-‐time-‐signatures
(kern-‐file2ontimes-‐signatures notation-‐path&name)))
; -‐-‐> ((1 4 4 0))
; If there was a change, it might look like:
; -‐-‐> ((1 4 4 0) (9 3 2 32))
; Load point-‐set representations.
(setq
artic-‐set
(kern-‐file2points-‐artic-‐dynam-‐lyrics
notation-‐path&name))
#|
-‐-‐> ((0 46 52 4 3 NIL NIL ("place."))
(0 58 59 2 2 NIL NIL ("place."))
(0 65 63 2 1 NIL NIL ("place."))
(0 70 66 2 0 NIL NIL ("place"))
(3 58 59 1 2 NIL NIL ("With-‐")) ...)
|#
#| **********************************************
**** 3. TIME-‐INTERVAL ANSWERS TO QUESTION(S) ****
********************************************** |#
(setq
ans-‐harmonic-‐interval
(mapcar
#'(lambda (x)
(harmonic-‐interval-‐of-‐a
(first x) point-‐set staff&clef-‐names))
questions))
; -‐-‐> (NIL NIL)
; Mel. intervals to be prefaced by "melodic".
(setq
ans-‐melodic-‐interval
(mapcar
#'(lambda (x)
(melodic-‐interval-‐of-‐a
(first x) point-‐set staff&clef-‐names))
questions))
; -‐-‐> (NIL NIL)
(setq
ans-‐dur-‐pitch
(mapcar
#'(lambda (x)
(duration&pitch-‐class-‐time-‐intervals
(first x) point-‐set staff&clef-‐names))
questions))
; -‐-‐> (NIL NIL)
; This is the function of interest on this occasion...
(setq
ans-‐pitch
(mapcar
#'(lambda (x)
(pitch-‐class-‐time-‐intervals
(first x) point-‐set staff&clef-‐names))
questions))
#|
-‐-‐> (((32 36) (57 115/2) (58 60))
((12 13) (12 13) (24 32) (26 27) (28 32) (46 48)
(48 50) (49 50) (50 52) (50 52) (54 57) (60 62) (60 61) (61 62) (62 63) (63 64)))
|#
#| ****************************************************
**** 4. Cross-‐check timings for compound questions ****
**************************************************** |#
; Put the answers next to one another in one list.
(setq
time-‐intervals
(list
ans-‐harmonic-‐interval ans-‐melodic-‐interval
ans-‐dur-‐pitch ans-‐pitch ans-‐dur ans-‐nadir-‐apex
ans-‐triad ans-‐triad-‐inversion ans-‐texture ans-‐cadence
ans-‐word&event ans-‐word ans-‐artic ans-‐dur-‐rest
ans-‐tied&event))
#|
-‐-‐> ((NIL NIL) (NIL NIL) (NIL NIL)
(((32 36) (57 115/2) (58 60))
((12 13) (12 13) (24 32) (26 27) (28 32) (46 48)
(48 50) (49 50) (50 52) (50 52) (54 57) (60 62)
(60 61) (61 62) (62 63) (63 64) (63 64)))
(NIL NIL) (NIL NIL) (NIL NIL) (NIL NIL) (NIL NIL)
(NIL NIL) (NIL NIL) (NIL NIL) (NIL NIL) (NIL NIL)
(NIL NIL))
|#
; Cross-‐check the timings of compound questions.
(setq
time-‐intervals
(if (> (length questions) 1)
(cross-‐check-‐compound-‐questions
questions ontimes-‐signatures time-‐intervals)
(if (index-‐item-‐1st-‐doesnt-‐occur (list nil) time-‐intervals)
(first
(nth (index-‐item-‐1st-‐doesnt-‐occur (list nil) time-‐intervals) time-‐intervals)))))
; -‐-‐> ((58 62))
(bar&beat-‐number-‐of-‐ontime 58 ontimes-‐signatures)
; -‐-‐> (15 3 58)
Spider Plot
0.25 0.5 0.75 1All0.25
0.5
0.75
1
Pitch
0.25
0.5
0.75
1
Duration
0.25
0.5
0.75
1
Pitch and Duration
0.25
0.5
0.75
1Articulation
0.25
0.5
0.75
1Voice Specific
0.250.5
0.751
Lyrics
0.250.5
0.751
Compound
0.25
0.5
0.75
1
Melodic Interval
0.25
0.5
0.75
1
Harmonic Interval
0.25
0.5
0.75
1Cadence
0.25
0.5
0.75
1 Triad
0.25
0.5
0.75
1 Texture
Measure RecallMeasure PrecisionBeat RecallBeat Precision
Triad Spider Plot
0.25 0.5 0.75 1All0.25
0.5
0.75
1
Pitch
0.25
0.5
0.75
1
Duration
0.25
0.5
0.75
1
Pitch and Duration
0.25
0.5
0.75
1Articulation
0.25
0.5
0.75
1Voice Specific
0.250.5
0.751
Lyrics
0.250.5
0.751
Compound
0.25
0.5
0.75
1
Melodic Interval
0.25
0.5
0.75
1
Harmonic Interval
0.25
0.5
0.75
1Cadence
0.25
0.5
0.75
1 Triad
0.25
0.5
0.75
1 Texture
Measure RecallMeasure PrecisionBeat RecallBeat Precision
Mean
• The Stravinsqi algorithm has effectively solved seven of the twelve C@merata task categories (pitch, duration, pitch and duration, articulation, voice specific, lyrics, and melodic interval)
• As for the remaining five categories, precision for one category (compound queries) can be improved by fixing a selection-‐criteria bug
• More data are required for the triad, cadence, and texture query categories
• A big “thank you” to Richard Sutcliffe and team
• The Stravinsqi algorithm has effectively solved seven of the twelve C@merata task categories (pitch, duration, pitch and duration, articulation, voice specific, lyrics, and melodic interval)
• As for the remaining five categories, precision for one category (compound queries) can be improved by fixing a selection-‐criteria bug
• More data are required for the triad, cadence, and texture query categories
• A big “thank you” to Richard Sutcliffe and team
• The Stravinsqi algorithm has effectively solved seven of the twelve C@merata task categories (pitch, duration, pitch and duration, articulation, voice specific, lyrics, and melodic interval)
• As for the remaining five categories, precision for one category (compound queries) can be improved by fixing a selection-‐criteria bug
• More data are required for the triad, cadence, and texture query categories
• A big “thank you” to Richard Sutcliffe and team
• The Stravinsqi algorithm has effectively solved seven of the twelve C@merata task categories (pitch, duration, pitch and duration, articulation, voice specific, lyrics, and melodic interval)
• As for the remaining five categories, precision for one category (compound queries) can be improved by fixing a selection-‐criteria bug
• More data are required for the triad, cadence, and texture query categories
• A big “thank you” to Richard Sutcliffe and team
Researchers at CCARH for creating and hosting kern scores
Collaborators at:
Johannes Kepler University: Gerhard Widmer, Andreas Arzt, Sebastian Böck, and Sebastian Flossmann
Center for Mind and Brain, UC Davis: Petr Janata, Fred Barrett, and Joy Geng
Aalborg University: David Meredith
University of Lyon: Barbara Tillmann
The Open University: Robin Laney, Alistair Willis, and Paul Garthwaite
Funding bodies: Austrian Science Fund (FWF) project number Z159 (Wittgenstein Grant), NSF grant #1025310, ESPRC
!"
!"
Allegro non tanto
# $$$$$ %%!
& % !
& ! & & & '% !
& ! & & & %!
& ! & & & '%!
& ! & & ! &' (
) $$$$$ * &+ & & & & & & * &+ & & &' & & & * &'+ & &
6
# $$$$$ % !
& !
& & &' '%!
& ! & &' &&' '%& !
&' % &' '%!
& ! & & ! &' ( % !
& !
& & &'
) $$$$$ ,%' ! %%'''% !&& %%''
% !&& %%'' %%!
!
* &' ( & & '% !&& %%''
11
# $$$$$ '%!
& ! & &' &&' %$*!%
!
&- &. && ''$ &&% !
& & &% & & / & & & & / & & & &&&(
) $$$$$ '% !&& %%''&&& &' & &&
& ! &+ %& / & % & %& / & % & &(
!
P1,2
!
P2,2
!
P3,3
!
P3,2
!
P2,1
!
P3,4
!
P2,3
!
P3,1
!
P1,1
Beginning of op.56 no.1 by Chopin
!"
!"
Allegro non tanto
# $$$$$ % %& %&' $$$$$
% %&
6
# $$$$$
' $$$$$ %&
11
# $$$$$
' $$$$$
!
P22
!
P33
!
P32
!
P21
!
P34
!
P23
!
P31
!
P11
!
P12
Beginning of op.56 no.1 by Chopin
!"
!"
Allegro non tanto
# $$$$$% % % % % % % & % ''
'%' % ! %%%% % % %' % % % ( %' %' % %' %'
3
3
3
# $$$$$ %%%' %%' %%% %%%% %!
( )%%$'&%' % !* &# %%' %%%' %%%'' % & +'%' %' %%%% !* & %%'' %%%''
6
# $$$$$ %' % %' % ! %' %' %' % ''+ %
%%% %%'+ '''%' & %' %%%% %%%!
% %' %' % %' %' %' % %' % ! %'3
# $$$$$%' % %%' %% ) %'
% ( %' ( ( # %%%+' %%'' %%%'' %' % %%' %% )
11
# $$$$$ %' %' % ''+ %
% (%% %%'+ '''%' & %' %%%% %%% $% !
% %%* , ,3
% % ! %- %%%$- %%%$-- %' %' % % %$) $$$$$ %'
% ( %' ( ( %%%$$$ %%%$$$ (%%$$ %%-- %%'$ %%'
%%%%
!"#"#"
#"$" %"
&" &"
&"
Generated passage