Sound and Music Computing Research: Trends and...
-
Upload
dinhnguyet -
Category
Documents
-
view
214 -
download
0
Transcript of Sound and Music Computing Research: Trends and...
Sound and Music Computing Research:
Trends and Challenges
Xavier SerraMusic Technology Group
Universitat Pompeu Fabra, Barcelona
Xavier Serra - 2008
Music communication framework
Symbolic representation
Temporal controls
Source sound
Sound field
PerceptionCognition
Listener
Composer
Performer
InstrumentRoom
“Musical” Knowledge base
“Physical” Knowledge base (Moorer, 1990)
Xavier Serra - 2008
SMC research disciplines
Music
Psychology
PhysicsEngineering
SMC
TheoryPerformance CompositionComputer
Science
ArtificialIntelligence Psychomusicology
Cognition
Psychoacoustics
Acoustics
Device design
Digital signal
processing
Digitalhardware
Programming
(Moorer, 1990)
Xavier Serra - 2008
New context for SMC research
CONTENT(databases)
COMMUNITY(social tools)
MUSIC3.0
CREATION(production
tools)
collaborativecreativity
contentsharing
contentprocessing
Xavier Serra - 2008
Sound synthesis
Current trends:
Audio content processing
Physical modeling based on measurements
Gesture-sound modelingTraditional approach to sound
synthesis
Xavier Serra - 2008
Physical modeling from measurements
Measuring impulse responses from a violin
Various sensors on a violin
Xavier Serra - 2008
Gesture-sound modeling
Finger position
Playing state estimator
String
Bow- bridge distance
Bow pressing force
Bow acceleration
Bow velocity
Bow position
Audio
Performed scores
Violin synthesisPachelbel Canon
Xavier Serra - 2008
Sound/Music description
(Lesaffre et alt., 2003)
Current trends:
Search and retrieval
Combining bottom-up and top-down approaches
Xavier Serra - 2008
Interaction
Mathews with the Radio Drum
Current trends:
Understanding performance expression
New musical interfaces
Xavier Serra - 2008
Understanding performance
High-qualityrecordings
analyze
encode
extract
obtain
Analyze
Machinerepresentation
Structure of pieces
SymbolicdescriptionExpressive
aspects ofrecordings
Machinelearning
Models
Synthesizedscore
Automatic expression
Xavier Serra - 2008
Bridging the semantic gap
Signalfeatures
ContentObjects
HumanKnowledge
semantic gap
Audio(music recordings)
Text(lyrics, editorial text,
press releases, …)
Image(video clips, CD covers,
printed scores, …)
frequency
duration
spectrum
intensity
pitchtimbreloudness time
melody
harmony
rhythm
source
dynamics
emotions
genre
understanding
musicscores
graphicstyle
similarity
personalidentity
memories
opinions
nouns
adjectivesscenes
signstags
labels
motions
shotrhythm
expectations
colorsshapes
textures
contrastsverbs
sentences
articles
numbers
semanticfeatures
Music information plane
Xavier Serra - 2008
Signalfeatures
ContentObjects
HumanKnowledge
semantic gap
Audio(music recordings)
Text(lyrics, editorial text,
press releases, …)
Image(video clips, CD covers,
printed scores, …)
frequency
duration
spectrum
intensity
pitchtimbreloudness time
melody
harmony
rhythm
source
dynamics
emotions
genre
understanding
musicscores
graphicstyle
similarity
personalidentity
memories
opinions
nouns
adjectivesscenes
signstags
labels
motions
shotrhythm
expectations
colorsshapes
textures
contrastsverbs
sentences
articles
numbers
semanticfeatures
Bridging the semantic gap
Signalprocessing
Machinelearning
Webmining
MusictheoryStatistical
modeling
Music information plane
Xavier Serra - 2008
Signalfeatures
ContentObjects
HumanKnowledge
semantic gap
Audio(music recordings)
Text(lyrics, editorial text,
press releases, …)
Image(video clips, CD covers,
printed scores, …)
frequency
duration
spectrum
intensity
pitchtimbreloudness time
melody
harmony
rhythm
source
dynamics
emotions
genre
understanding
musicscores
graphicstyle
similarity
personalidentity
memories
opinions
nouns
adjectivesscenes
signstags
labels
motions
shotrhythm
expectations
colorsshapes
textures
contrastsverbs
sentences
articles
numbers
semanticfeatures
Bridging the semantic gap
Signalprocessing
Machinelearning
Webmining
MusictheoryStatistical
modeling
Computationalneuroscience
Multimodalprocessing
Musiccognition
Ontologies
Reasoningrules
Computationalmusicology
Textunderstanding
Music information plane
Xavier Serra - 2008
Understanding communication
Score A
Score B
Score C
Score D
Performer A
Performer B
Performer C
Performer D
Instrument A
Instrument B
Instrument C
Instrument D
Audience indiv. 1
COMPOSITION
Audience indiv. 2
Audience indiv. 3
Compositional channel: musical message + role in the performance (solo, accompanist, etc…)
Sonic channel
Instrumental channel: sound-producing and modifying movements / actions, haptic feedback
Visual channel
AUDIENCE
PERFORMANCE