Visually and Acoustically Exploring the High-Dimensional Space of Music

DistributedComputing

Group

Visually and Acoustically Exploring the High-Dimensional Space of Music

Lukas BossardMichael KuhnRoger Wattenhofer

SocialCom 2009Vancouver, Canada

2Michael Kuhn, ETH Zurich @ SocialCom 2009

• Storage media– Vinyl records– Compact cassettes– Compact discs

• An Album is stored on a single physical storage medium– Sequence of songs given by album– Album is typically listened to as a whole

History

organization by album


Music today

• Huge offer, easily available – filesharing, iTunes, amazon, etc.

• Large collections– The entire collection is stored on

a single electronic storage medium

– Organization by albums (and other lists) is no longer appropriate

organize by similarity


Organization by Similarity

• Our Goals– Mobile application (portable player)– Play songs the user likes– Overview of a collection

• Problems on mobile devices– Limited input– Limited output– Limited processing power– Limited memory

• Solution– Use song coordinates provided by www.musicexplorer.org


Which songs are similar?

• Goussevskaia et al., WI 2008:– Each song is positioned in a Euclidean „Map of Music“– Similar songs are close to each other in this Euclidean space


The Map of Music

• Based on usage data– „behaviour of the crowd“ – Gathered from social music platform (last.fm)– NO audio-analysis!

• Underlying similarity measure– Item-to-item collaborative filtering (Amazon)

[Linden et al., IEEE Internet Computing]

– „users who listen to song A also listen to song B“

• Coordinates available through webservice– www.musicexplorer.org


Hey Jude

Imagine

My Prerogative

I want it that way

Praise you

Galvanize

rock

pop

electronic

Using the Map

• Similar songs are close to each other

• Quickly find nearest neighbors

• Span (and play) volumes

• Create smooth playlists by interpolation

• Visualize a collection

• Low memory footprint– Well suited for mobile domain

convenient basis to build music software


That‘s easy – is it?

10 dimensional!


Contributions

Visual and acoustic guide to the

high-dimensional music galaxy

Proof-of-concept application for Android devices („Google-phone“)


Visual ExplorationVisual Exploration


The Reference: SensMe (Sony Ericsson)

slow

fast

happysad

Create playlist by selecting

areas

Based on audio-analysis


Requirements

Global Overview

Local Overview

Orientation

Our problem: 10 dimensions!


Lens Metaphor

Few details in the border rings

Detailed view in the center


Lens: Recursive Clustering

High resolution in the center

Few details in the border regions


Cake Metaphor

Used to represent song clusters


The Visual Exploration Interface

• Browsing– Touch cluster to bring it to the

center

• Playlist Generation– Select a number of seed songs– Playlist will consist of songs

around these seeds– Similar to SensME (but songs are

selected in a different interface)

Touch to make this area the new

center


Evaluation (1)

• User Experiment– 9 participants– Collection (1400 songs)– 5 minutes to create playlist of 20 songs (for both systems)

• Evaluation: Participants had to...– ...rate each individual song in the playlists– ...fill in a questionaire

vs.


Evaluation (2)

• Average song rating (scale: 0..10):– 5.5 (SensMe)– 6.3 (this paper)

• Questionaire (scale: 1..5):

SensMe This paper

Playlist (overall) 2.4 3.3

Diversity (3 is best) 2.4 3.4

Usability 4.7 3.7

Underlying space 2.4 4.0

Use again? 44% 67%

Trade-off:

Accurracy of high-dimensional space versus simplicity of interface


Acoustic Exploration


Idea

Shuffle

(play songs in random order)

Can we do better? Yes!Idea: Learn on the fly which songs the user likes!

Skip = bad songListen = good song


Realization

Basic algorithm: Voronoi TesselationFirst song was skipped

Supposed to be the user‘s region of

interest


Improvements

• Weighting– Account for strong/weak feedback

• Aging– Allows to adapt to changing mood

• Centering– Border regions are risky => go to center

• Escaping– Sometimes play random song to avoid getting stuck somewhere

Rating bar (left = skip, right =

good)


References

• Random shuffling (e.g. iPod-Shuffle)

• Pampalk et al. (ISMIR, 2005)– Designed for (Euclidean) audio feature spaces

dg

db

If there are songs with dg < db:

select such song with smallest dg

Else:

select song with largest ratio dg/db


Evaluation

• 9 Participants• Song ratings are used as input and for evaluation

Diversity clearly better than Pampalk

Ratings clearly better than

random


Conclusion

www.musicexplorer.org


Questions?

Visually and Acoustically Exploring the High-Dimensional Space of Music

Documents

Transcript of Visually and Acoustically Exploring the High-Dimensional Space of Music