ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Experiments with the Sonic Experiments with the Sonic BrowserBrowser

Eoin Brazil, Mikael FernströmEoin Brazil, Mikael Fernström

Interaction Design CentreInteraction Design Centre

University of Limerick, University of Limerick, IRELANDIRELAND

www.idc.ul.iewww.idc.ul.ie

Laura OttavianiLaura Ottaviani

Università di Verona, Università di Verona, Dipartimento Scientico e Dipartimento Scientico e

Tecnologico, Tecnologico,

ITALYITALY

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

OutlineOutline

• IntroductionIntroduction

• The Sonic BrowserThe Sonic Browser

• Audio Similarity Rating ToolAudio Similarity Rating Tool

• Sound Designer ToolSound Designer Tool

• ConclussionsConclussions

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Introduction 1Introduction 1

• A new approach to A new approach to psychophysical psychophysical experiments experiments for scaling audio stimulifor scaling audio stimuli

• Browsing & Sorting of a stimuli setBrowsing & Sorting of a stimuli set

• Interactive 2-dimensional interface for creating a Interactive 2-dimensional interface for creating a similarity space for audiosimilarity space for audio

• Method can be used to supplement Method can be used to supplement pairwise pairwise comparisoncomparison

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Introduction 2Introduction 2• Premix environment via the use of an Premix environment via the use of an AuraAura

• Browsing & Searching of a sound collectionBrowsing & Searching of a sound collection

• Dynamic property filtering mechanismsDynamic property filtering mechanisms

• Tight CouplingTight Coupling– Objects under the Objects under the AuraAura are high-lighted and are high-lighted and

played simultaneouslyplayed simultaneously– Changes in one visualisation are reflected in Changes in one visualisation are reflected in

othersothers

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Sonic BrowserSonic Browser

• Screenscape and SoundscapeScreenscape and Soundscape

• Multiple coupled visualisationsMultiple coupled visualisations

• Direct sound representationDirect sound representation

• Data loggingData logging

• Dynamic property filter using Dynamic property filter using text filtering or text filtering or AlphaSlidersAlphaSliders

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Sonic Browser - VisualSonic Browser - Visual

Visual Representation:Visual Representation:

• Star-fieldStar-field, TreeMap, HyperTree, TouchGraph , TreeMap, HyperTree, TouchGraph visualisationsvisualisations

• Arbitrary mapping, controlled by the userArbitrary mapping, controlled by the user

– shape, colour, size, horizontal and vertical shape, colour, size, horizontal and vertical locationlocation

• Sounds playing are high-lighted.Sounds playing are high-lighted.

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Sonic Browser - AuditorySonic Browser - Auditory

Audio: Direct representation of sounds Audio: Direct representation of sounds

• Idea -> use “cocktail party effect” to exploit ability to switch Idea -> use “cocktail party effect” to exploit ability to switch between sounds in auditory scenebetween sounds in auditory scene

• Sounds are panned out in a stereo field controlled by the Sounds are panned out in a stereo field controlled by the visual location of the sounds under the cursor / visual location of the sounds under the cursor / auraaura..

• The The auraaura is user controlled and can be reduced or is user controlled and can be reduced or increased in sizeincreased in size

• By reducing the aura allows for “zooming” in on a particular By reducing the aura allows for “zooming” in on a particular sound/s. sound/s.

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Sonic Browser – UsageSonic Browser – Usage

• Cursor on object - the object plays, max volume, Cursor on object - the object plays, max volume, centered.centered.

• Click - show dialogue box with details of the Click - show dialogue box with details of the properties of the soundsproperties of the sounds

• Double-click - access the object, e.g. show the Double-click - access the object, e.g. show the details in the database, edit, etc.details in the database, edit, etc.

• Single-click & Drag – move the Single-click & Drag – move the soundsound in display in display

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Study - OverviewStudy - Overview

• Explore an interactive method for gathering a Explore an interactive method for gathering a audio similarity spaceaudio similarity space

• Explore if participants noticed any differences Explore if participants noticed any differences between sounds – real versus synthesisedbetween sounds – real versus synthesised

• Examine the application’s suitability for this type of Examine the application’s suitability for this type of scenarioscenario

• Explore if the application provides an engaging Explore if the application provides an engaging interface for participant in this type of scenariointerface for participant in this type of scenario

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Users and Data setUsers and Data set

• Postgraduates in the CSIS at ULPostgraduates in the CSIS at UL

• contained contained 1818 sounds sounds, three types, three types

– 6 recorded sounds6 recorded sounds

– 6 PD-modules modelling impact interactions of 6 PD-modules modelling impact interactions of two modal resonators simplified to return only two modal resonators simplified to return only one modeone mode

– 6 PD-modules modelling impact interactions of 6 PD-modules modelling impact interactions of two modal resonators as well as the dropping two modal resonators as well as the dropping eventevent

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

PD ModulePD Module

Rath, M. et al, An Introductory Catalog of Computer Synthesized Contact Sounds in Real-Time, Colloquium on Musical Informatics 2003

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

TasksTasks

• Familiarisation Familiarisation

– practice taskspractice tasks

• TestingTesting

– similar way with a different sound data set and similar way with a different sound data set and taskstasks

• Post testing debrief & questionnairePost testing debrief & questionnaire

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Experiment ScreenshotExperiment Screenshot

perceived size of the dropped object

perceived height of the object drop

Aura

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

User’s Perceived ScalingUser’s Perceived Scaling

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Study FindingsStudy Findings

• ``detailed comparison without reference points is ``detailed comparison without reference points is very difficult and would be much easier with only a very difficult and would be much easier with only a single scale'‘single scale'‘

• `` aura gives a comparison for similar sounds''`` aura gives a comparison for similar sounds''

• ``the longer I spent working with the sounds, the ``the longer I spent working with the sounds, the more difficult it was to sort them''more difficult it was to sort them''

• Easy to learn and understand how to use the Easy to learn and understand how to use the application for the tasksapplication for the tasks

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Study Study ConclusionConclusionss

• Synthesised sounds, even when found to be Synthesised sounds, even when found to be unrealistic, can still convey information in two unrealistic, can still convey information in two dimensions and even if one mode soundsdimensions and even if one mode sounds

• Realism of a sound does not affect its ability to Realism of a sound does not affect its ability to convey information about particular perceptual convey information about particular perceptual dimensions - sound cartoonification.dimensions - sound cartoonification.

ICAD-03, Boston, USA, July 6-9, 2003

interactiondesigncentre

Future Work & ReferenceFuture Work & Reference• Development of Development of zoomzoom• Addition of mouse click context menuAddition of mouse click context menu• Addition of audio scrubbing for swifter browsing of Addition of audio scrubbing for swifter browsing of

individual soundsindividual sounds • Inclusion of “Inclusion of “tagging” or “shading” for files of tagging” or “shading” for files of

interestinterest linked to sound editor applicationslinked to sound editor applications• Use of cue points to play sound areas/regions of Use of cue points to play sound areas/regions of

interestinterest• Website:Website: http://www.http://www.idcidc..ulul..ieie• Sonic Browser available from: Sonic Browser available from:

http://http://richierichie..idcidc..ulul..ieie//eoineoin// or or http://www.http://www.soundobjectsoundobject.org.org