Project Number: JP-0208

HEARING DATA

A Major Qualifying Project

Submitted to the faculty

Of the

WORCESTER POLYTECHNIC INSTITUTE

In partial fulfillment of the requirements for the

Degree of Bachelor of Science

By

Nedunceliyan Kulasekaran

Gary Barber Jr.

Date: December 13, 2002

Professor Joseph D. Petruccelli, Advisor of Record

Professor Michael Ciaraldi, Co-Advisor

Abstract

The aim of this project is to study whether sonification (mapping sounds to data) can be used to

aid the interpretation of data, much as visualization techniques are used today, and if so, to find

effective sonification methods. The methodology of the project will involve development of

efficient techniques of sonification implemented using MAX/C++ environments with future

software integration considered. The testing of the efficacy of the developed methods will

follow.

(This must be rewritten for the present, and expanded to give the project results.)

i

Acknowledgements

[to be added after project completion]

ii

CONTENTS

1 INTRODUCTION...................................................................................................1

2 BACKGROUND.....................................................................................................3

2.1 Auditory Perception.................................................................................................3

2.2 Sonification Tools....................................................................................................4

2.3 Current Applications of Sonification.......................................................................6

2.4 Time-Series..............................................................................................................6

2.4.1 Time-Series Analysis in the Time Domain (Stationary Processes and the

Autocorrelation Function).......................................................................................7

2.4.1.1 Sample Autocovariance & Autocorrelation.........................................................8

2.4.2 The “Additive Classical Decomposition” Time Series Model [27]......................10

3 METHODOLOGY................................................................................................13

3.1 Statistical Analysis and Mapping Program............................................................14

3.1.1 Designing the Program..........................................................................................14

3.1.1.1 Linear Mapping Algorithms..............................................................................17

3.1.2 Program Specifications.........................................................................................21

3.1.2.1 File Formats......................................................................................................21

3.1.2.2 File Specification Type (1).................................................................................21

3.1.2.3 File Specification Type (2).................................................................................21

3.1.2.4 File Name and Extension Conventions..............................................................22

3.1.3 Using the Program................................................................................................23

3.2 Sonification Program.............................................................................................25

3.2.1 Designing the Program..........................................................................................25

3.2.1.1 Parser Patch (fl_parser)....................................................................................26

3.2.1.2 Core Hearing Data Patch (Core.maxb)............................................................28

3.2.1.3 Core II Hearing Data Patch (Core2.maxb).......................................................29

3.2.1.4 The Panning Relationship Patch (PanRel.maxb)..............................................29

3.2.1.5 Hearing Data Main Patch (Hearing Data.maxb)..............................................30

3.2.2 Program Specifications.........................................................................................30

3.2.3 Using the Program................................................................................................30

3.3 Testing: Controlled Experiment using Human Subjects.......................................31

3.3.1 Designing the Experiment......................................................................................31

3.3.2 Experiment Procedures.........................................................................................34

4 RESULTS AND ANALYSIS................................................................................36

Results 36

4.2 Analysis.................................................................................................................37

4.2.1 Generalized Linear Model [28].............................................................................37

5 CONCLUSION......................................................................................................39

5.1 Recommendations for Future Work......................................................................39

6 REFERENCES......................................................................................................41

7 APPENDICES.......................................................................................................44

7.1 Time-Series Analysis in the Frequency Domain...................................................44

7.1.1 Spectral Density.....................................................................................................44

7.1.2 Discrete Fourier Transform..................................................................................45

7.2 Analysis and Mapping Program (C++ Source).....................................................47

7.3 Sonification Program Patch Schematics (MAX Source).......................................63

7.3.1 Floating Point Parser (fl_parser)..........................................................................64

7.3.2 Core Hearing Data Patch......................................................................................65

7.3.3 Core II Hearing Data Patch (left end part)...........................................................66

7.3.4 Core II Hearing Data Patch (right end part)........................................................67

7.3.5 Panning Relationship Patch for Hearing Data.....................................................68

7.3.6 Hearing Data Patch (top part)..............................................................................69

7.3.7 Hearing Data Patch (bottom part)........................................................................70

7.3.8 Hearing Data User Interface.................................................................................71

7.4 Controlled Experiment – Introductory Handouts for Subjects..............................72

7.5 Controlled Experiment – Airline Data Example Handout.....................................75

7.6 Controlled Experiment – Graphs Handout............................................................76

7.7 Controlled Experiment – Questionnaire Form......................................................77

2

1 INTRODUCTION

This project intends to (Gents, it is not enough to resubmit your proposal. You must at least

update the language for the present.) explore sonification technology, and its potential

applications to data analysis. Sonification is:

A mapping of numerically represented relations in some domain under study to

relations in an acoustic domain for the purposes of interpreting, understanding, or

communicating relations in the domain under study. [5]

As far as data analysis is concerned, sonification stands to enhance existing techniques by

providing a more sensory-immersed experience. Multimedia presentations tend to make the data

they present more attractive, and help their audience to retain the information, if utilized properly

[21]. Lessons presented with complementary media spanning separate senses (e.g. hearing and

sight), as long as they are not distracting to the learners, seem to enhance and speed-up the

learning process. (Where is your evidence for the previous statement?) Sonification working

alongside existing visual methods of data analysis can provide a sort of “multimedia” data

analysis, potentially adding a new dimension to existing techniques. The potential benefits of a

combined presentation medium (sonification and visualization) lead one to wonder whether an

auditory medium by itself would also be an effective data presentation method.

Sonification is an interdisciplinary field of research. It combines the efforts of experts in such

fields as statistics, psychology, computer science, engineering, physics, and music (among

others). Statisticians are interested in finding ways of using sonification to present and

summarize data, much as visual graphs are currently used. For example, this project is

concerned with exploring ways to sonify time series data: data taken over time. Psychologists

want to study how humans perceive and interpret audio, in hopes of finding efficient sonification

methods. Computer scientists hope to find applications where auditory displays in software

would be more natural to users and ways to present this information to the user (sound

synthesis). Engineers are interested in how sonification can be incorporated into hardware to

ease in its use. Physicists study how the properties of sound affect methods of sonification. And

1

musicians are interested in the types of sound that are useful for efficient sonification as well as

the artistic applications of sonification.

As technology advances, computing power continues to increase. With this increase, scientists

can explore previously inaccessible problems (e.g the Human Genome Project1). These new

problems provide large sets of data that must be analyzed. Because it has already proven more

effective than visualization for presenting some kinds of data (e.g., the Quantum Whistle [17]

and the Geiger counter (see 2.3)), sonification holds promise for analyzing certain large data sets.

In certain problems data presentation techniques such as sonification may not only be helpful,

but also perhaps necessary: for example, when data must be presented but the eyes are occupied

with other tasks.

Most data-based communication is done in a visual format: graphs, tables, etc. But a good

portion of general communication is done using sound. Therefore, sound is a familiar medium

for human communication. This would suggest that an effective way for representing data

through an auditory medium might take advantage of the natural role of sound in human

communication. Sound also preserves one of the conveniences of visual communication: that one

needn’t come into physical contact with the medium to analyze it, as would be necessary with

touch or taste (and it is beyond the scope of this paper to suggest an effective method for

communicating data through smell). Furthermore, audio capabilities have been incorporated into

almost all computer systems and workstations for a number of years. The technology that makes

sonification possible is widely available even in schools, where it will be necessary to teach these

techniques should they prove useful. On the same workstations that students are currently taught

visualization techniques, they could also be shown sonification techniques. The capabilities

exist; what remains to be studied is if they are valuable.

1 Human Genome Project Information http://www.ornl.gov/hgmis/

2

2 BACKGROUND

This section focuses on general information of current developments and tools in relation to the

project’s objectives.

The field of sonification consists of three key components: psychological research involving the

study of human perception and cognition, sonification methods (design and application of

sonification to problems), and tools that perform sonification [17].

2.1 Auditory Perception

Sonification involves the manipulation of sounds to convey information to a human listener. This

implies that the perception of sound becomes important within this research discipline. The study

of the perception of sound and psychoacoustics relates closely to the paper’s (What paper?)

objective of finding effective and creative ways to sonify data.

Human hearing can detect changes in sounds over time, such as periodic and aperiodic events,

and can detect small changes in frequency in continuous signals [26]. Translation of data into an

auditory medium can be done in many ways, some more effective than others. For successful

translation, it is important to develop systematic and practical solutions for the mapping of data

while considering the meaning of the actual data and preserving this relationship. (What

relationship?) An analogy is found in language translation. The method of natural language

translation [2], maps between two languages by taking a word in one language and replacing it

with an equivalent word in the other. This doesn’t always work, however, because the meanings

of individual phrases are not always preserved. Instead, entire phrases and even sentences must

be considered, as well as the words that make them up, and translated as a unit to preserve

meaning.

Human perceptual studies suggest that humans reference new inputs based on existing similar

data types and conclude by comparison and association [11]. Hence, the question of whether

designers and researchers should synthesize entirely original sound waves to represent data, or

3

whether they should incorporate sounds that are already familiar to listeners (e.g. existing

musical instruments) is one researchers are still arguing [17]. How sonified audio is perceived is

also affected by the level of training of the user in the task being performed. Sonification

displays needn’t be natural, just as some visual displays do not present themselves naturally.

Take, for example, the various gauges and lights in the cockpit of an airplane. The pilot is

trained to glance at these and immediately know the status of the plane. Similar training could

take place for an auditory display. A particular example of this would be skilled sonar operators

[17].

In addition, a listener can focus on several distinct auditory stimuli at a time. For example,To

illustrate this, a familiar situation is presented.

While listening to music, it is a trivial processcommon for most music listeners to notice the

introduction of different instruments to the chorus, or . It is also trivial to monitor fluctuations in

several of these instruments at a time. This is generally how one enjoys music. Similarly, one

can view several visual stimuli at a time as well. (What is the point of this last sentence?)

The human ability to observe subtle changes in a continuous signal presented in audio more

quickly than one presented in video, proves the distinct advantage of auditory over visual

perception for certain situations [17]. It also proves to be an open resource when other sensory

organs are occupied for other tasks (e.g. driving). This implies that auditory perception could be

utilized when visual and other sensory organs are occupied or inefficient for the observed task.

2.2 Sonification Tools

The development of sonification tools should provide the user with flexible control over data

manipulation and sound parameters. Recently-developed tools for sonification rely on a number

of basic hardware and software requirements. These specifications are available commercially

and include sound synthesis support for the specification and control of waveforms, control

software to aid in manipulation of produced sound, analysis and editing tools, and signal

processing hardware and software.

4

A common music protocol widely used for composing and editing music is MIDI (Musical

Instrument Digital Interface)2. MIDI controllers and sequencers provide sound synthesis support

on most modern multimedia personal computers. MIDI is a communications protocol that

determines the data transmission format between electronic musical instruments and computers.

Rather than representing musical sounds directly, MIDI transmits information on how music is

produced through a set of commands and variables. The command set includes note-on, note-off,

balance, pan, volume, velocity, release and attack times, pitch bend and other continuous control

methods for synthesizers. MIDI also allows the use of separate data tracks – enabling separate

assignment of data streams, which can be treated individually or collectively. The waveforms

produced are stored in a “wavetable” on the receiving instrument or sound card and are accessed

by the MIDI data file.

Control/editing software such as Goldwave3 and Wavelab4 provide a variety of cut, copy, paste,

record and wave editing functions. These packages provide an efficient and accessible way to

manipulate produced waveforms. Signal analysis and synthesis tools such as Csound (MIT),

CMix (Princeton), MAX (Cycling ’74) and Softstep (Algorithmic Arts) allow a range of

functionality for the control and manipulation of sound and MIDI programming environments.

Besides existing technology borrowed from the music industry, powerful mathematical analysis

programs are also used within this area of research. Mathematica5, Matlab6 and even Maple7 have

been used for sound synthesis algorithms, signal processing and developing specific application

tools [20]. Although these software packages have various limitations, mathematicians have been

using these programs to create audible data from mathematical equations to study a different

method of perceiving the phenomena on the equations describe (see Cummerow [7], and MIDI

music from DNA, Protein and Math [20]).

2 August 1983 – the birth of MIDI protocol by musically-involved corporations (e.g. Kawai, Korg, Roland, Sequential Circuits and Yamaha) in Japan resulted in MIDI 1.0 Specifications, which established the requirements of the interface in detail.3 Goldwave Inc. http://www.goldwave.com/ 4 Steinberg.net http://www.steinberg.net/en/ 5 Wolfram Research Inc., http://www.wolfram.com/ 6 The Mathworks, Inc., http://www.mathworks.com/ 7 Waterloo Maple Inc., http://www.maplesoft.com/

5

2.3 Current Applications of Sonification

Sonification research is evolving from a field of inquiry to a field of application [17]. As the field

becomes more developed, focus is moving from exploring the technology of sonification to

finding new applications of sonification and its efficacy.

One example of a successful basic application of sonification is the Geiger counter8. The counter

is used to notify the listener of radiation levels by the frequency of “clicks”. The counter

provides continuous awareness of the degree of danger that may be visually unnoticed.

Conducted Eexperiments have proven that people are more receptive to audio monitoring of

radiation levels than to visual or multimodal (visual + audio) monitoring [31].

A recent application of sonification of time dependent data conducted by researchers at the

Department of Psychology at the University of Nebraska – Lincoln demonstrates the illustration

of climatic trends and variations using sonification. Monthly weather records of different years

from Lincoln, Nebraska were compared with recent observations. Investigations on the choice of

data-sound mapping schemes and temporal properties of the display were investigated in relation

to human auditory perception, attention and cognition [16].

Other current examples applications of sonification are described in Cummerow [7], DNA and

Protein Music [9], Flowers [13], Kramer [17], Life Music [18], Meinert [19], Nesbitt & Barrass

[22], and Noihomme-Fraiture, Scholler, Demousin & Simoff [24].

2.4 Time-Series

A time series is a set of data recorded over time [4]. Time series whose observations are taken at

fixed time intervals are termed discrete time; those whose observations are taken continuously

are termed continuous time. Unlike most statistical methods, which assume the data are

realizations of independent random variables (called random samples), the analysis of time series

is based on the assumption that successive values in the data are correlated over time.8 The Geiger Counter and how it works http://www-istp.gsfc.nasa.gov/Education/wgeiger.html

6

Time series analysis is the statistical methodology used to identify the nature of the phenomenon

represented by the sequence of observed measurements and to forecast future values. Most

approaches to time series analysis can be classified as time domain approaches or frequency

domain approaches.

2.4.1 Time-Series Analysis in the Time Domain (Stationary Processes and the

Autocorrelation Function)

Analyses in the time domain are methods based on the evolution of the series dependence over

time. The simplest of these methods is plotting the data versus time. If the series is stationary

(defined below), plots and inference based on the autocorrelation function are common time

domain methods.

Stationary processes are processes whose distributional properties do not vary with time.

Generally, a time series is said to be stationary if it has identical statistical

properties to those of the “time-shifted” series for each integer h. For

example, a time series is strongly stationary if the joint distribution of any subset

is the same as that of for any integer h.

Given a time-series with where the mean function of is given by

, we define the covariance function of :

for all integers r and s.

is said to be weakly stationary if its mean is independent of time ( is independent of t)

and if is independent of t for each h (that is to say that the covariance of the “time-

shifted” series is also independent of t). In what follows, the term stationary will refer to weak

stationarity unless indicated otherwise.

7

2.4.1.1 Sample Autocovariance & Autocorrelation

The autocovariance function (ACVF) of a stationary series at lag h is given by:

The autocorrelation function (ACF) of at lag h is given by:

The autocovariance function satisfies the following basic properties:

(i)

(ii) for all h

(iii) is an even function, i.e. for all h.

The autocorrelation function satisfies all the properties of an autocovariance function as

described above, and satisfies the additional condition . In fact,

can be considered the autocorrelation function of a stationary process if and only if is

an autocovariance function with .

The autocovariance and autocorrelation functions are used in the theory of time series, but

because they are population quantities, are often unknown in applications. In such cases, we

must rely on a set of observed data to estimate them using the sample

autocovariance and sample autocorrelation functions.

8

If the data are realized values of a stationary time-series then the sample autocorrelation

function will provide an estimate of the ACF of . This estimate may suggest one of the many

possible time-series models (e.g. idd noise, White noise, the Random Walk, First-order Moving

Average, etc.) as a suitable representation of the dependence of the data.

To define the sample autocorrelation function, let be observations of a time-series.

Then the sample mean of is given by , and t. The sample autocovariance

is then defined as:

Following the definition of the sample autocovariance, the sample autocorrelation is given by:

Note that the sample autocovariance and autocorrelation functions can be computed for any data

set and are not restricted to observations from a stationary time-series [4].

However, if the series is not stationary, their meaning as estimates may be unclear.

The sample ACVF and ACF provide a useful way to measure the degree of dependence among

the values of a time-series at different lags. They are also important when considering the

prediction of future values of the series in terms of the past and present values (further reading:

pg 46 of [4]).

9

2.4.2 The “Additive Classical Decomposition” Time Series Model [27]

A general approach to observing time series data is to plot (visualize) a graph of the observed

measurements against time. The plot will illustrate important properties such as trend,

seasonality, discontinuities and outliers.

The “classical decomposition” time-series model is one of the fundamentalearliest models used

in time series analysis. This model assumes that the data observed is composed of a systematic

pattern and random noise (error) which can make the pattern difficult to identify. The systematic

pattern is further divided into a trend component and a seasonal component. In summary:

Where Yt is the time-series observation at time t, Tt is the value of the trend at time t, St is the

value of the seasonal component at time t, and Et is the value of the error or irregular component

at time t.

A trend represents a systematic linear or nonlinear component that varies over time and does not

repeat within the time interval observed. A seasonal component may be similar in nature to a

trend but it repeats itself in systematic intervals over time (cycles). These two components may

co-exist within real-life data (e.g. sale of real estate properties may increase over a 5 year period

but may follow consistent repeating seasonal patterns). A series represented by the classical

decomposition model may be nonstationary because of the presence of a trend.

To estimate the trend component, a moving average designed to eliminate the seasonal

component is used. Let s = period of the seasonal component. If s is even then s = 2r, and the

corresponding moving average is:

10

If s is odd, then s = 2r+1, and the corresponding moving average is:

The de-trended data values are defined by .

To estimate the seasonal component, we begin by estimating for each of the s periods, where k

= 1, 2, …, s. For each k value, is the average of all ( ) where j is the range over all

values for which r < k +js n-r. We want the seasonal component estimates to sum to zero, so

is adjusted to obtain the seasonal component estimate:

The seasonal component, now defined to be where is the remainder when t is

divided by T (do you mean s?). A deseasonalized series Dt can be obtained once the seasonal

component has been established, by subtracting the seasonal component from the observed

value:

After having done this, we will be able to re-estimate the trend in the series. To accomplish this,

a regression model is fitted to the deseasonalized series Dt. If the trend indicated is linear, then:

where = random errors

If the indicated trend is quadratic, then:

11

The equation may be expanded for higher order polynomial equations. The fitted values are

denoted by .

Now the estimate, , of the irregular component of the series may be estimated by subtracting

the trend and seasonal components. This component checks on the adequacy of the entire model.

The irregular component is relative to the trend and seasonal components. The irregular

component will be autocorrelated over time, but should not have a trend.

12

3 METHODOLOGY

Our primary objective is to investigate creative and effective methods of sonifying statistical

data, specifically time-series.

Since time-series is time-dependent data, an intuitive way to approach the sonification problem is

to map the data into frequencies by using linear - chromatic scale9 mapping. This is called pitch-

based mapping. The range of data values can be obtained from a completed time-series and is

mapped into the specific frequency range. This process is described in section 3.1.1.1.

Our approach to the sonification of a time series starts by decomposing the time series into its

respective components using the Additive Classical Decomposition model. The resulting

components: its trend, seasonality, de-trended, de-seasonalized and irregular components are

then sonified. These various components are sonified using the pitch-based mapping method as

described previously.

Our designed sonification program in MAX enables the user to listen to the data in various ways.

The first sonification method involves listening to a single data file (or component) of a time

series at a time. This method is intended to allowprovide the user with means of a singleto focus

on the sonification of a particular component of a time series, and audibly. This method does not

have any clutter of sound sources thus provides a simple and direct method of listening to any

componentdata.

The second sonification method involves listening to a pair of data files A and B simultaneously.

Data file A will be heard over the left speaker and data file B over the right. This method allows

the user to compare between components or data simultaneously. Relative relationships between

the components may also be explored audibly through this method.

The third sonification method involves listening to a panning relationship between two data files.

Panning is a technique of controlling the balance of sound around the listener, i.e. how much of

the sound (volume) is coming from each speaker. In this study, the terminologywhen “data file 9 A musical scale in which an octave consists of twelve notes separated from one another by a half-step.

13

A is “panned by” danother data file B,” this means that data file A is played over both speakers

(stereo), but the balance is controlled by data file B. This method intends to explore the

possibility and effectiveness of a simple spatial sound representation of data.

Having established several solutions to approaching the sonification of time series, we needed to

design and implement programs to accommodate and execute these methods.

3.1 Statistical Analysis and Mapping Program

Since we were going to base the sonification of time series on the Classical Decomposition,

In the designing of this project, we needed a way to analyze time-series; that is, to break them

down into their components. In order to accomplish this, we looked to the technique of Classical

Decomposition. This is a standard method of breaking a time-series into its trend, seasonal and

residual components. Wwe needed to produce a computer program that could execute this

Classical Decomposition on variable-size data sets.

3.1.1 Designing the Program

First, we explored various open-sourced means that might be available. Our first thought was to

use a widely available GNU-licensed statistics package called R.10. R is a free implementation of

an original computing language called S, developed by Bells Labs, and marketed commercially

by MathSoft under the name S-PLUS. While R is quite extensible using several popular

programming languages, including C++, we couldn’t find any concrete documentation on using

it as a library for C++ (i.e. extending C++ with functionality from R). Although R has all the

functionality required to do the Classical Decomposition, we found it to be too lacking in such

areas as file I/O to be the basis language for the entire program. Thus, we decided not to use R.

Next, we looked into the possible use of MATLAB libraries. MATLAB, developed by

MathWorks, is a general technical computing language and application with built-in

functionality and libraries encompassing the ability to do everything and anything we needed for 10 http://www.r-project.org/

14

this project. MATLAB also has stand-alone libraries provided with it that allow it to be

compiled into a C++ program. These features made it a prime candidate for our project.

However, the reason we chose not to use it is that we lacked access to a Macintosh version of the

software. At this point in the project, we had already decided upon the use of the MAX

programming environment for the main Sonification programming. At the time of this writing,

MAX is only available in a Macintosh environment. InabilityWith a lock of being able to

compile our statistical program in a Macintosh environment, it would make full integration of the

software it impossible now and in the future for full integration of the software. Thus, we

decided against the use of MATLAB.

Finally, we decided that doing the entire analysis program in the C++ language would be the

most efficient route to take. We explored already available code, to see if any fit our needs.

Mathtools.net is a free repository of code managed by Mathworks. This repository includes

several languages, including C++. We searched this repository for code pertaining to time series

and the Classical Decomposition. We found nothing that was usable for the project.

Numerical Recipes in C11 is a book provided online by Numerical Rrecipes Software. This book

provides various well-known mathematical algorithms, coded in the C programming language.

We looked through this listing for anything pertinent to the project. There was nothing

pertaining to Classical Decomposition, so we decided, although there was an algorithm for

calculating a Fourier Transformation (note: all Fourier analysis approach has been removed from

the project, due to time constraint, and is only discussed as a suggestion for future expansion.).

This left us with the decision that we would code the algorithm for the Classical Decomposition

ourselves.

With the help of reference books [30, 6] and our advisor [27], By explanations and examples in

textbooks and discussion with the math component of our advisory, we acquired a basic

understanding of how to Additive Classical Decomposition model works. This was a basis to

start the coding. wWe produced a simple algorithm that took in the time-series data in the form

of a file (file formats discussed in the specification section) and calculated the three basic

11 http://www.library.cornell.edu/nr/bookcpdf.html15

components of the decomposition: trend, seasonal, and residual. To test this algorithm, we used

a set of airline passenger data over the course of 12 years. We used a pre-existing SAS

calculation of the Additive Classical Decomposition and checked the data produced by our

algorithm against it. Through consulting the method in reference books [30, 6] and clarifying it

with our advisor [27], we were able to work the bugs out of our algorithm. The algorithm was

improved to the point where it matched the output produced by the SAS program on the given

dataset.

Additions were made to the statistical program in the form of extra output. Originally, the

program output files pertaining to the three basic components of the Classical Decomposition:

trend, seasonal, and residual components. As we proceeded to decide what would make a

meaningful sonification, we added two new components to this: the de-seasonal and de-trended

components.

Our basic algorithm originally calculated the trend using a linear regression method. The main

reason for this was that the data we used for testing the algorithm was a linear series. We

acquired a quadratic time-series dataset from an online time-series data library12, and used this to

test an implementation of the quadratic regression method described in our reference material.

This extension allowed us to test varied time-series data for ourt sonification methods.

In order to guarantee that all the data of a sonified time-series is audible, another extension was

made to the statistical program. The data of a time-series has no limitation. It can span over any

values, and have any range. But in order to play the sonification, the range of the sonified data

has to be restricted to the note range of the MIDI instrument used (in our case, the grand piano).

A linear mapping algorithm (see 3.1.1.1) was added to the program in order to restrict any time-

series data to the audible range specified by our sonification medium.

A few modifications, or rather additions, were made to this mapping algorithm as well.

Originally, each component was linearly mapped according to its own range alone. This works

12http://www-personal.buseco.monash.edu.au/~hyndman/TSDL/ 16

well for single components, but when multiple components are played together, it is important to

have them centered and mapped relative and centered to each other. The linear mapping

algorithm was extended to allow all the components of the series to be mapped on their

collective range. This seemed to be more effective for playing individual components, but didn’t

always work for playing two components at a time. For example, say you were to play the

seasonal and residual components together. Both of these are generally small components

compared to the trend, so when playing them together, it is better to map them relative to only

each other, or effectively “zoom” (as it would be referred to graphically) them. The program

was therefore extended to allow each pair of components to be linearly mapped relative to each

other.

Another modification was necessary when we started using the pan relationship. Originally, we

used the individuallysimple mapped (not mapped relative to another component) files to

accomplish the panning. We quickly noted that this didn’t seem to work properly. On closer

inspection, we discovered that the range for the panning is different from the MIDI note range of

the grand piano. The program was therefore extended to allow each component to be linearly

mapped on the panning range, in order to allow that component to be used to control the

panning.

This is the basic step-by-step process that we followed to create our statistical analysis program.

3.1.1.1 Linear Mapping Algorithms

The following algorithms are incorporated into the statistical analysis program. Once a time

series has been decomposed into its respective components, a mapping method is required to

accommodate any numerical data (in this case from any given time series) and to map it into a

specific range that our sonification program can handle. Since the piano13 instrument is used in

this project as the primary sonification timbre14, the numerical range of tones for the piano is to

be considered.

13 See section [] for the reasoning behind the initial choice of the piano timbre.14 Tone quality or tone color = difference between sounds. Each sound is composed of a unique tonal mix of fundamental and harmonic frequencies (additive or subtractive synthesis of waveforms).

17

The piano instrument has eighty-eight notes across its keyboard, spanning from A (27.5Hz, 0 th

octave) to C (4186.0Hz, 8th octave). These notes and theirits corresponding octaves are denoted

by A0 and C8 respectively. The first note on the piano keyboard, A0, is represented in the MAX15

with the value of 21. The final note on the piano keyboard, C8, is represented in MAX with the

value of 108.

The numerical values of given time series and its components have to be mapped linearly into a

chromatic scale. This is the same as being able to produce every note of the range of notes from

the piano.

Several algorithms were developed to handle this mapping for three cases. The first case

involves mapping only one data file at a time into the MAX range. The second case involves

mapping two data files relative to one another and centered at a reference point into the MAX

range. The third case is a mapping algorithm to handle the mapping of a data file into a MAX

range suitable for panning controls.

Before describing the mapping algorithms, a propagate function has to be defined as it is

necessary for application in all three cases. The function is defined as follows:

The first case, used in the primarily in the first sonification method, for mapping a single data

file (either a time series or a particular component) is described by the following algorithm:

15 See 3.2 for a description of the MAX programming environment.18

For the second case, particularly used in the second sonification method, two data files need to

be mapped into the desired range described, then mapped relative to each other and centered at a

reference point. The following algorithm describes the relative mapping of the two data files into

the described MAX range.

(Check the above. The m2 and M2 definitions were given as m1 and M1.)

Having done this, the two new mapped data files now need to be centered at a reference point.

The reference point is dependent upon the range that the two data files span. The data that spans

a smaller range will be centered respective to the mid point the data that spans the larger range.

The reasoning behind transposing the range such that it centered about a point is due to the way

we perceive sounds. It was noted during the development of this project that any variation heard

19

in the lower frequencies wasere hard to distinguish. The seasonal and irregular components

usually fell within these lower frequencies, and thus were hard to identify. Transposing them to a

center reference point with respect to a comparison component proved beneficial to the

perception of the sonification. The following algorithm allows this:

An expansion of this algorithm (similar in structure) also accommodates several data files to be

mapped relative to each other, in the case of two or more (see 7.2). This is used for mapping the

time series and all its decomposition components such that they are all relative to one another

and are centered at a specific reference point.

Finally, a third mapping algorithm is needed to map the values in a data file intended for use

with the third sonification method into a range suitable for the panning controller in MAX. The

algorithm essentially follows the same structure as the first case, with the exception of the

spanning range of the values. The data file values have to be mapped linearly into a range

between 0 and 127 (over 128 values) instead of 21 to 108 (over 88 values), since the MAX

panning controller handles values from 0 (extreme left pan) through 127 (extreme right pan). The

algorithm is described as follows:

20

3.1.2 Program Specifications

3.1.2.1 File Formats

All data files used by our programs are simple ASCII text files. All files are of file specification

type (2), except for the original data file.

3.1.2.2 File Specification Type (1)

This is the specification for the time-series data files provided to our statistical analysis program.

The first line of this file must contain an integer that represents the number of functional pairs

found in the file. The rest of the lines of the file consist of these functional pairs. A functional

pair consists of a time value followed by a space and then the corresponding value of the series

for that time. There should be only one functional pair per line. The file should contain no

ending blank lines. The fFollowings is an example of this format:

50 101 23

2 343 214 12

21

3.1.2.3 File Specification Type (2)

This is the specification type used for all the output files from the statistical program. It is much

simpler than the one above. All this format consists of is the functional values of the

series/component, one per line. Again, no blank lines should appear at the end of the file. The

following Follows is an example of this format:

1023342112

3.1.2.4 File Name and Extension Conventions

The statistical program uses a variety of file extensions and naming conventions to identify

different files. Follows is a table of the file extensions used and what they stand for:

Extension Description

.dat

Specifies the original data of the series. The original file supplied to the program should be a .dat. The program produces one .dat file, which doesn’t follow this convention. It is the original data for the time-series with the time and number of data pairs portions removed, in order to play the original data in the sonification patch.

.trn Specifies that the file corresponds to the trend component

.ssea Specifies that the file corresponds to the seasonal component

.err Specifies that the file corresponds to the residual component

.dsea Specifies that the file corresponds to the deseasonal component

.dtrn Specifies that the file corresponds to the detrended component

Table 1 - File extension descriptions for the statistical analaysis program

22

The program also uses a series of underscore ‘_’ accentuated headings to identify the content of

the file. The following Follows is a table of these markers and what they stand for:

23

Extended file markers Description

_max: A file marked with this header alone is mapped to its own range (independent of other components)

_rel:

A file marked with this header is mapped relative to the other components marked with the header (i.e. all the components linearly mapped into the same range)

_rel_X:Where X is an integer. These files are mapped linearly in pairs. Any two files that have the same number (X) are specified as a mapped pair.

_pan: File marked with this header are mapped individually on the panning range.

Table 2 - Extended file marking descriptions for the statistical analysis program

Files that do not have a header but have an extension corresponding to a component (as specified

above) are the originalactually data for that component, not mapped iIn any way (useful for

graphing purposes).

3.1.3 Using the Program

The statistical analysis program is standard C++ and should be portable to any compiler. It uses

no special libraries. Originally, it was developed on Digital UNIX on a GNU g++ compiler

version 2.95.2.

Once compiled, the program runs like a simple console application. After execution, the

program will prompt the user for the input file. The user is then prompted to enter an s value.

This is the period of the seasonal component to use for the analysis. The user is then asked to

choose whether he/she would like to perform a linear or quadratic regression on the data. After

all these choices, the program will perform the Additive Classical Decomposition on the data, as

specified by the user, and will output several files with the results. These files follow the

specification outlined above. The names of the output files are generated based upon the name

of the input file. For example, if an input file called Data.dat is supplied, the program will output

the following files:

24

Data.datData.dseaData.dtrnData.errData.sseaData.trnData_max.datData_max.dseaData_max.dtrnData_max.errData_max.sseaData_max.trnData_pan.dseaData_pan.dtrnData_pan.errData_pan.sseaData_pan.trnData_rel.dseaData_rel.dtrnData_rel.errData_rel.sseaData_rel.trnData_rel_0.sseaData_rel_0.trnData_rel_1.errData_rel_1.trnData_rel_2.dtrnData_rel_2.trnData_rel_3.dseaData_rel_3.trnData_rel_4.errData_rel_4.sseaData_rel_5.dtrnData_rel_5.sseaData_rel_6.dseaData_rel_6.sseaData_rel_7.dtrnData_rel_7.errData_rel_8.dseaData_rel_8.errData_rel_9.dseaData_rel_9.dtrn

These files represent the various components and mappings that fuel our sonification methods.

They follow the naming conventions outlined above. These files are formatted properly such

that they can be played (sonified) using the sonification portion of this application.

25

3.2 Sonification Program

During the initial stages of the project, it was decided that we would engage the application of

sonification towards time series in MIDI. Several programming environments were investigated,

and it was decided that we were to embark on the implementation of the project using the MAX

programming environment.

MAX is a graphical programming environment for music and media applications currently

available on Macintosh systems. Generally, MAX allows versatile control over anything with

anything. For instance, you can use MIDI to control a laserdisc player, or mouse movement to

control the playback of a QuickTime movie [8]. MAX also provides as a platform to customize

the logic of building an interface to interactive media. Events are executed with millisecond

accuracy, create complex mappings for incoming data, and run a large number of operations in

parallel. Our project team was also exposed to prior background in MAX programming; this was

another factor in our choice of MAX over other programming environments.

3.2.1 Designing the Program

There are several layers to the design of the MAX sonification program. The first layer contains

the main engine of the sonification named Core Hearing Data Patch (see Appendix 7.3.2). This

patch serves as a basis for the first sonification method, where the sonification of one data file is

utilized. Gary’s Floating Point Parser (see Appendix 7.3.1) is used to process incoming data and

underlies every patch for the sonification program. The integer portion of the incoming data is

fed directly to an assigned MIDI channel where every integer value corresponds to a particular

music note/tone. For example, the numeric value 60 produces middle C or C4 (261.63Hz). This

note is located in the fourth octave of a piano keyboard. Consequently, the numeric value 61

produces the next note above C4, which is C#4 (277.18Hz). This is called the pitch based

mapping, where a range of values (0 through 127) represents a corresponding musical note.

26

3.2.1.1 Parser Patch (fl_parser)

We created a parser in MAX to take the numbers in the files created by the statistical analysis

program and reproduce them so MAX can use them. The logical question in this case would be

“Why can’t MAX just read the files and get the numbers normally?”. While MAX does have

built-in functionality for handling files, it does not read these files as one would expect. Instead,

it reads a file character at a time, and the data it actually reads is the ASCII16 value for each

character.

Because MAX reads these values, we needed a way of converting them into the original

numbers. Also, numbers aren’t the only characters of interest in the files. In the same token that

MAX cannot differentiate what is a number and what isn’t, it also has no knowledge of what a

decimal point character is, or even a space or end-line character. All of these characters are

essential to the parsing of our data. In addition, since MAX is only able to identify file

characters in their ASCII values, it must be introduced to what these ASCII values correspond to.

Each key on the keyboard has an ASCII value assigned to it. When the key is pressed, it triggers

this value to be sent to the operating system, which in turn, most times, causes a particular

character to be displayed on the screen. This can also be used as a reverse-lookup, which is

exactly how MAX is to be made aware of what’s in the file. The numeric keys have ASCII

values 48 higher than their numeric value; therefore, in order to get the original values, 48 is

subtracted from the ASCII values. The ASCII value of the decimal point is 46. The ASCII

values of space and end-line are 32 and 13 respectively. These are the values that MAX needs to

process in order to understand the data in the files provided to it for our project.

Now the actual parsing process will be described. This explanation is divided into sections based

on different parts of the patch (see Appendix 7.3.1). First, what happens when the parser is

started by hitting the initializing bang (“Bang” means what?) is covered. This isn’t a fancifully

labeled button since the parser is not to be executed individually, but rather as a subroutine (or

subpatch) in other patches. When the patch is executed, all of the individual components are

initialized. All the counters, accumulators and variable number boxes are zeroed, all the

16 http://glossary.its.bldrdoc.gov/fs-1037/dir-003/_0374.htm27

collections are cleared, the gate is set to integer numbers, and the metronome is started. The

system is set to a metronome for automation purposes. Otherwise, the user would be required to

repeatedly hit a button for every character read from the input file.

When the metronome is hit, it begins incrementing the attached counter in 1ms intervals. The

attached counter is an index into the filein module, one index per character read. Every time a

character is read from the file, the ASCII value of that character is displayed in the number box

connected to the outlet of the filein module. It is then passed through a series of conditional

statements to determine what character it is, and what course of action should be taken. If it isn’t

a space (ASCII value 32), end-line (ASCII value 13) or decimal point (ASCII value 46), it is

determined to be numerical. Forty-eight is subtracted from the ASCII value, and the new value

is passed to the gate. The gate passes the value to one of two accumulators, depending upon if it

is in the integer or decimal mode (described below). Regardless of which accumulator it is

passed to, the same process occurs. The current value stored in the accumulator is multiplied by

10 (next place value), and the new value is added to it. This ends this process of execution, and

it is repeated for the next character read off the filein module.

If the character is determined to be a decimal point, execution proceeds to a message box that

sends the signal to the gate to change to decimal mode. The mode of the gate determines which

of the accumulators and, ultimately which of the collection objects, in which the data being read

will be stored. (This last sentence is confusing.) The integer and decimal portions of the

numbers are kept separately because they are used for different purposes within the other

sonification patches (explained later in the document).

If the character is determined to be a space or an end-line character, execution proceeds to get

ready for the next number to be read (in this case, number means an entire piece of numeric data,

for example 143.234). The gate is set back to integer mode, and both accumulators are banged.

This causes the numbers that were stored in them to be sent out their respective outlets. Both

outputs are combined with the number stored in their respective index counters. This combined

data is prepended (“prepended” means what?) with the set directive, and each is sent to the

respective collection object (one for the integer data and one for the decimal data). The

28

collection is a simple MAX storage object, which is similar to an array in C++. Any type of data

(character string, integers, floats, or any combination) can be stored. Data is stored by indices.

The indices, in this case, are received from a pair of counters that are incremented every time a

number is read. Continuing with the execution, both index counters are incremented, both

accumulators are zeroed by multiplying their stored values by -1 and adding the values.

Execution then proceeds to read the next value from the file.

This process continues until the end of the data file is encountered. End-of-file is a well defined

state in the MAX filein module. When this occurs, execution diverts into two paths. The first

path bangs on both of the accumulators and sends the output to the collections, as was done for

space and end-line (because the file format doesn’t allow for blank lines, the end-of-file will

occur after the last data entry). The second execution path sends a signal out the outlet of the

entire patch to tell the executing patch to proceed with execution. Execution of the Floating point

parser patch is complete.

Please note that the possibility of the read character being a space is included in case the data in

the file is space-separated instead of line-separated. The file specification for MAX input files

details that the files should consist of only numeric data and end-line characters (one piece of

data per line). If only files created using the statistical analysis portion of this project are used,

as is encouraged, this option will never be executed. This is provided in case users use their own

files, to allowas flexibility in the file specification.

3.2.1.2 Core Hearing Data Patch (Core.maxb)

As a starting point, the grand piano timbre was chosen as the choice of timbre to convey the data

audibly. The reasoning behind this is that since the MIDI sampling range of instruments are

based on the actual physical range of the particular instruments in the physical world, the piano

accommodates the widest general key range. This allows any incoming data to be mapped into a

wide musical key range allowing more room for perception of tonal changes or other subtle

variations within the data or tonal changes. The possible notes on a grand piano in the MIDI

29

environment ranges from the numeric value of 21 through 108, which gives us a possible of 88

whole notes.

However, whole notes may only be used to represent integer values. Given that most time series

may not be perfectly fitted into integers, but can comprise of floating point numbers (as most real

world observations are never perfectly rounded), we needed to design a patch to allow the

representation of the floating points as well.

A pitch bend module solves this. It bends a whole note pitch proportional to the scale of its

floating point. For example, the numeric value of 60.5 would be broken down into two parts: 60

and 0.5. 60 is represented into a whole note pitch as described previously. It floating point

portion however, controls the pitch bend module. Hence, 60.5 would produce a tone exactly half

way between the tone of the numeric value 60 and the tone of the numeric value 61. That is to

say that 60.5 would produce a tone half way in between C4(261.63Hz) and C#4 (277.18Hz)

which is approximately (269.41Hz).

The basic core also allows user control over the tempo or speed at which the sonification is

played back. Note velocity is also another input feature allowing the user to set how hard each

note is struck (the initial volume of each note struck on a MIDI instrument).

3.2.1.3 Core II Hearing Data Patch (Core2.maxb)

The second core patch, Core II Hearing Data Patch (see Appendices 7.3.3, 7.3.4), is an extension

of the first core patch. Its functionality is extended to accommodate the input of two data files

instead of one. Each of these data files isare treated the same way as described in the first core

patch. The only difference is the assignment of two distinct MIDI channels to the respective data

files to allow one data file to be played audibly on a right speaker, and the other on the left.

3.2.1.4 The Panning Relationship Patch (PanRel.maxb)The Panning Relationship Patch (see Appendix 7.3.5) is designed for the third sonification

method. It allows the first input data file to be processed in a manner similar to that described in 30

the first core, and the values of a second input file to control the panning controller for the notes

of the first file.

3.2.1.5 Hearing Data Main Patch (Hearing Data.maxb)

The main patch that ties the three described patches together is the main Hearing Data patch (see

Appendices 7.3.6, 7.3.7, 7.3.8). This features the user interface and serves as the main control of

the sonification program.

3.2.2 Program Specifications

The sonification program is basic in terms of input specifications. The statistical analysis and

mapping program has been tailored to fit the specifications of this program. The typical file

formats have been described in 3.1.2. The sonification program is currently dependent on MAX

run time files. These run time files are available for download on Cycling 74’s company

website17.

3.2.3 Using the Program

By opening the Hearing Data MAX binary file, the user interface will automatically be launched.

The user can then follow the listed on- screen instructions listed in steps (see Appendix 7.3.8).

The program consists of four steps. The first step is to initialize the tempo (in milliseconds) at

which the sonification will be played at. This done by simply dragging the mouse over the

numeric window until a desired value is reached or the user may choose to directly input the

tempo value using the keyboard. If no value is initialized, the default tempo of 100 milliseconds

will be automatically initialized.

17 http://www.cycling74.com/31

The second step is to initialize the note velocity. This may be done by simply hitting any key on

the MIDI keyboard with the intended velocity. The user may also choose to drag the mouse over

the numeric input window until the desired value is reached. A default note velocity of 60 will be

initialized if no input is defined.

The third step involves selecting a method to listen to theyour data (or method of sonification).

Upon selection, the user will be prompted a file browser to select the desired input data file.

Once the data file(s) have been selected, the user must allow some time for the loading of the

data. Once the data is loaded, the toggles will indicate the load status of the corresponding data

files.

After the files are loaded, the user can then hit the PLAY button to listen to the selected data

using the selected sonification method.

3.3 Testing: Controlled Experiment using Human Subjects

Once the sonification methods and applications hadhave been established by the project team,

the efficacy of these methods was investigated by conducting a controlled experiment on human

subjects. The experiment was designed to accommodate comparisons between the effectiveness

of the three sonification methods subject to different condition variables (with respect to the

types of time series sonified). From this investigation, we hope to gain insight into the inherent

problems of the developed mapping and perhaps even sonification in general for naturally

present (What does “naturally present” mean?) time-series data.

3.3.1 Designing the Experiment

The experiment accommodateds twenty-eight human subjects, whom are selected randomly.

(Describe how they were selected. They were self-selected, not randomly-selected, by the way.

The randomization consisted in the assignment of experimental conditions to the subjects.)

This has to be described more directly and clearly. The main points are:

32

1. For each sonification method, each subject was asked to identify two time series by

listening to sonification of the data. While listening to the sonification, they were to

choose the graph of the sonified series from among six possible graphs displayed to them

on a piece of paper. The subject could be asked to identify the same series twice.

2. Each subject was tested using each of the three sonification methods.

3. For each sonification method, the two sonified series were related by one of four

conditions: same trend and same seasonal components, same trend and different seasonal

components, different trend and same seasonal components, and different trend and

different seasonal components. Each subject was tested using 3 of these 4 possible

conditions, with a different choice corresponding to each sonification method.

4. Give the design (i.e., how the sonification method and conditions were assigned to

subjects).

Subjects were assigned to separate experiment blocks, each block containing seven randomly

assigned subjects (see Table 5). Each block wais subject to different conditions respective to the

sonification methods. The conditions describe specific traits of the given two sonified time

series to be identified. Four specific conditions were designed in order to investigate the effect of

varying time series traits on the efficacy of the sonification methods. For example, condition 1

comprises of two sonified time series both having same trends but differ in seasonality. These

conditions are further described in Table 3.

Each subject is required to listen to all three different sonification methods in a given random

order of sequence. For example, subject 1 listens to sonification method 3 first, then sonification

method 2, and finally sonification method 3. Subject 2 however, listens to method 1, method 2,

followed by method 3. The different ordering of sequence of the sonification methods in which

the subjects were exposed to first was to eliminate any possible correlation due to the possibility

of subjects “learning” the data and sonification traits. Table 4 describes the given order of

methods for each subject.

33

Block Subjects Order Method 1 Method 2 Method 3

1

1 A

Condition 1 Condition 2 Condition 3

2 B3 C4 D5 E6 F7 A

2

8 B

Condition 2 Condition 3 Condition 4

9 C10 D11 E12 F13 A14 B

3

15 C

Condition 3 Condition 4 Condition 1

16 D17 E18 F19 A20 B21 C

4

22 D

Condition 4 Condition 1 Condition 2

23 E24 F25 B26 C27 F28 A

Table 3 - Condition and Order table for Subjects

Order Sonification Method Sequence

A 3, 2, 1B 1, 2, 3C 2, 3, 1D 1, 3, 2E 2, 1, 3F 3, 1, 2

Table 4 - Order of Sonification Methods imposed on Subjects

Condition Trend Seasonal

1 same different

2 different same

3 different different

4 same same

Table 5 - Condition Descriptions

34

3.3.2 Experiment Procedures

The experiment was conducted in two isolated studios located in the Computer Music Labs,

WPI. Each subject was proctored on a one-to-one basis by one of the project team, allowing the

experiment to be conducted for at most only a possible of two subjects at any given time.

Subjects were given an introductory handout (see 7.4) describing the objectives and instructions

of the controlled experiment. A sheet containing an example of a time series (airline data) and

its respective decomposition followed the introductory handouts to illustrate the classical

decomposition method described in the introductory handout (see 7.5). The first five minutes

were given to allow the subjects to get comfortable with the experimental procedures using these

handouts. A short demonstration describing the structure of a time series and the sonification of

the Additive Classical Decomposition followed. The demonstration also included some

examples of sonifications of the airline data example using the different methods described with

an example time series.

Subjects were then given a handout (see 7.6) containing six unlabelled time series graphs.

Subjects weare later required to label two of the graphs as being those they had heard

accordingly after listening to specifically given sonifications of two of the graphs.

For sonification method 1 (listening to one component at a time), subjects were presented with

two sets of audio files, P.1 and Q.1, each corresponding to different sonifications of one of the

six graphed time series. Each set containeds three files: one for the trend, one for the seasonal,

and one for the irregular component.

For sonification method 2 (listening to two components simultaneously), subjects were presented

with two sets of audio files, P.2 and Q.2, each corresponding to different sonifications of one of

the six graphed time series. Each set containeds three files: one for trend and& seasonal, one for

trend and& irregular, and one for seasonal and& irregular components.

For sonification method 3 (listening to a panned relationship between two components), subjects

were presented with two sets of audio files, P.3 and Q.3, each corresponding to different 35

sonifications of one of the six graphed time series.the time series. Each set containeds four files:

one for “trend panned by seasonal”, one for “trend panned by irregular”, one for “seasonal

panned by trend”, and one for “irregular panned by trend”.

Subjects were then required to listen to the respective sonifications based on assigned order as

described in 3.3.1 and label the six graphs accordingly. At the end of the experiment, subjects

were then required to fill out a quick questionnaire.

36

4 RESULTS AND ANALYSIS

The table above is a summary of the results of the controlled experiment. Working with our

advisor Prof. Petruccelli, the results were fitted into SAS for analysis.

EXPERIMENT RESULTS (1= CORRECT, 0 = WRONG)

Sonification Method 1 Sonification Method 2 Sonification Method 3Subjects P1 Q1 P2 Q2 P3 Q3 Total Correct

1 1 1 0 0 1 1 42 0 0 1 1 1 1 43 1 1 0 1 1 0 44 1 1 1 1 1 1 65 1 1 1 1 1 1 66 1 1 1 1 1 1 67 1 1 1 1 1 1 68 1 1 1 1 1 1 69 1 1 1 1 1 0 5

10 1 1 1 1 0 0 411 0 0 0 1 0 0 112 1 0 1 0 1 0 313 0 1 1 1 1 0 414 0 0 1 0 0 1 215 1 1 1 1 1 1 616 1 1 1 1 0 1 517 1 1 1 1 1 1 618 1 1 1 1 1 1 619 1 1 1 1 1 1 620 1 1 1 1 1 1 621 1 1 1 1 1 1 622 1 1 1 1 1 1 623 1 1 1 1 1 1 624 1 1 1 1 0 0 425 1 1 1 1 1 1 626 1 1 0 0 1 1 427 1 1 1 1 1 1 628 1 1 1 1 1 1 6

SUBTOTAL: 140

Condition Legend

Condition 1

Condition 2

Condition 3

Condition 4

4.1 Generalized Linear Model [28]

(To begin with, I don’t think you need the general formulation. I provided you with that as background. You can start with the logit model for the pij’s)

Table 6 - Summary of Experiment Results

The Generalized Linear Model was used to fit the results of the experiment. Twohe models were

used. The first is described as follows:

State Equation:

Observation Equation:

Here is a vector of independent variables or factors. is a vector of parameters such that

.

is called a link function. It links the variables giving the state of the system ( ’s) to the

parameters in the destination of the observations ( ’s).

is a family of distribution functions indexed by the parameter vector .

One of our models:

For our case, thus can be written as

State equation: where is called the logit function, .

Observation equation:

The role of is played by , which for us is the probability of making a correct selection in

block i using method j where . The role of is played by

which can be thought of as an overall level ( ) plus an effectamount due to the block ( ) plus

an effectamount due to method ( ). is what is actually observed (i.e. how many correct out

of two guesses)., and has is athe binomial distribution with as parameters.

There is one further complication for us: the same subject used all three methods, which might

lead to correlation. We have accounted for these by using a fitting method that assumes

observations from the same subject are correlated. The method is called the Generalized

Estimating Equation (GEE).

A similar model was fit in which was replaced by (the conditions). We could not fit both

factors in the same model because of the design.

For the “methods” model, none of these methods differed subsequently from method 3, but the

results from block 2 were significantly different than from the other blocks. The model with only

blocks was *As a result a model with only the block factor was fit, and level .05 CI’s for the

(i.e. probability in block i) were obtained:

Block Interval1 (.62, .95)2 (.34, .81)3 (.79, 1.00)4 (.64, .98)

Table 7 - Results summary

Notice that all the interval values are significantly greater than the value that indicates of

guessing. Similar results were obtained for the “conditions model”.

In summary,

1. Sonification was effective in the sense that rates of correct identification were statistically

significantly higher than the value 1/6 that would be expected if the subjects were

guessing.

2. T here were no differences found among methods or conditions in the probability of

correctly identifying the sonified time series.

3. There was a significant difference due to low identification rates in block 2. This result

remains unexplained.no difference with respect to methods or condition. (sentence). In all

blocks, sonification works.

4.

5 CONCLUSION

The results of the experiment suggest that the sonification methods are effective. This concludes

the first study of sonification performed at WPI. The project has directed the first sonifications of

the Additive Classical Decomposition model. (I don’t understand the last sentence.) In the

process, a versatile sonification program was developed to accommodate our sonification

methods. The program may be revised and expanded easily to accommodate different kinds of

data and other approaches to sonification methods for future studies (see 5.1).

5.1 Recommendations for Future Work

From the beginning of the project, despite the fact that we knew we were using multiple distinct

programming languages, integration between the individual systems was discussed as a

desirablen option. that would be nice to do. However, it became unrealistic as the project

progressed. MAX has a robust system for expansionding it with C++, but most of the

documentation available covers accomplishing expansions similar to what MAX already does

(sound processing), not processes that depend upon the flexibility of C++ itself. We have no

doubt that it could be done, but given that MAX is currently restrained to the Macintosh, the lack

of a freely available and robust C++ programming environment for the Macintosh (at least on the

machines we used), and our relative newness to programming on the Macintosh as well as to

MAX itself (at the beginning of the project), time constraints outweighed us inprevented this

attempt. We do suggest it as a possible expansion to the project, as we see it as a valuable

resource to be able to provide a MAX patch with a single data file and have the patch decompose

the series and provide the user with sonification options all at the same time.

The fully automated integration would benefit users intending to listen to their data without the

hassle of learning how to manually operate the mechanics ofr both programs. The integration

would simplify training and learning of the program, and would enable the user to focus more on

analyzing and hearing the data.

Several cosmetic modifications may be also introduced to enhance the current user interface.

Some considerations may be placed on making the interface more intuitive for any user.

Graphing capabilities may also be considered within the MAX patch. This allows the user, as the

data is being sonified, to immediately see a graph of what’s being sonified. This would

especially be useful as this technology is advancing, in order to visualize the sonified data and be

able to relate and compare it to the sonification. Once data is loaded into the programmed data

structure in MAX, a graphing module may be implemented to plot a simple graph of the given

time series. Due to time constraints, this project was unable to accommodate this feature.

Besides future work extending in the design of the programs, the analysis of a time series in the

frequency domain, i.e. Fourier analysis, can be used to produce different a kind of sonification. A

frequency-domain approach would use the same representation of the series in its Fourier form

(see 7.1), and apply the spectral density estimates in order to map a frequency band with its

respective powers. The different frequencies across the band may be represented by different

corresponding timbres while the respective band’s powers may be expressed using volume.

A combination of the time-domain and frequency-domain approach may also be further

investigated. We could assign separate audio channels (left/right pan) for each approach and

have them audibly express their properties simultaneously.

6 REFERENCES

1. Alten, S. (2002) Audio in Media 6th Edition, Wadsworth Publishing Company.

2. Barger, J. (2000) Natural Language Translation, Natural Language Processing (NLP)

Natural Language Translation, Natural Language Processing (NLP),

http://www.robotwisdom.com/ai/nlp.html

3. Bloomfield, P. (1976) Fourier Analysis of Time Series: An Introduction, Wiley & Sons,

Inc.

4. Brockwell, P. & Davis, R. (2002) Introduction to Time Series and Forecasting 2nd

Edition, Springer text in statistics.

5. C. Scaletti (1994) "Sound synthesis algorithms for auditory data representations," in

Auditory Display (G. Kramer, ed.), no. XVIII, (Jacob Way, Reading, MA 01867), pp.

223251, Santa Fe Institute, Addison-Wesley Publishing Company.

6. Chatfield, C. (1980) The Analysis of Time Series, An Introduction 2nd Edition, Chapman

and Hall Ltd., London.

7. Cummerow, Daniel, The Sound of Mathematics, http://www.geocities.com/Vienna/9349/

8. Cycling ’74, http://www.cycling74.com/

9. DNA and Protein Music, http://linkage.rockefeller.edu/wli/dna_corr/music.html

10. Dunn, John, Algorithmic Arts, http://algoart.com

11. Flowers, J. & Hauer, T. (1993) “Sound” alternatives to visual graphics for exploratory

data analysis, Psychonomic Society, Inc.

12. Flowers, J. & Hauer, T. (1995) Musical versus Visual Graphs: Cross-Modal Equivalence

in Perception of Time Series Data, Human Factors and Ergonomics Society September

1995.

13. Flowers, J. (1996) Data Sonification from the Desktop: Should Sound be part of Standard

Data Analysis Software?, University of Nebraska – Lincoln

14. Flowers, J. Buhman, D., Turnage, K. & Bonebright, T. (1996) The Effects of Task

Demands on the Equivalance of Visual and Auditory Representations of Periodic

Numerical Data, Behaviour Research Methods, Instruments & Computers.

15. Flowers, J., Buhman, D. & Turnage, K. (1997) Cross-Modal Equivalence of Visual and

Auditory Scatterplots for Exploring Bivariate Data Samples, Human Factors 1997, 39(3),

341-351

16. Flowers, J., Whitwer, L., Grafel, D. & Kotan, C. (2001) Sonification of Daily Weather

Records: Issues of Perception, Attention and Memory in Design Choices, Proceedings of

the 2001 ICAD, Finland.

17. Kramer, G. & International Community for Auditory Display (1997) Sonification Report:

Status of Field and Research Agenda.

18. Life Music (Sonification of Proteins),

http://mitpress2.mit.edu/e-journals/Leonardo/isast/articles/lifemusic.html

19. Meinert, K. (2002) Introduction to Sonification in Virtual Environments,

http://www.vrjuggler.org/pub/sonification.notes.ieeevr2002.pdf

20. MIDI music from DNA, Protein and Math, http://education.llnl.gov/msds/music/

21. Najjar, L.J. (1996) Multimedia Information and Learning, Journal of Educational

Multimedia and Hypermedia, 5, 129-150,

http://mime1.gtri.gatech.edu/MiME/papers/multimedia_and_learning.html

22. Nesbitt, K. & Barrass, S. (2002) Evaluation of a Multimodal Sonification and

Visualisation of Depth of Market Stock Data, International Conference on Auditory

Display

23. Neuwirth, E. (2001) Listen to Your Data – Sonification with Standard Statistics Software

24. Noihomme-Fraiture, M., Scholler, O., Demoulin, C. & Simoff, S. (2002) Sonification of

Time Dependent Data, University of Namur, Belgium.

25. Patricio, C. Pitch Detection Methods Review,

http://ccrma-www.stanford.edu/~pdelac/154/m154paper.htm

26. Pellman, S. (1994) An Introduction to the Creation of Electroacoustic Music, Belmont,

California: Wadsworth Publishing Company

27. Petruccelli, J. (2002) The Classical Additive Decomposition of A Time Series, Handout

Notes.

28. Petruccelli, J. (2003) Generalized Linear Model and SAS result analysis, Notes.

29. Petruccelli, J., Nandram, B. & Chen, M. (1999) Applied Statistics for Engineers and

Scientists 1st edition, Prentice Hall, Inc., 83 – 117.

30. Pollock, D. Lecture Notes in Time-Series Analysis and Forecasting,

http://www.qmw.ac.uk/~ugte133/courses/tseries/proserie.htm, Queen Mary College, The

University of London.

31. Tzelgov, J., Srebro, R., Henik, A., & Kushelevsky, A. (1987). Radiation detection by ear

and by eye. Human Factors 29(1), 87-98.

7 APPENDICES

7.1 Time-Series Analysis in the Frequency Domain

Frequency domain analysis is concerned with the decomposition of the observed series into

periodic components called a spectral representation. The observed series is assumed to satisfy

the conditions of stationarity. The spectral representation of a stationary time-series

decomposes the variation of the process into portions attributable to various frequencies. This

method is an alternative approach of analyzing the process that is equivalent to the time domain

analysis based on the autocovariance function.

The spectral density of a stationary process defines the frequency decomposition of the variance.

7.1.1 Spectral Density

Given a zero-mean stationary time-series with the autocovariance function , the spectral

density of is defined as

where and i is equal to the square root of negative one. The function

has period since it is expressed in terms of cos and sin which also have period .

satisfies the following properties:

(i) f is an even function,

(ii) for all

(iii)

The third property expresses the autocovariances of a stationary time-series with absolute

summable ACVF as the Fourier coefficients of the nonnegative, even function on [4].

Note also that since is the variance of , the third property implies the spectral

decomposition of the variance: .

For , there is a corresponding spectral density if:

(i) for all

(ii) for all integers h.

Spectral densities are also unique. For example, if f and g are two spectral densities

corresponding to the autocovariance function , then f and g have the same Fourier

coefficients and therefore both are equal [4].

The relationship between the time-domain and frequency-domain analyses is implied by the

relationship between the spectral density function and the sequence of autocovariances. The

autocovariance may be obtained through a Fourier transform of the spectral density function. The

spectral density function is equal to the Fourier transform of autocovariances.

7.1.2 Discrete Fourier Transform

Given a realization of a stationary time-series with spectral density , an

estimate of f is given by the periodogram . To define the periodogram, we first define the

discrete Fourier Transform (DFT) as , where for

. , where is the cosine

transform and is the sine transform. The periodogram is defined as

, and measures the sample variance or power at . The periodogram

is extended to any by defining where is a multiple of

closest to .

It can be shown that as , and . This

suggests that be used to estimate , . However, is not a consistent

estimator, since as , converges in distribution to an exponential random variable with

mean .

To obtain a consistent estimator, the periodogram is averaged over adjacent frequencies. The

result is called a discrete spectral average estimator.

The discrete spectral average estimator is defined as where

the bandwidths and as , and the weight functions satisfy

, and .

7.2 Analysis and Mapping Program (C++ Source)

/*************************************************************** * Author : Gary Barber Jr. * * Version : 1.0 * * Project : Hearing Data * * Partner : Nedunceliyan Kulasekaran (Mathematical algorithms)* * Advisors: Joseph Petruccelli (MA) * * Michael Ciaraldi (CS) * * * * Purpose : This program performs a classical decomposition * * on a provided set of data (provided file). * * It will calculate components of the time series * * data provided and output them to files. * * Please refer to the "Using the Program" * * provided with the project report for specific file* * formats, data outputted, and questions on how to * * use this program. * * * * * * * *************************************************************** */

#include <math.h>#include <iostream.h>#include <fstream.h>#include <string>

// Applies a mapping scheme to all elements of a provided arraytemplate<class T>void propagate(T q,T k, int len,T* s){

for(int i=0; i < len; i++) {

s[i] = s[i]*q+k; }}

// Calculates and returns (by reference) the min and max of a provided arraytemplate<class T>void min_max(T* s, int len,T& min,T& max){

min = s[0]; max = s[0];

for(int i=1; i<len; i++) {

if (s[i] < min) min = s[i]; else if(s[i] > max) max = s[i]; }}

// Scheme to map elements of an array to MAX panning scaletemplate<class T>void mapper_pan(T *s, int len){

double min_s, max_s;

min_max(s,len,min_s,max_s);

double p = max_s - min_s; double q = 127/p;

double z = min_s * q; double k = 0 - z;

propagate(q,k,len,s);}

// Scheme to map a single array to MAX piano MIDI scaletemplate<class T>void mapper(T *s, int len){

double min_s, max_s;

min_max(s,len,min_s,max_s);

double p = max_s - min_s; double q = 87/p;

double z = min_s * q; double k = 21 - z;

propagate(q,k,len,s);}

// Scheme to map two arrays to MAX piano MIDI scale relative to each othertemplate<class T>void mapper(T *s1, T *s2, int len){

double min_s, max_s; double *vals;

double *mid;

vals = new double[4]; mid = new double[2];

min_max(s1,len,vals[0],vals[1]); min_max(s2,len,vals[2],vals[3]);

min_max(vals,4,min_s,max_s);

double p = max_s - min_s; double q = 87/p;

double z = min_s * q; double k = 21 - z;

propagate(q,k,len,s1); propagate(q,k,len,s2);

min_max(s1,len,vals[0],vals[1]); min_max(s2,len,vals[2],vals[3]);

mid[0] = (vals[1] - vals[0])/2.0+vals[0]; mid[1] = (vals[3] - vals[2])/2.0+vals[2];

if((vals[1]-vals[0]) > (vals[3]-vals[2])) propagate(1.0,mid[0]-mid[1],len,s2); else propagate(1.0,mid[1]-mid[0],len,s1);

}

// Scheme to map four arrays to MAX piano MIDI scale relative to each othertemplate<class T>void mapper(T *s1, T *s2, T *s3, T *s4, T* s5, int len){

double min_s, max_s; double *vals;

vals = new double[10];

min_max(s1,len,vals[0],vals[1]); min_max(s2,len,vals[2],vals[3]); min_max(s3,len,vals[4],vals[5]); min_max(s4,len,vals[6],vals[7]); min_max(s5,len,vals[8],vals[9]);

min_max(vals,10,min_s,max_s);

double p = max_s - min_s; double q = 87/p;

double z = min_s * q; double k = 21 - z;

propagate(q,k,len,s1); propagate(q,k,len,s2); propagate(q,k,len,s3); propagate(q,k,len,s4); propagate(q,k,len,s5);}

// main decomposition functionvoid ClassDecomp(){

// *** MODE VARS ***

int regression_mode; // 0=linear reg, 1=quadratic reg

// *** STORAGE VARS ***

string iname; // input filename string oname; // output filename int s; // range of moving averages int r; // split range over a pivot int inp_num; // number of pairs in time series double *inp_t; // store t-values for series double *dtrend; // store values of detrended series double *inp_Y; // store Y values for series double *out_T; // store function values for trend double *out_T_mapped; // store singly mapped trend values double *out_T_map_rel; // store relatively mapped trend double *out_S; // store function values for season double *out_S_mapped; // store singly mapped seas values double *out_S_map_rel; // store relatively mapped seas double *out_DS; // store Deseasonal values double *out_DS_mapped; // store singly mapped deseas double *out_DS_map_rel; // store relatively mapped deseas double *out_DT; // store values for detrended double *out_DT_mapped; // store singly mapped detrend double *out_DT_map_rel; // store relatively mapped detrend double *out_E; // store function values for error double *out_E_mapped; // store singly mapped error values double *out_E_map_rel; // store relatively mapped error double *temp_sig; // avg of dtrend vals in ith pos of s double avg_sig; // average value of temp_sig for seas double *sig_tild; // ~sigma for seasonal calculation double avg_DS; // average of *out_DS double *DS_var; // variances of *out_DS (val - avg) double *DS_var_sq; // values of *DS_var squared double avg_time; // average of values in *inp_t double *var_t; // variances of *inp_t (val - avg) double *var_t_sq; // values of *var_t squared double *DSxT; // *out_DS multiplied by *var_t double t_sq; // sum of values in *inp_t squared double slope; // slope of linear regression line double sum_var_t; // sum of values in *var_t_sq double intercept; // intercept of lin regression line double sig_t; // sum of values in *inp_t double t_cube; // sum of values in *inp_t cubed double t_quad; // sum of values in *inp_t ^4 double TxY; // sum of *inp_t * inp_y double T_SQxY; // sum of *inp_t squared * *inp_y double t_last; // store the las time value in data double sig_y; // sum of values in *inp_y double B0, B1, B2; // constants for quad regression

/* Below are the storage arrays for pairs mapped relatve to each other. Any two arrays ending in the same number are a pair. The same trend abbreviations are used as above (T=trend, S=seasonal, E=error, DT=detrended, DS=deseasonalized) */

double *out_T_rel_0; double *out_S_rel_0; double *out_T_rel_1; double *out_E_rel_1; double *out_T_rel_2; double *out_DT_rel_2; double *out_T_rel_3; double *out_DS_rel_3;

double *out_S_rel_4; double *out_E_rel_4; double *out_S_rel_5; double *out_DT_rel_5; double *out_S_rel_6; double *out_DS_rel_6; double *out_E_rel_7; double *out_DT_rel_7; double *out_E_rel_8; double *out_DS_rel_8; double *out_DT_rel_9; double *out_DS_rel_9;

// Data mapped for MAX panning range. Same abbreviations as above. double *out_T_pan; double *out_S_pan; double *out_E_pan; double *out_DS_pan; double *out_DT_pan;

// *** FILE STREAMS ***

ifstream inp; // input data file ofstream outp_data_max; // orig data in MAX-readable form ofstream outp_T; // output Trend data file ofstream outp_T_mapped; // Trend mapped to its own range ofstream outp_T_map_rel; // Trend mapped rel. to all comps ofstream outp_S; // output Seasonality data file ofstream outp_S_mapped; // Seasonal mapped to its own range ofstream outp_S_map_rel; // Seasonal mapped to all comps ofstream outp_DS; // output Deseasonal data file ofstream outp_DS_mapped; // Deseasonal mapped its own range ofstream outp_DS_map_rel; // Deseasonal mapped to all comps ofstream outp_DT; // output Detrended data file ofstream outp_DT_mapped; // Detrended mapped its own range ofstream outp_DT_map_rel; // Detrended mapped to all comps ofstream outp_E; // output Error data file ofstream outp_E_mapped; // Error mapped its own range ofstream outp_E_map_rel; // Error mapped to all comps

/* Below are the output streams for pairs mapped relatve to each other. Any two streams ending in the same number are a pair. The same trend abbreviations are used as above (T=trend, S=seasonal, E=error, DT=detrended, DS=deseasonalized) */ ofstream outp_T_rel_0; ofstream outp_S_rel_0; ofstream outp_T_rel_1; ofstream outp_E_rel_1; ofstream outp_T_rel_2; ofstream outp_DT_rel_2; ofstream outp_T_rel_3; ofstream outp_DS_rel_3; ofstream outp_S_rel_4; ofstream outp_E_rel_4; ofstream outp_S_rel_5; ofstream outp_DT_rel_5; ofstream outp_S_rel_6;

ofstream outp_DS_rel_6; ofstream outp_E_rel_7; ofstream outp_DT_rel_7; ofstream outp_E_rel_8; ofstream outp_DS_rel_8; ofstream outp_DT_rel_9; ofstream outp_DS_rel_9;

// Streams mapped for MAX panning range. Same abbreviations as above. ofstream outp_T_pan; ofstream outp_S_pan; ofstream outp_E_pan; ofstream outp_DS_pan; ofstream outp_DT_pan;

// *** GET INPUT FILE ***

cout << "Please enter input file: "; // prompt for input filename cin >> iname; // get filename inp.open(iname.c_str(),ios::in); // setup file for input if(inp.bad()) // abort if file open failed {

cerr << "File Does Not Exist\n\n"; // error if open failed exit(0); // abort execution on error }

// *** DIMENSION DATA ARRAYS ***

inp >> inp_num; // get number of pairs (amount // (to dimension all arrays by) inp_t = new double[inp_num]; inp_Y = new double[inp_num]; dtrend = new double[inp_num]; out_T = new double[inp_num]; out_T_mapped = new double[inp_num]; out_T_map_rel = new double[inp_num]; out_S = new double[inp_num]; out_S_mapped = new double[inp_num]; out_S_map_rel = new double[inp_num]; out_DS = new double[inp_num]; out_DS_mapped = new double[inp_num]; out_DS_map_rel = new double[inp_num]; out_DT = new double[inp_num]; out_DT_mapped = new double[inp_num]; out_DT_map_rel = new double[inp_num]; out_E = new double[inp_num]; out_E_mapped = new double[inp_num]; out_E_map_rel = new double[inp_num]; temp_sig = new double[inp_num]; sig_tild = new double[inp_num]; DS_var = new double[inp_num]; DS_var_sq= new double[inp_num]; var_t = new double[inp_num]; var_t_sq = new double[inp_num]; DSxT = new double[inp_num];

out_T_rel_0 = new double[inp_num]; out_S_rel_0 = new double[inp_num];

out_T_rel_1 = new double[inp_num]; out_E_rel_1 = new double[inp_num]; out_T_rel_2 = new double[inp_num]; out_DT_rel_2 = new double[inp_num]; out_T_rel_3 = new double[inp_num]; out_DS_rel_3 = new double[inp_num]; out_S_rel_4 = new double[inp_num]; out_E_rel_4 = new double[inp_num]; out_S_rel_5 = new double[inp_num]; out_DT_rel_5 = new double[inp_num]; out_S_rel_6 = new double[inp_num]; out_DS_rel_6 = new double[inp_num]; out_E_rel_7 = new double[inp_num]; out_DT_rel_7 = new double[inp_num]; out_E_rel_8 = new double[inp_num]; out_DS_rel_8 = new double[inp_num]; out_DT_rel_9 = new double[inp_num]; out_DS_rel_9 = new double[inp_num];

out_T_pan = new double[inp_num]; out_S_pan = new double[inp_num]; out_E_pan = new double[inp_num]; out_DS_pan = new double[inp_num]; out_DT_pan = new double[inp_num];

for (int i=0; i<inp_num; i++){ // populate input arrays

inp >> inp_t[i]; inp >> inp_Y[i]; }

inp.close(); // close input file

cout << endl << endl << "Please enter s to use: "; cin >> s; // get range for moving averages

cout << endl << endl << "1 for linear regression" << endl << "2 for quadratic regression" << endl << endl << "Please enter the regression mode to use: "; cin >> regression_mode;

if(s%2==0){ // moving average for an even s

r = s/2; // calculate r

for(int i=r; i<inp_num-r; i++){

out_T[i]=0.5*inp_Y[i-r]+0.5*inp_Y[i+r];

for(int j=i-r+1; j<=i+r-1; j++){ // take sum of values over range

out_T[i] = out_T[i] + inp_Y[j]; } out_T[i] = out_T[i]/s; // average the sum }

}

else{ // moving average for odd s

r=(s-1)/2; // calculate r

for(int i=r; i<inp_num-r; i++){ // calculate averages

out_T[i]=0.0; // initialize trend value for average

for(int j=i-r; j<=i+r; j++){ // take sum of values over range out_T[i]= out_T[i] + inp_Y[j]; }

out_T[i]=out_T[i]/s; // average the sum } }

// calculate detrended series for(int i=r; i<inp_num-r; i++) dtrend[i] = inp_Y[i] - out_T[i];

// calculate intermediate vals for seasonal for(int i=0; i<s; i++){

temp_sig[i] = 0.0;

/* avg the detrended vals of each row of the series with vals divided into groups of s */

for(int j=0; j<=(inp_num-2*r)/s; j++){ temp_sig[i] = temp_sig[i] + dtrend[r+s*j+i]; } temp_sig[i] = temp_sig[i]/((inp_num-r)/s); }

// average the rows just calculated avg_sig = 0.0; for(int i=0; i<s; i++) avg_sig = avg_sig + temp_sig[i]; avg_sig = avg_sig/s;

// subtract the avg value from each sig for(int i=0; i<s; i++) sig_tild[i] = temp_sig[i] - avg_sig;

// calculate the seasonal values for(int i=0; i<inp_num; i++) out_S[i] = sig_tild[(i+r)%s];

avg_DS=0.0; // calculate desesonal values and avg them for(int i=0; i<inp_num; i++){ out_DS[i] = inp_Y[i] - out_S[i]; avg_DS = avg_DS + out_DS[i]; } avg_DS = avg_DS / inp_num;

avg_time = 0.0;

/* calculate the deseasonal variances variances squared and avg time */

for(int i=0; i<inp_num; i++){ DS_var[i] = out_DS[i] - avg_DS; DS_var_sq[i] = pow(DS_var[i],2); avg_time = avg_time + inp_t[i]; } avg_time = avg_time/inp_num;

slope = 0.0; sum_var_t = 0.0; sig_t = 0.0; sig_y = 0.0; t_sq = 0.0; t_cube = 0.0; t_quad = 0.0; TxY = 0.0; T_SQxY = 0.0; t_last = inp_t[inp_num-1];

/* calculate values necessary for regression - sum of time - sum of series values - time variance and time variance squared - deseasonal variance mult. by time variance - time squared, cubed, and ^4 - time mult. by series values - time squared mult. by series values */

for(int i=0; i<inp_num; i++){

sig_t = sig_t + inp_t[i]; sig_y = sig_y + inp_Y[i]; var_t[i] = inp_t[i] - avg_time; var_t_sq[i] = pow(var_t[i],2); sum_var_t = sum_var_t + var_t_sq[i]; DSxT[i] = DS_var[i] * var_t[i]; slope = slope + DSxT[i]; t_sq = t_sq + pow(inp_t[i],2); t_cube = t_cube + pow(inp_t[i],3); t_quad = t_quad + pow(inp_t[i],4); TxY = TxY + inp_t[i] * inp_Y[i]; T_SQxY = T_SQxY + pow(inp_t[i],2) * inp_Y[i]; }

// calculate constants for linear regression slope = slope / sum_var_t; intercept = avg_DS - slope*avg_time;

// calculate constants necessary for quadratic regression // (the braces allow the unnecessary intermediates to // scope-out after calculations)

{ double p = (TxY * t_last) - (sig_y * sig_t); double s = (t_cube * t_last) - (sig_t * t_sq); double q = (T_SQxY * t_last) - (sig_y * t_sq); double r = (t_sq * t_last) - (sig_t * sig_t);

double t = (t_cube * t_last) - (t_sq * sig_t); double u = (t_quad * t_last) - (t_sq * t_sq); double a = (sig_y * sig_t); double b = (t_last * sig_t); double c = (sig_t * sig_t); double d = (t_sq * sig_t);

B2 = ((p*s)-(q*r)) / ((t*s)-(u*r)); B1 = (p - (t*B2)) / r; B0 = ((a) - (B1 * c) - (B2 * d)) / b; }

switch(regression_mode){

// linear regression case 1: for(int i=0; i<inp_num; i++){

out_T[i] = intercept + slope*inp_t[i]; out_E[i] = inp_Y[i] - out_T[i] - out_S[i]; } break;

//quadratic regression case 2: for(int i=0; i<inp_num; i++){

out_T[i] = B0 + (B1 * inp_t[i]) + (B2 * pow(inp_t[i],2)); out_E[i] = inp_Y[i] - out_T[i] - out_S[i]; } break;

// invalid input, instruct the user and abort default: cout << endl <<"Invalid regression mode: " << endl << "1 for linear regression" << endl << "2 for quadraic regression" << endl << endl; exit(0); }

/* copy the results into the arrays to be mapped it's obvious that it would be rather inefficient to run the algorithm again for each of them */

for(int i=0; i<inp_num; i++){

out_T_mapped[i] = out_T[i]; out_T_map_rel[i] = out_T[i]; out_S_mapped[i] = out_S[i]; out_S_map_rel[i] = out_S[i]; out_DS_mapped[i] = out_DS[i]; out_DS_map_rel[i]= out_DS[i]; out_DT[i] = inp_Y[i] - out_T[i]; out_DT_mapped[i] = out_DT[i]; out_DT_map_rel[i] = out_DT[i]; out_E_mapped[i] = out_E[i]; out_E_map_rel[i] = out_E[i];

out_T_rel_0[i] = out_T[i]; out_S_rel_0[i] = out_S[i]; out_T_rel_1[i] = out_T[i]; out_E_rel_1[i] = out_E[i]; out_T_rel_2[i] = out_T[i]; out_DT_rel_2[i] = out_DT[i]; out_T_rel_3[i] = out_T[i]; out_DS_rel_3[i] = out_DS[i]; out_S_rel_4[i] = out_S[i]; out_E_rel_4[i] = out_E[i]; out_S_rel_5[i] = out_S[i]; out_DT_rel_5[i] = out_DT[i]; out_S_rel_6[i] = out_S[i]; out_DS_rel_6[i] = out_DS[i]; out_E_rel_7[i] = out_E[i]; out_DT_rel_7[i] = out_DT[i]; out_E_rel_8[i] = out_E[i]; out_DS_rel_8[i] = out_DS[i]; out_DT_rel_9[i] = out_DT[i]; out_DS_rel_9[i] = out_DS[i];

out_T_pan[i] = out_T[i]; out_S_pan[i] = out_S[i]; out_E_pan[i] = out_E[i]; out_DS_pan[i] = out_DS[i]; out_DT_pan[i] = out_DT[i]; }

/* Perform all the mappings necessary for the sonification */

// map all the individual components mapper(inp_Y, inp_num); mapper(out_T_mapped, inp_num); mapper(out_S_mapped, inp_num); mapper(out_DS_mapped, inp_num); mapper(out_DT_mapped,inp_num); mapper(out_E_mapped, inp_num);

// map all the pairs of components mapper(out_T_rel_0,out_S_rel_0,inp_num); mapper(out_T_rel_1,out_E_rel_1,inp_num); mapper(out_T_rel_2,out_DT_rel_2,inp_num); mapper(out_T_rel_3,out_DS_rel_3,inp_num); mapper(out_S_rel_4,out_E_rel_4,inp_num); mapper(out_S_rel_5,out_DT_rel_5,inp_num); mapper(out_S_rel_6,out_DS_rel_6,inp_num); mapper(out_E_rel_7,out_DT_rel_7,inp_num); mapper(out_E_rel_8,out_DS_rel_8,inp_num); mapper(out_DT_rel_9,out_DS_rel_9,inp_num);

// map all the components relative to each other mapper(out_T_map_rel, out_S_map_rel, out_DS_map_rel,out_DT_map_rel,out_E_map_rel,inp_num);

// map all the components to the panning range mapper_pan(out_T_pan,inp_num); mapper_pan(out_S_pan,inp_num); mapper_pan(out_E_pan,inp_num); mapper_pan(out_DS_pan,inp_num); mapper_pan(out_DT_pan,inp_num);

// get position of '.' in filename int pos = iname.find('.');

/* Each 2-line group of code does the same two steps 1. Set the name for the respective output file 2. Open file and assign to repsective stream */

iname.replace(pos,iname.length()-pos+1,"_max.dat"); outp_data_max.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos,iname.length()-pos+1,".trn"); outp_T.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos,iname.length()-pos+1, ".ssea"); outp_S.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1,".err"); outp_E.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, ".dsea"); outp_DS.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, ".dtrn"); outp_DT.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos,iname.length()-pos+1,"_max.trn"); outp_T_mapped.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos,iname.length()-pos+1, "_max.ssea"); outp_S_mapped.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1,"_max.err"); outp_E_mapped.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_max.dsea"); outp_DS_mapped.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_max.dtrn"); outp_DT_mapped.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos,iname.length()-pos+1,"_rel.trn"); outp_T_map_rel.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos,iname.length()-pos+1, "_rel.ssea"); outp_S_map_rel.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1,"_rel.err"); outp_E_map_rel.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel.dsea"); outp_DS_map_rel.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel.dtrn"); outp_DT_map_rel.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, "_rel_0.trn"); outp_T_rel_0.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, "_rel_0.ssea"); outp_S_rel_0.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_1.trn"); outp_T_rel_1.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_1.err"); outp_E_rel_1.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_2.trn"); outp_T_rel_2.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_2.dtrn"); outp_DT_rel_2.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_3.trn"); outp_T_rel_3.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_3.dsea"); outp_DS_rel_3.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_4.ssea"); outp_S_rel_4.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, "_rel_4.err"); outp_E_rel_4.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_5.ssea"); outp_S_rel_5.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_5.dtrn"); outp_DT_rel_5.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_6.ssea"); outp_S_rel_6.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_6.dsea"); outp_DS_rel_6.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_7.err"); outp_E_rel_7.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_7.dtrn"); outp_DT_rel_7.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_8.err"); outp_E_rel_8.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_8.dsea"); outp_DS_rel_8.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_9.dtrn"); outp_DT_rel_9.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_rel_9.dsea"); outp_DS_rel_9.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, "_pan.trn"); outp_T_pan.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_pan.ssea"); outp_S_pan.open(iname.c_str(), ios::out | ios::trunc);

iname.replace(pos, iname.length()-pos+1, "_pan.err"); outp_E_pan.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_pan.dsea"); outp_DS_pan.open(iname.c_str(), ios::out | ios::trunc); iname.replace(pos, iname.length()-pos+1, "_pan.dtrn"); outp_DT_pan.open(iname.c_str(), ios::out | ios::trunc);

// Write first value to all files (a way to avoid an // extra end-line character at the end of the files)

outp_data_max << inp_Y[0]; outp_T << out_T[0]; outp_S << out_S[0]; outp_DS << out_DS[0]; outp_DT << out_DT[0]; outp_E << out_E[0]; outp_T_mapped << out_T_mapped[0]; outp_S_mapped << out_S_mapped[0]; outp_DS_mapped << out_DS_mapped[0]; outp_DT_mapped << out_DT_mapped[0]; outp_E_mapped << out_E_mapped[0]; outp_T_map_rel << out_T_map_rel[0]; outp_S_map_rel << out_S_map_rel[0]; outp_DS_map_rel << out_DS_map_rel[0]; outp_DT_map_rel << out_DT_map_rel[0]; outp_E_map_rel << out_E_map_rel[0];

outp_T_rel_0 << out_T_rel_0[0]; outp_S_rel_0 << out_S_rel_0[0]; outp_T_rel_1 << out_T_rel_1[0]; outp_E_rel_1 << out_E_rel_1[0]; outp_T_rel_2 << out_T_rel_2[0]; outp_DT_rel_2 << out_DT_rel_2[0]; outp_T_rel_3 << out_T_rel_3[0]; outp_DS_rel_3 << out_DS_rel_3[0]; outp_S_rel_4 << out_S_rel_4[0]; outp_E_rel_4 << out_E_rel_4[0]; outp_S_rel_5 << out_S_rel_5[0]; outp_DT_rel_5 << out_DT_rel_5[0]; outp_S_rel_6 << out_S_rel_6[0]; outp_DS_rel_6 << out_DS_rel_6[0]; outp_E_rel_7 << out_E_rel_7[0]; outp_DT_rel_7 << out_DT_rel_7[0]; outp_E_rel_8 << out_E_rel_8[0]; outp_DS_rel_8 << out_DS_rel_8[0]; outp_DT_rel_9 << out_DT_rel_9[0]; outp_DS_rel_9 << out_DS_rel_9[0];

outp_T_pan << out_T_pan[0]; outp_S_pan << out_S_pan[0]; outp_DS_pan << out_DS_pan[0]; outp_DT_pan << out_DT_pan[0]; outp_E_pan << out_E_pan[0];

// *** Write OUTPUT TO FILES ***

for (int i=1; i<inp_num; i++){

outp_data_max << '\n' << inp_Y[i]; outp_T << '\n' << out_T[i]; outp_S << '\n' << out_S[i]; outp_DS << '\n' << out_DS[i]; outp_DT << '\n' << out_DT[i]; outp_E << '\n' << out_E[i]; outp_T_mapped << '\n' << out_T_mapped[i]; outp_S_mapped << '\n' << out_S_mapped[i]; outp_DS_mapped << '\n' << out_DS_mapped[i]; outp_DT_mapped << '\n' << out_DT_mapped[i]; outp_E_mapped << '\n' << out_E_mapped[i]; outp_T_map_rel << '\n' << out_T_map_rel[i]; outp_S_map_rel << '\n' << out_S_map_rel[i]; outp_DS_map_rel << '\n' << out_DS_map_rel[i]; outp_DT_map_rel << '\n' << out_DT_map_rel[i]; outp_E_map_rel << '\n' << out_E_map_rel[i];

outp_T_rel_0 << '\n' << out_T_rel_0[i]; outp_S_rel_0 << '\n' << out_S_rel_0[i]; outp_T_rel_1 << '\n' << out_T_rel_1[i]; outp_E_rel_1 << '\n' << out_E_rel_1[i]; outp_T_rel_2 << '\n' << out_T_rel_2[i]; outp_DT_rel_2 << '\n' << out_DT_rel_2[i]; outp_T_rel_3 << '\n' << out_T_rel_3[i]; outp_DS_rel_3 << '\n' << out_DS_rel_3[i]; outp_S_rel_4 << '\n' << out_S_rel_4[i]; outp_E_rel_4 << '\n' << out_E_rel_4[i]; outp_S_rel_5 << '\n' << out_S_rel_5[i]; outp_DT_rel_5 << '\n' << out_DT_rel_5[i]; outp_S_rel_6 << '\n' << out_S_rel_6[i]; outp_DS_rel_6 << '\n' << out_DS_rel_6[i]; outp_E_rel_7 << '\n' << out_E_rel_7[i]; outp_DT_rel_7 << '\n' << out_DT_rel_7[i]; outp_E_rel_8 << '\n' << out_E_rel_8[i]; outp_DS_rel_8 << '\n' << out_DS_rel_8[i]; outp_DT_rel_9 << '\n' << out_DT_rel_9[i]; outp_DS_rel_9 << '\n' << out_DS_rel_9[i];

outp_T_pan << '\n' << out_T_pan[i]; outp_S_pan << '\n' << out_S_pan[i]; outp_DS_pan << '\n' << out_DS_pan[i]; outp_DT_pan << '\n' << out_DT_pan[i]; outp_E_pan << '\n' << out_E_pan[i];

}

//*** CLOSE OUPUT FILES ***

outp_T.close(); outp_S.close(); outp_DS.close(); outp_DT.close(); outp_E.close(); outp_T_mapped.close(); outp_S_mapped.close(); outp_DS_mapped.close(); outp_DT_mapped.close(); outp_E_mapped.close(); outp_T_map_rel.close();

outp_S_map_rel.close(); outp_DS_map_rel.close(); outp_DT_map_rel.close(); outp_E_map_rel.close();

outp_T_rel_0.close(); outp_S_rel_0.close(); outp_T_rel_1.close(); outp_E_rel_1.close(); outp_T_rel_2.close(); outp_DT_rel_2.close(); outp_T_rel_3.close(); outp_DS_rel_3.close(); outp_S_rel_4.close(); outp_E_rel_4.close(); outp_S_rel_5.close(); outp_DT_rel_5.close(); outp_S_rel_6.close(); outp_DS_rel_6.close(); outp_E_rel_7.close(); outp_DT_rel_7.close(); outp_E_rel_8.close(); outp_DS_rel_8.close(); outp_DT_rel_9.close(); outp_DS_rel_9.close();

outp_T_pan.close(); outp_S_pan.close(); outp_DS_pan.close(); outp_DT_pan.close(); outp_E_pan.close();}

void main(){

ClassDecomp(); // call classical decomp algorithm}

7.3 Sonification Program Patch Schematics (MAX Source)

The following subsections contain several MAX Patch Schematics necessary for our Hearing

Data Sonification Program. All patches were designed and coded by Nedunceliyan Kulasekaran,

with the exception of the Floating Point Parser, which was principally authored by Gary Barber

Jr. Each patch is also available as MAX binary files in the Report’s Companion CD.

If there are any questions pertaining the design and application of these patches, feel free to e-

mail cel@wpi.edu or mysticgb@wpi.edu for further information.

7.3.1 Floating Point Parser (fl_parser)

7.3.2 Core Hearing Data Patch

7.3.3 Core II Hearing Data Patch (left end part)

7.3.4 Core II Hearing Data Patch (right end part)

7.3.5 Panning Relationship Patch for Hearing Data

7.3.6 Hearing Data Patch (top part)

7.3.7 Hearing Data Patch (bottom part)

7.3.8 Hearing Data User Interface

7.4 Controlled Experiment – Introductory Handouts for Subjects

MQP: HEARING DATA (Introduction)

BACKGROUND1. Sonification:

a process of mapping data (in this experiment, time-series) into audio.

2. Time-series:

A time-series is data that are taken over time. A time-series plot/graph is most

often visually displayed by connecting successive observations with lines.

3. The Classical Decomposition of a Time-Series:

A time-series (Yt) may be decomposed into three components: its Trend (Tt),

Seasonal (St) and Irregular Components (Et) such that:

See given example illustration.

4. Panned relationship:

Panning is a technique of controlling the balance of sound around the listener,

i.e. how much of the sound (volume) is coming from each speaker. In this study,

when a component A is “panned by” another component B, this means that

component A is played over both speakers, but the balance is controlled by

component B.

OBJECTIVE

Sonification is the representation of data in an auditory format. This project is concerned with

the analysis of time-series using sonification. Traditionally, the analysis of time-series has been a

task that is primarily done via graphical techniques. We are interested in determining if auditory

formats have the capacity to convey an adequate representation of the data to be used for the

analysis.

WHAT WE WOULD LIKE YOU TO DO

1. Sonification Method 1 (Listening to one component at a time)

a. You will be presented with two sets of audio files, Set P.1 and Set Q.1. There are

3 files in each set: one for the trend, one for the seasonal, and one for the irregular

component.

b. You will be given a printed handout containing six time-series graphs.

c. Listen to each set of audio files and look at the graph handout.

d. Identify the graphs of the time-series you think correspond to sonification P.1 and

Q.1.

2. Sonification Method 2 (Listening to two components at a time)

a. You will be presented with two sets of audio files, Set P.2 and Set Q.2. There are

3 files in each set: one for trend & seasonal, one for trend & irregular, and one for

seasonal & irregular. You will hear the pairs from (Left speaker) & (Right

speaker)

b. You will be given a printed handout containing six time-series graphs.

c. Listen to each set of audio files and look at the graph handout.

d. Identify the graphs of the time-series you think correspond to sonification P.2 and

Q.2.

3. Sonification Method 3 (Listening to panned relationship between two components)

a. You will be presented with two sets of audio files, Set P.3 and Set Q.3. There are

4 files in each set: one for “trend panned by seasonal”, one for “trend panned by

irregular”, one for “seasonal panned by trend”, and one for “irregular panned by

trend”. Please see the definition of panned relationship if confused.

b. You will be given a printed handout containing six time-series graphs.

c. Listen to each set of audio files and look at the graph handout.

d. Identify the graphs of the time-series you think correspond to sonification P.3 and

Q.3.

4. Fill out the Questionnaire form.

Please feel free to ask the project team any questions you may have at any time.

7.5 Controlled Experiment – Airline Data Example Handout

Trend Component

050

100150200250300350400450500

Jan-49

Jan-50

Jan-51

Jan-52

Jan-53

Jan-54

Jan-55

Jan-56

Jan-57

Jan-58

Jan-59

Jan-60

time (monthly)

Tren

d va

lues

7.6 Controlled Experiment – Graphs Handout

7.7 Controlled Experiment – Questionnaire Form

Major: Class Year:

Do you have any background in statistics? (List relevant courses if applicable)

Have you ever worked with time-series?

QuestionnairePlease answer the following questions after completing the experiment by circling one on the

scale (where SD = Strongly Disagree; D = Disagree; A = Agree; SA = Strongly Agree)

1. It was easy to identify the appropriate graphs from sonification method 1.

SD D A SA

2. It was easy to identify the appropriate graphs from sonification method 2.

SD D A SA

3. It was easy to identify the appropriate graphs from sonification method 3.

SD D A SA

4. It will be easier to identify the appropriate graphs if utilizing two sonification methods.

SD D A SA

If you agree, which two?

If you disagree, why not?

5. If any one sonification method seemed most effective for identifying the data, please list

it and explain why you think it’s effective

6. Do you have any comments or suggestions?