Introduction to Interactive Media 10: Audio in Interactive Digital Media.

Introduction to Interactive Media

10: Audio in Interactive Digital Media


What is Sound?

• All sounds are produced by the conversion of energy into vibrations.

• Vibration becomes a wave. When it reaches the ear, it causes the eardrum to vibrate at the same frequency


Audible Spectrum

• Humans can hear sounds in range of 20 Hz – 20 kHz.

• Highest note on piano is approx. 4kHz.


Sound Over Time

• Most sounds change frequency over time.

• Represented in waveform.


Music vs. Speech

Music waveform

Speech waveform

Waveform conveys gross character and dynamics of sound. Can provide cues in syncing media.


The Nature of Sound

• Wave captures three features of sound:– Amplitude

• Perceived as volume.

– Frequency• Perceived as pitch.

– Duration• Length of time sound lasts.


Getting Sound in and Out of the Computer

• Analog to Digital Converter captures separate measures of sound amplitude.– Samples are recorded as

digital numbers.

• Digital values are used to recreate the analog form using a Digital to Analog Converter.


Recording Sound

• Best to not do directly into computer as noise from within the computer is picked up.

• Sample rate can be reduced when recording speaking.


Recording Sound

• When capturing sound, adjust levels.– If amplitude is too low, sound quality is reduced; – If amplitude is too high, clipping occurs and

produces distortion


Sampling Sound

• Quality of the sampling depends on:– Sample resolution– Sample rate.


Sample Resolution

• Number of bits to encode amplitude. • Like images, more bits used to describe

information – more accurately it will be represented.

• Two common sample resolutions are 8-bit and 16-bit.– 8-bit resolution captures 256 different amplitude

levels.– 16-bit sound has 65,000 different levels.

• CD quality sound.

• Inadequate sample resolution can distort the sound.


Sample Rate• Number of samples taken in a fixed interval of time.

– Stated in thousands of Hertz, or kilohertz.– CD-quality sound captures 44.1kHz

to record frequencies as high as 22.05kHz. (The highest frequency the human ear can detect is 20kHz.)

• Two measurements capture each cycle of the sound wave:

• High value or peak• Low value or trough.


Sound File Formats

• Common sampled sound file formats: – WAV– AIFF– AU

• MP3 is most popular compressed format (lossy)– 10:1 file size ratio – WAV:MP3


Manipulating Sound

• Common applications that allow for recording and generate effects.– Apple’s GarageBand– Adobe’s Soundbooth– Sony’s Sound Forge


Common Audio Edits

• Hiss Removal– sample the hiss and remove hiss based on

what you sampled.

• Normalization – brings the average or peak amplitude to a

target level

• Time stretching

• Pitch alteration


Use of Audio in Interactive Media

• Ambience

• Sound Effects

• Auditory Feedback to aid Usability

• Music

• Speech


Sound on the Web

• Not always needed (can be annoying) – Became popular on the Web when it

became possible (late 1990s – early 2000s)

– Fell out fashion on the Web– Web has become more cinematic


How to Integrate Sound

• Authoring applications allow for precise syncing of audio within interactive experience.


How to Integrate Sound

• HTML5– Prior to HTML 5, there was not a standard

for playing audio files on a web.– HTML 5 defines a new element which

specifies a standard way to embed an audio file on a web page: the <audio> element.

• For more information re: format and browser support: http://www.w3schools.com/html/html5_audio.asp


Beyond Sampled Sound

• Sampled sound– All audio discussed so far: WAV, MP3,

AIFF, AU, etc…– Sound is stored as a description of the

sound.


Synthesized Sound

• Sound is stored as a series of commands for the computer to reproduce the sounds.

• Analogous to vector-based graphics.• MIDI (Musical Instrument Digital Interface).

– Codes provided for:• Specific instruments• Notes• Force and duration of note• Routing commands to different instrument channels• Specialized control functions.


MIDI

• Simplest system contains:– Digital musical instrument to create messages– Sound synthesizer to interpret the messages – Amplifier/speaker output system.


MIDI Videos

• Composing – http://www.youtube.com/watch?v=pjPf24XQo1s

• Building Your MIDI Studio– http://www.youtube.com/watch?v=JJLgwaIFePk&feature=fvw

Introduction to Interactive Media 10: Audio in Interactive Digital Media.

Documents

Transcript of Introduction to Interactive Media 10: Audio in Interactive Digital Media.