Audio TechFull Training

CryEngine3Tomas Neumann

(constantly WIP)

FarCry Facts

• Shipped Sounds.pak 240MB– 1,400 individual wave files (reduced Wav)– 35 subfolders

• Shipped Music.pak 465MB– Low/high spec version (22k wav, OGG)– 2h 35 minutes

• Shipped English.pak 425MB– 3,700 lines of voice + lipsync files

• Total – 1.1 Gig of audio data for Audio of 3Gig– 35% of Gold-ISO were audio data

Crysis Facts

• Shipped Sounds.pak 656MB– 3,500 events in 170 groups in 9 projects (factor 20)– 15,000 individual wave files– 26 categories

• Shipped Music.pak 570MB– 3 Gig of adaptively-scored music data– 5h 10 minutes

• Shipped English.pak 190MB– 8,000 lines of voice + lipsync files

• Total – 18% of Gold-ISO were audio data– 20 Gig of data on Audio Build PC for Game02– 14 Languages, 250,000 voice files in overall process

Warhead Facts

• Shipped Sounds.pak 900MB– x events in x groups in x projects (factor x)– x individual wave files– 26 categories

• Shipped Music.pak 190MB– 1h 45 minutes

• Shipped English.pak 181MB– 5,000 lines of voice + lipsync files

• Total – 1,2 Gig of Audio of 5.5 Gig– 22% of Gold-ISO were audio data

Overview Audio• Level placed

– Volume-based (Area)– Spots– FlowGraph– DialogManager– Trackview

• Systematic– Weapons– Vehicles– Physics– Particles– Flash

• Animation– Blended / Layered– Foley– Footstep signals

• Asset Production– Data-driven events– FMOD Designer

• .FDP (xml project)• .FEV (bin. project)• .FSB (wavebank)

• Dialog– LocalizationManager– Sub-Titles

• Runtime Effects– SoundMoods– Reverb– Dialog effects

FMOD EventsystemIntegration

Eventsystem Overview

• Data-driven– Distance, individual falloff curve, mixing– Looping, 3D, 2D, variations, priority

• Syntax – Path/project:group(/group):eventSounds/weapons:scar:select

• Code controlled– Start, stop, pause– Parameters: rpm, mass, speed, damage

Asset Production

Workflow

• Sound Designer– Create event (wave data + definition + behavior)– Save and checkin project file (.fdp)

• Audio Build PC– Build process, compress data per platform– Creates bin. project and wavebanks (.fev/.fsb)

• Game Programmer– Play sound when availible– Update parameter, communicate to directly to SD

• Sound Designer (or GameD, ParticleD, AnimationD)– Pick event from SoundBrowser– Implement in game (FlowGraph, TrackView, Particle, Facial..)

Workflow Example• Create project

– Project name has to be the same than folder it is placed in• Create group

– As many levels of group depths are supported• Create event

– „Holes“ in placing sound definition might cause unexpected problems– Add layer, assign to parameter– Add parameter controlled effect

• Create category– Sort event into category

• Save you work– Save and checkin project file (.fdp)

• Editor uses .fdp for sound browser functionality

Build Process

• Build process creates .fev and .fsb– Local / remote (dedicated audio build PC)– Create wavebank– Wavebank settings per platform– Creates bin. project and wavebanks (.fev/.fsb)

• Code uses .fev for accessing event structure• Restart audio while in Game/Editor

– #Sound.DeactivateAudioDevice()– #Sound.ActivateAudioDevice()– Mfx_reload tab– Reload... dialog

Dynamic Event Behavior

• Individual Event Parameter– Distance-based aspects of sound („distance“)– Interactive:

• Vehicles: rpm, speed, surface, damage• Weapons: spin-up• Particles: lifetime• Physics: mass, speed• More: „doppler“, „front_left_rear“, „top_rear_down“, „top_left_down“

– Battle-sensitive animal ambiences („battle“)– Area-sensitive via reverb area („environment“)– Dynamic background noise on radio chats

• Global Parameter– TimeOfDay-sensitive ambiences („daylight“)– SP/MP tweak („sp_coop_mp“)

Dynamic System Behavior

• CVar-based volume• SoundMoods

– Category-based mixing system

– Mixing deskish control over faders and effects

– Effects:• Volume / Pitch• Lowpass / Highpass• Compressor• 2 ParamEQs• (Any VST-Plugin)

• SoundMoods:– Mission objective– Underwater– Low-health– Explosion– Vehicle– Cloak– Frozen– Cutscene– Huge alien

Mixing

• Static Mix– Set volume on event prop, not in sound definition– Network Audition

• Enable cvar s_NetworkAudition 1 (opens port)• Connect to localhost or remote machine• Ingame mixing and profiling

• Dynamic in-game mix: Soundmoods!– Category based mixing– Music ducking

Other Audio Middleware

• FMOD Eventsystem in CE2 since mid.‘05• Audiokinetic‘s WWise• CRI Audio• Unreal engine integrate FMOD and WWise

•                       

FMOD (Firelight Technologies)

•                       

Wwise (Audiokinetic)

Audio GUI

Sound Scripts

Sound/Dialog Browser

• SoundBrowser– shows all curent events

scanning FDP files

• DialogBrowser– Shows all entries in

LocalizationManager

Database Music

Database SoundMoods

Database Reverb Presets

Dialog Editor

Cutscene Audio

• Place events from SoundBrowser– Use Sound-Track, don‘t add SoundEventSpot

• Place dialog from DialogBrowser

Audio Details

Sound Implementation

• Volumetric Effects– Ambience– Random Oneshots– Reverb– Opt. Soundmoods

• FlowGraph– PlaySound-Node– Dialog-Node– Use instances of sound entities

Script Entities• Sound

– SoundEventSpot (simple placable sound)– AmbientVolume (attachable to area)– RandomVolume (attachable to area)– SoundMood (attachable to area or flowgraph)– ReverbVolume (attachable to area)

• Dialog– Dialog (also flowgraph)

• Music– ThemeSelector (attachable to area or flowgraph)– MoodSelector (attachable to area or flowgraph)– EndTheme (attachable to area or flowgraph)– Stinger (also flowgraph)– MusicLogic (also flowgraph)– PlayPattern

Volumetric Sounds

• Point– Simple radial propagation (arbritary volume behavior)

• Line– 2 points, closest to listener

• Sphere– Radius, closest or at listener if inside

• Cone– Specified by sound designers

• Area-attached (shape, box, sphere)– Reverb, ambiences, randomsound, soundmood

Obstruction

• Effect if sound is blocked from listener– Direkt (dry) signal by ray cast– Indirekt (wet) signal by Visarea

• Controllable by CVars

• Obstruction flag set by code (might change)

• Listener spawned sounds usually skipped

• Material based by pierceability

Culling

• Effect if playback gets skipped

• Outdoor– Distance is larger than radius

• Indoor– Max. visarea propagation reached

• Priority– Event priority lower than current threshold– (FMOD counts zero as being highest)

Dialog

Dialog

• Playback– 2D– 3D on character‘s head bone position

Dialog

• Source– Languages/Dialog/– Uncompressed WAVs– Dialog_recording

Excel

• Build– Localized/English.pak– Compressed

MP2/MP3– XMA in development

via FSB

• Syntax (File/key)File:

languages/dialog/subfolder/soundfile.wav

Key: subfolder/soundfile

• Playback– SoundBuffer of wav

attached to empty prototype event

sounds/dialog:dialog:clean

• Format priority– s_MpegCompression 1/2/3

Localization Manager

• Xml sheet– Stores keys/parameter– Dialog_recording_list.xml (ai/story/mp)

Code

Overview Code

• SoundSystem– Sound

• PlatformSound

– AssetManager• Manages audio cache• Controls SoundBuffer

– MoodManager– ReverbManager

• NULL Implementation

• Sound and parameter names are lower case

• Prevent hardcoded sound names

• gEnv->pSoundSystem is always valid

Overview Code

SoundSystem

Interface

Sound

AudioDevice

FMOD-Ex (4.26.00)

EventSystem GroupProject Event

Buffer PlatformSound

Xbox360 XAudio2 PS3 XP: DSound Vista: WASAPI (OpenAL)

Parameter

Reverb SoundMoods

PSP Mac

FMOD specific code FMOD Designer (4.27.00)

Code Interface

• ISound.h for ISoundSystem and ISound– Sound will not move unless you do it

• IEntitySoundProxy for sounds on entities– Sound will move/rotate with entity– Faster access to find a sound by SoundID– Hidden entities (layer) dont play sounds

Modular Interfaces

• ISoundSystem and ISound– Core functionality– GetExtended()

• know what you are doing• Ask me if you are unsure!

– GetDeprecated()• No support• Dont count on it

– GetWorkInProgress()• Might have bugs

Ref Counting

• Sounds, SoundBuffers, Platformsounds– Release() dont destroy– Reset and stored in pool to be reused

• SoundIDs used to prevent ref-couting– No guarantee sound still exist– Test for valid SoundID– Only use scoped smart_ptr

Script Interface

• ScriptBind_Sound.h– Most functionality– Also Reverb, SoundMood, Music

• ScriptBind_Entity.h– proxy access

• ScriptBind_MusicLogic.h

Golden Rules

• Query information– Create the sound– Query: Loop, 3D/2D, Radius

• Dont hardcode:

• Sound references

• Parameter names

• Set Flags: Event, Voice

Music

Overview Music

• MusicSystem– Theme

• Mood– Pattern

• MusicLogic (optional)– AnimationGraph– MusicLogic events

• Editor– Database-Music– Animationgraph

• Sources– Music: Game/Music– Events:

Game/Libs/MusicLogic– Music graph:

Game/Animation/graphs/ (will change)

Database View

• Theme– Wider range of music

of a certain area or situation

• Mood– Special motive

• PatternBlock– Variation of motive

• Layer– Start (optional)– Main (loop)– Rythmic (synched)– Incidental (synched)– Stinger (async)– End (optional)

• Pattern– Reference to file– Always link to WAV

Music Logic

• Dynamic Music– Picks Mood– AI Intensity

• Idle/alerted/combat• 100 m radius

– Player Intensity• Wounded• Killed enemy• Suit Mode

– Game Intensity• OR of above

• Code– CryAction

• Specification– Music Events and

Points– Game/Libs/MusicLogic

• Graph– /Animation/Graphs

Music Graph

• Animation Graph Code

• Traverse Nodes– Match input values– Enter wait time– Leave wait time– Allow change early

• Mood Node– Set Mood on enter– Wait leave time– Leave early on Allow

Change

• Stinger Node

Script / FlowGraph Control

• Script Entity– SetTheme– SetMood– EndTheme

• Can be triggered or linked to area

• Flowgraph– SetMusicLogic

Questions?

Please post them in the Sound and Music forum on MyCryENGINE.