Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for...
-
Upload
markus-marley -
Category
Documents
-
view
213 -
download
0
Transcript of Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for...
![Page 1: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/1.jpg)
![Page 2: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/2.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Advanced Granular Synthesis for Next-Generation Games
Game Developer's Conference 2007San Francisco
Leonard J. Paul
Vancouver Film School
Game Audio Instructor
info {at} VideoGameAudio.com
604-685-5808 x4035
![Page 3: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/3.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Overview
1)How to utilise granular for speech, sound effects, music and engines
2)Prototyping granular synthesis using Pure Data and Open Sound Control
3)How to add granular synthesis to your sound driver
![Page 4: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/4.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Using granular forspeech
sound effectsmusic andengines
1)
![Page 5: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/5.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
1) Selection order (fwd/rev or freeze)
2) Pitch shift
3) Amplitude range
4) Spatialization / panning
5) Grain duration
6) Grain density (grains/sec, # of grain voices)
7) Envelope (ASR shape or windowing function)
8) DSP effect (reverb, filtering...)
9) Feedback amount (for granular delay lines)
Granular Parameters
A S R
Vol
![Page 6: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/6.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Game Parameters
● Granular parameters driven by game state
● (ex. RPM -> pitch)● Physics allow for continuous control
over playback of sample● (ex. ragdoll -> foley)
● Input parameters need to be carefully tuned to drive synthesis
● (ex. instrument played by novice)
![Page 7: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/7.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Game Audio Data Diagram
Game(Physics, GFX,
AI etc..)
GameAudioCode
GranularDriverCode
DAC
Commercial
SoundTool
In-houseSoundTool
AudioData
![Page 8: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/8.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Speech
● Time warping speech for variation● Change formants of voice● Realtime granular effects to change
character of voice
![Page 9: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/9.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Sound Effects
● Crossfade morph via grains● Spacialization in 5.1/7.1● Contact physics for sliding/scraping● Add variation to existing sounds
![Page 10: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/10.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Engines
● Add as a layer for variation● Pre-segmentation and playback via
RPM/load band:
Loa
d
RPM
Samples
Low RPM & low load grains
![Page 11: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/11.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Music and Ambiences
● Stretch a sound for a granular drone● Tempo-sync grains for interesting
gating effects● Merge ambiences by selecting grains
from multiple files● Making an ambience sound more
musical and abstract
![Page 12: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/12.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Prototyping Granular Synthesis using Pure Data and OSC
2)
![Page 13: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/13.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Coding PD Objects
● FlexT is a cross-platform method of generating custom C++ PD objects: http://grrrr.org/ext/flext/
● Easily possible to port part of the Synthesis Toolkit (STK) to PD using Code Blocks or any other DSP code
![Page 14: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/14.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Flext (gain~ object code)class gain: public flext_dsp
{ FLEXT_HEADER(gain,flext_dsp)
float factor;
gain(): factor(0)
{ AddInSignal("In"); // add signal inlet
AddInFloat("Gain");
AddOutSignal("Out"); // add signal outlet
FLEXT_ADDMETHOD(1,m_gain); }
virtual void m_signal(int n,t_sample *const *insigs,t_sample *const *outsigs) {
const t_sample *in = insigs[0];
t_sample *out = outsigs[0];
while(n--) *(out++) = *(in++)*factor; }
void m_gain(float db) { factor = pow(10.f,db*0.05f); }
FLEXT_CALLBACK_F(m_gain); };
![Page 15: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/15.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
OSC Layer for Prototyping
● Add an Open Sound Control (OSC) layer to your game to communicate with a rapid prototyping audio environment such as PD
● Example of using OSC with a Half-life 2 Source Mod
● Possible to compile PD to a language such as Java (pd2j2me)
![Page 16: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/16.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
OSC Code for Sending Msg#include "osc/OscOutboundPacketStream.h"
#include "ip/UdpSocket.h"
#define ADDRESS "127.0.0.1" // localhost but can be any address
#define PORT 7000
#define OUTPUT_BUFFER_SIZE 1024
int main(int argc, char* argv[])
{
UdpTransmitSocket transmitSocket( IpEndpointName( ADDRESS, PORT ) );
char buffer[OUTPUT_BUFFER_SIZE];
osc::OutboundPacketStream p( buffer, OUTPUT_BUFFER_SIZE ); // create buffer
p << osc::BeginMessage( "/test" ) << "hi world" << osc::EndMessage; // message
transmitSocket.Send( p.Data(), p.Size() ); // send message
}
![Page 17: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/17.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Adding OSC to Half-life 2
● Download the OSCpack library from:http://www.audiomulch.com/~rossb/code/oscpack/
● Download Visual C++ 2005 Express, Half-Life 2 and create a new mod
● Add the OSC files to the mod source (some tweaking needed) and modify SoundEmitter.cpp etc.
● This is a similar process to adding OSC to your own sound driver
![Page 18: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/18.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Adding Granular Synthesis to your Sound Driver
3)
![Page 19: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/19.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Granular Sound Driver
● PD C++ code can be ported to the next-gen consoles as the consoles are primarily software based
● Synthesis should be creatively used in conjunction with samples for sound design
● Advantage is that real-time control over synthesis parameters is from game parameters
![Page 20: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/20.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Granular Code ExamplesRoss Bencina – Granular Delay Line● Uses Borland C++ Builder
http://www.audiomulch.com/~rossb/
Synthesis Toolkit (STK) – Granulate● Can use flext to port to PD
http://ccrma.stanford.edu/software/stk/
SyncGrain~ for PD (from Csound)● Ported using flext
http://footils.org/cms/pms/
![Page 21: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/21.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Related Research
● Concatenative Sound SynthesisSound "Mosaic" Analysis & ResynthesisUses grains from one sample to
approximate grains of a second samplewww.mat.ucsb.edu/~b.sturm/sand/VLDCMCaR/VLDCMCaR.html
![Page 22: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/22.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Related Research
● Natural Grain ResynthesisMake grains that preserve transientshttp://www.cs.ubc.ca/~reynald/naturalgrains.html
![Page 23: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/23.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Related Research
● Pure Data Compiler (pd2j2me)Compile PD code to Javahttp://www.uow.edu.au/~mh675/publications/nime_2005.pdf
public void met0Bang() { double a; a = counter.bang(); a = a * 3; a = a / 4; }
![Page 24: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/24.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Example
● Example of gameplay utilizing granular synthesis
![Page 25: Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com Advanced Granular Synthesis for Next-Generation Games Game Developer's Conference 2007 San.](https://reader035.fdocuments.us/reader035/viewer/2022062809/56649cb95503460f9497f96a/html5/thumbnails/25.jpg)
Web:: VideoGameAudio.com Email:: info{at}videoGameAudio.com
Questions?
Email: info-at-VideoGameAudio.com
Web: www.VideoGameAudio.com