Demonstration of 3D audio library for video games 1
Akademia Górniczo-Hutnicza im. Stanisława Staszica w
Krakowie
Wydział Informatyki, Elektroniki i Telekomunikacji
KATEDRA ELEKTRONIKI
Praca dyplomowa inżynierska
Demonstration of 3D audio library for video games
Autor: Paweł Rosner
Kierunek studiów: Elektronika i Telekomunikacja
Opiekun pracy: dr inż. Bartosz Ziółko
Kraków, rok akademicki 2015/2016
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 2
Oświadczam, świadom odpowiedzialności karnej za poświadczenie nieprawdy, że
niniejszą pracę dyplomową wykonałem osobiście i samodzielnie i nie korzystałem
ze źródeł innych niż wymienione w pracy.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 3
Chciałbym w tym miejscu złożyć
serdeczne podziękowania dla
Pana Doktora Bartosza Ziółko
oraz Pana mgr inż. Szymona
Pałki z Zespołu Przetwarzania
Sygnałów AGH za nieocenioną
pomoc w trakcie realizacji tej
pracy dyplomowej.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 5
Contents
1. Introduction
1.1 Goals
1.2 Content
2. Sound in video games
2.1 Current approach in game development
3. RAYAV audio engine
3.1 Raytracing
3.2 Soundtracing
4. Unity
4.1 Overview
4.2 Unity Asset Store
5. Implementation
5.1 Workflow
5.2 Game description
5.3 Mechanics
5.4 Models and graphics
6. Summary
6.1 Future plans
7. Bibliography
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 6
1. IntroductionThis project was chosen as a diploma thesis because of great interest in game
development and virtual entertainment. Although I never had any possibility to
gain practical experience in this matter I decided to face this challenge. The topic
of dynamic generation of sound in video games is something new, not only for
me but I think for most people nowadays who are either related to game industry
or not. RAYAV can have huge impact for future commercial productions and I
really wanted to extend my knowledge and connect love to video games and
academic work.
1.1 Goals
The most important part of this thesis was to provide demonstration
environment for RAYAV audio engine. To achieve that the simple video game
needed to be created. The choice fell on First Person Shooter designed in Unity,
as this game engine seemed to be best for beginners.
1.2 Content
Following document contains all information gathered in order to create this
game and all workflow and steps that was necessary to completion. Starting from
general description of sounds in video games, next through ray- and
soundtracing, description of capabilities of Unity, implementation workflow and
ending on summary of whole project.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 7
2. Sound in video gamesOne of the most important feature of every video game since the beginning was
always sound. Music can create tension in thrillers, pathos in war games,
sublimity in fantasy. But aside of soundtrack in-game sound means also sound
effects (sound fx). Even the best graphics can not really let the player feel
surrounding world. Lifelike looking forrest would still be phony without lifelike
singing bird. Sound design for video games has definitely different approach than
sound design for movies. While making films we already know the outcome and
in case of video games such thing is unpredictable. We know that the player has
to perform some actions in order to complete objectives but there is no possible
way of knowing how long does it take him or which paths he use. Most important
thing to keep in mind is that, in games we are unable to control exact sequence
of events that player may experience. So that is why designing and creating
sound effects for this kind of media needs to, on some level, relay more on
probability of events than on some linear script. Vast majority of game
developers simply prepare sound in easy usable chunks that can be use in
runtime.[1]
2.1 Current approach in game development
Gaming industry is currently focused on two ways to achieve realistic sound
effects.[2] First of them concerns producing large numbers of samples and
processing them in order to obtain satisfying result. For example single foot step
should be available in different environments. There is a huge dissimilarity
between walking on the wooden floor or stone or metal stairs. Moreover we also
have to remember about sound propagation. Narrow corridors, dense jungles,
open spaces and many other cases which have to be taken into consideration.
These all locations have huge impact on sound propagation. So this static way of
preparing sound samples requires a lot of work and is very time and resource
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 8
consuming and in the end we still do not can be sure that we provided enough
sound samples which will match every in-game situation and every location
where player might be.
Second way of preparing sound effects rely on dynamic sound processing in real
time. This solution makes single sound sample possible to play in whichever
location in game and every time it will be propagated in a way corresponding to
actual environment. The point is to fully automatize part responsible for working
with sound. [3]
Figure 2.1[4]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 9
3. RAYAV audio engine
The project of developing tool to simulate realistic behavior of sound in closed
and open spaces was challenged by Digital Signal Processing team from AGH
University of Science and Technology in Cracow. The goal was to create hybrid
image renderer by using raytracing and similar soundtracer which can be used in
commercial project in game industry.[5]
3.1 Raytracing
Raytracing is a technology of generating 3D scenes which allows to model physics
of light propagation like reflections, diffractions, refractions and other physical
phenomenons. The algorithm is following:
1. The ray starts from the point where the observer currently.
2. The nearest cross point with objects on the scene in searched.
3. For every light source placed on the scene brightness in that point is marked
according to defined lightning model.
Figure 3.1.1
The ray tracing algorithm [6]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 10
Also in recursive version of the algorithm there is one additional step. If cross
point is part of reflecting or transparent object, derivative rays are produced and
sent and the algorithm recursively repeat itself starting from second step. Color
of the point is generated when results of computing of all derivative rays are
known.
Figure 3.1.2
Image generated with use of ray tracing [7]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 11
3.2 Soundtracing
Soundtracing is a method based on raytracing to generate and track sound waves
in the scene. In most cases with have more than one sound source, single
receiver and every object that is part of the environment and is able to affect
propagation of sound, geometry, materials and so on. Based on this data we can
calculate how sound should be propagated from sources to the receiver. For
example some locations can suppress lower frequencies, some can produce
reverberations or interferences. All physics of sound is very important for realism
and can be simulated by use of soundtracing.
The following image helps to explain how it works.
Figure 3.1
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 12
Z – source P1 , P2 - obstaclesFp – original beam FO1 ,FO2 – reflections
Fd – diffraction [5]
As we can clearly see in Figure 3.1 sound source „Z” is generating sound wave as
a ray. This ray facing obstacle „P1”. In the result new rays are produce. FO1 and FO2
are representation of reflections from obstacle „P1” and ray „Fd” is an effect of
diffraction on edge of obstacle. If original ray does not meet any object on it's
way it can travel unchanged forward. All these rays are create in real time by
RAYAV.
Figure 3.2 [5]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 13
Figure 3.3
Figures 3.2 and 3.3 shows how raytracing works in closed locations. Figure 3.3
was obtained by using Raya Viewer. The tool is also provided with RAYAV and
helps to visualize how sound behave in our scene. Pink cube on this scene
represents sound source and red one in our receiver. We can see how sound rays
are reflected.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 14
4. Unity
4.1 Overview
Unity is a cross-platform game engine created by Unity Technologies. It can be
used in game development for PC, consoles, mobile devices and websites.[4]
Unity is free to personal and commercial usage, although there are some legal
requirements when commercial product starts generating profit.
Unity provides dedicated editor for creating 3D or 2D scenes.
Figure 4.1.1
Unity editor view [9]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 15
It can be used to create every type of scene, either close or open areas. Beside of
graphical part of game engine support also various programming languages to
write scripts like C# or JavaScript. These script allow us to control every aspect of
game behavior. Every script can be attached to game object and starts to be
integral part of our scene.
4.2 Unity Asset Store
The Asset Store is a repository of free and paid-for assets including editor
extensions, scripts, 3D models and more. It provides everything that could be
needed in order to complete project.[10] The repository itself can be accessed
via browser and directly from editor which makes using it even easier and user-
friendly. Almost all models and graphics that are part of my project came from
Asset store and it proves that everyone, even inexperienced in 3D modeling and
graphics can create games.
Figure 4.2.1Asset Store as seen in editor [11]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 16
5. ImplementationThe main goal of this project was to create a game that can show full capabilities
of RAYAV. In order to achieve that it was needed to provide variety of locations.
Thanks to use of narrow corridors, spacious rooms or open spaces in the scene,
player can easily feel aspects of three dimensional sound effects. Moreover it was
necessary to create game in first person perspective. In this perspective it is
possible to most accurately observe environment and to receive sound effects.
The genre of this game is commonly known First Person Shooter. This kind of
video games was chosen because they are very user-friendly. As this project aim
to be a demonstration of capabilities of RAYAV it simply cannot be
overcomplicated or too time consuming for regular person to finish the level. Too
high amount of possible failures, too complex controls or objectives could lead to
premature given up. Furthermore too much pressure on surviving in game and
finishing the level could prevent proper reception of sound effects and finally the
whole presentation would lost it's point.
Figure 5.1
DOOM95 as an example of classic first person shooter [12]
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 17
5.1 Workflow
Creation of project needed several points to complete.
First of them was to build a scene in Unity editor. It was a graphical part of the
project. All geometry of terrain and environment had to be provided, then
enemies had to placed on positions and player had to be given control over his
character. It was also necessary to write considerable amount of C# scripts. The
purpose of these scripts was to control every aspect of game, starting from
controlling character and weapons, through artificial intelligence of enemies and
ending on RAYAV itself.
Figure 5.1.1
Unity editor view
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 18
Next step was to design and produce acoustic map using Blender program. This
kind of geometry is mandatory for RAYAV since it has no ability to read or "see"
graphical models from Unity level. Based on provided acoustic map RAYAV can
perform calculations and generate sound beams. It is essential that these two
maps would have exactly the same dimensions, in other case sound could simply
reflect from walls that we can not see and it would have very negative impact on
reception and realism.
Figure 5.1.2
Blender editor view
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 19
And in the end it was needed to provide set of configuration files which would
load proper acoustic model, set up RAYAV parameters and allow to use it in
game. From the side of Unity the only requirement was to use few simple scripts,
attach them in proper place in the project and inject them into the scripts
controlling for example firing weapon or behavior of enemies.
5.2 Game description
The scenario place player as the only survived soldier inside occupied by
unidentified monsters military facility. His objective is to find the way out of the
infested building and look for the rescue on the other side of the valley. On his
way he will have to face countless enemies and use environment to safely pass
through whole level.
Figure 5.2.1
Main menu
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 20
The designed location is composed of two large buildings separated by sizable
woody valley. First building, where the game starts, has two floors and a secret
tunnel leading to underground garage end then to the exit. After that player
enters to the guarded by monsters valley.
Figure 5.2.2
Designed level
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 21
This location is designed as forsaken remote village deep into the forest. Several
abandoned and ramshackled houses, rusty cars and overall dimness establish
unique nature of this place which brings to mind classic horror movies. On the
way out of the valley the player enters the second building where after he defeat
his opponents the final doors are available to use and he can leave this horrifying
location and successfully finish the game.
Figure 5.2.3
The Valley
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 22
5.3 Mechanics
The project is first person shooter game. To the player's disposal was added one
rifle which is powerful enough to kill an enemy with only few shots. But in order
for the gameplay to not be too easy the gun can only fire in semi-automatic
mode, so in the end player have to wait for certain amount of time until the
weapon is reloaded and available to shoot another time.
The movements are typical for first person shooters and we use keys W, S, A, D
which corresponds to move forward, backward, left and right. Control of the
character is fully supported by special Unity game object - FPS Controller. This
component allows to set up parameters of camera, movement and any special
actions. Additional options for movement was also provided by use of FPS
Controller. Jump - space bar, and run - movement key and shift brings in more
dynamic and can be used in various situations to avoid danger. However these
abilities are limited by statistics of character.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 23
The player's Head-Up Display, commonly known as HUD, is composed of several
important features. First of them is weapon view and crosshair. The gun was
centered in the way to make an impression that player actually hold it in his
hands. To make aiming possible the crosshair was also provided, with use of it
player can accurately fire to the enemies. Another element of HUD are status
bars. There are three of them and each one represents current status of
character's state. First of them - red - represents our health points. This bar
shrinks when player takes damage. If it attribute health reaches zero the
characters dies and level is restarted. The next is yellow armor bar. It is
unreachable and allow us to take some amount of hits from enemy until the
health starts to be reduce. Last of them is stamina bar. This green bar represents
ability to jump and run. When player make any special movement action this bar
shrinks, and when stamina drops to zero player should stop for a moment a wait
for it to regenerate, in other case he will only have ability to slow walk.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 24
5.4 Models and graphics
The vast majority of models and textures originate from Unity Asset Store or was
a standard asset in Unity editor. Asset store turned out to be almost totally
sufficient source of models and there was no need to create them by self.
Downloaded models were used in the project for cars, weapon, small building in
the valley, monsters, doors, containers, barrels and boxes. Only the two larger
buildings, mentioned before, terrain and acoustic map were modeled by use of
Unity editor and Blender. These two editors were enough to achieve all goals and
even huge lack of experience in 3D modeling and graphics was not an obstacle.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 25
6. Summary
Even though the project turned out to be very challenging for someone
inexperienced in game development, the main goal, which was to develop fully
functional demonstration of RAYAV, was accomplished thanks to many
facilitations offered by Unity. The result of all of this is completely playable level
with several minutes of gameplay. The overall framing of relatively simple and
user-friendly game allow user to focus on main aspect of the project - sound
effects. Fulfillment of all the requirements took many months of hard work and
especially lot of learning. Even with huge user friendliness of Unity and general
intuitiveness of this environment it was still very time consuming to get used to
all of the capabilities.
6.1 Future plans
At the moment project has a lot of different paths for development. In the future
it should be firstly extended by adding more levels with various locations which
could better show RAYAV's features. It is also worth to consider adding more
features to the game itself like enlarged user equipment, new weapons and
enemies and improvement of enemy AI. Project is still open and beside of
accomplishment of all main goals there are many ways to make it even better.
Paweł Rosner - AGH 2015/2016
Demonstration of 3D audio library for video games 26
7. Bibliography
[1] http://www.gamesounddesign.com/GettingYourSoundIntoTheGame.html
[2] Karen Collins, An Introduction To The History, Theory
And Practice Of Video Game Music And Sound Design, 2008 MIT
[3] http://www.agh.edu.pl/blog-naukowy/info/article/dzwiek-w-grach-komputerowych/
[4] https://www.youtube.com/watch?v=EWatzCC7rk0#t=32
[5] www.dsp.agh.edu.pl
[6] https://upload.wikimedia.org/wikipedia/commons/thumb/8/83/Ray_trace_diagram.svg/300px-Ray_trace_diagram.svg.png
[7] http://techreport.com/blog/16771/fun-with-raytracing
[8] www.unity3d.com/unity/
[9] http://forum.unity3d.com/threads/how-to-make-unity-
editor-display-in-higher-resolution.355954/
[10] http://unity3d.com/asset-store/help
[11] https://docs.unity3d.com/351/Documentation/Manual/AssetStor
e.html
[12] http://www.techspot.com/article/140-21-first-person-
shooters/
Paweł Rosner - AGH 2015/2016
Top Related