Kinect Audio-Runner: Audio feedback for improving...
Transcript of Kinect Audio-Runner: Audio feedback for improving...
Kinect Audio-Runner: Audio Feedback for Improving Performance
in Long-Distance Running
J O R D I B O L Í B A R
Master of Science Thesis Stockholm, Sweden 2012
Kinect Audio-Runner: Audio Feedback for Improving Performance
in Long-Distance Running
J O R D I B O L Í B A R
DT212X, Master’s Thesis in Music Acoustics (30 ECTS credits) Degree Progr. in Electrical Engineering 270 credits Royal Institute of Technology year 2012 Supervisor at CSC was Roberto Bresin Examiner was Sten Ternström TRITA-CSC-E 2012:062 ISRN-KTH/CSC/E--12/062--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.csc.kth.se
Kinect Audio-Runner: Audio feedback for
improving performance in long-distance running
Abstract
Unlike many other sports, running is perceived as a sport that can be easily practiced by
everybody without any need for a proper technique or previous training. Moreover, once a
running pose is adapted, it becomes fairly difficult to change it for good. However, audio
feedback is becoming a popular method for aiding people to improve their performance in
sports. Hence, the aim of this project is to help a runner to improve his technique on a treadmill
by sending him feedback by means of music and auditory icons. The movements of the runner
are tracked with a non-intrusive method using Kinect. Three different elements of the running
technique are taken into account and a specific training can be set for every runner by setting
different thresholds as goals to achieve during the training. Therefore, the training becomes
more intuitive and pleasant for the runner. All these features have been assembled in a program
which has been called Kinect Audio-runner.
Kinect Audio-Runner: Ljudåterkoppling för
träning av långdistanslöpare
Sammanfattning
Till skillnad från många andra sporter, löpning uppfattas som en sport som lätt kan praktiseras
av alla, utan något behov av en korrekt teknik eller tidigare utbildning. Dessutom, när man har
lärt en teknik, blir det ganska svårt att ändra den i framtiden. Å andra sidan är ljudåterkoppling
på väg att bli en populär metod för att hjälpa människor att förbättra sina resultat inom idrotten.
Därför är syftet med detta projekt att hjälpa en löpare att förbättra sin teknik på ett löpband
genom att ge honom återkoppling genom musik och ljud. Löparens rörelser analyseras med en
Kinect kamera. Tre olika delar av löpteknik tas hänsyn till och en särskild träning kan ställas in
för varje löpare genom att sätta olika tröskelvärden som mål att uppnå under träning. Därför blir
träningen mer intuitiv och trevlig för löpare. Alla dessa funktioner har samlats i ett program som
har kallats Kinect Audio-Runner.
Acknowledgments
I would like to thank everyone who helped me along this project. Daniele Cardinale, from the
Sport Physiology Laboratory, which is part of the Elite Performance Centre of the Swedish
Sports Confederation. Thanks for letting me use the sports lab to carry the tests and for all the
advice and feedback. My friends Etienne, Martin and Juan for putting up with many hours of
running on a treadmill, testing and improving the program. Roberto Bresin, my advisor, for all
the advice, help and material he has provided me. This work was supported by the Swedish
Research Council, Grant Nr. 2010-4654.
Table of contents
Introduction ................................................................................................................................... 1
Background ............................................................................................................................... 1
Goals ......................................................................................................................................... 2
Thesis content ............................................................................................................................ 2
Related work ................................................................................................................................. 3
Sonification and sports engineering .......................................................................................... 3
Biomechanics ............................................................................................................................ 3
Psychology of perception .......................................................................................................... 4
Problem formulation ..................................................................................................................... 5
Kinect and motion tracking ........................................................................................................... 6
Kinect functionalities ................................................................................................................ 7
Body tracking ............................................................................................................................ 8
Biomechanics and physics in running ........................................................................................... 9
The running cycle ...................................................................................................................... 9
What determines a good technique? .......................................................................................... 9
The tilt angle of the upper body .......................................................................................... 10
The distance between the landing foot and the projection to the ground of the centre of
gravity ................................................................................................................................. 11
The vertical displacement of the runner’s centre of gravity ................................................ 11
Methodology ............................................................................................................................... 12
Installing the drivers ................................................................................................................ 13
Pure data program ................................................................................................................... 13
Interface................................................................................................................................... 14
Audiorun ................................................................................................................................. 16
Record data .............................................................................................................................. 20
Skeleton and calculations ........................................................................................................ 21
Vertical displacement .......................................................................................................... 22
Step distance ........................................................................................................................ 27
Tilt of the torso .................................................................................................................... 30
Sonification ............................................................................................................................. 31
Sonification of the vertical displacement ............................................................................ 32
Sonification of the tilt .......................................................................................................... 35
Sonification of the step distance .......................................................................................... 36
Overall ................................................................................................................................. 38
Stats ..................................................................................................................................... 38
Results ......................................................................................................................................... 40
Set-up ...................................................................................................................................... 41
Perception tests ........................................................................................................................ 43
Vertical displacement perception ........................................................................................ 43
Tilt perception ..................................................................................................................... 44
Step distance perception ...................................................................................................... 44
Combined perception: Vertical displacement + tilt ............................................................. 45
Combined perception: Vertical displacement + tilt + step distance .................................... 46
Video ....................................................................................................................................... 46
Conclusions and further work ..................................................................................................... 47
Conclusions ............................................................................................................................. 47
Further work ............................................................................................................................ 47
Bibliography ................................................................................................................................ 49
Appendix ..................................................................................................................................... 51
List of used terms .................................................................................................................... 51
Table of figures
Figure 1 - Microsoft's Kinect (http://xboxconsole.blogspot.se) .................................................... 7
Figure 2 - Depth image from Kinect ............................................................................................. 8
Figure 3 - Tracked skeleton with Kinect ....................................................................................... 8
Figure 4 - The running cycle
(http://csmres.jmu.edu/biology/Bio490/Biomechanic%20Webposter/background.htm).............. 9
Figure 5 - Runner leaning forward and the produced resulting forces (edited from
http://wakatennis2011.blogspot.se) ............................................................................................. 10
Figure 6 - OSCeleton calibration pose (http://www.kinecthacks.com/guides/bvh-motion-
capture-guide/) ............................................................................................................................ 14
Figure 7 – Kinect audio-runner program structure. ..................................................................... 14
Figure 8 - User interface with its different buttons and sliders. It consists in four steps: 1-
Analysis of the current running technique, 2- Setting the desired thresholds for the training, 3-
Training with audio feedback and 4- Display of the results. ....................................................... 16
Figure 9 - Incoming Kinect OSC messages. A message is forwarded to different patches
depending on its header. .............................................................................................................. 17
Figure 10 - Skeleton plot and visualisation settings. ................................................................... 18
Figure 11- Input and output of the audio feedback. A local music file can be used or the line in
of the sound card can be directly read. ........................................................................................ 18
Figure 12 – Sound played once the program is started. .............................................................. 19
Figure 13 - Reset connections. Depending on what wants to be repeated different values will
need to be cleared. ....................................................................................................................... 19
Figure 14 - Record and play data Pd patch. The OSC messages are changed to fit the format of
the qlist function. ......................................................................................................................... 21
Figure 15 - draw-skeleton Pd patch. It creates the skeleton in the 3D space and also computes
the three analysed parameters of the training. ............................................................................. 22
Figure 16 - Filtering and de-normalization for the vertical displacement of the CG .................. 23
Figure 17 - Example of absolute vertical displacement with noise ............................................. 23
Figure 18 - Finding minimum values of the vertical displacement ............................................. 24
Figure 19 - Peak detection points ................................................................................................ 24
Figure 20 – Example of relative vertical displacement with noise ............................................. 25
Figure 21 – Peak detection for the vertical displacement ........................................................... 25
Figure 22 - Example of vertical displacement with step maximum values ................................. 25
Figure 23 - Transition detection for the vertical displacement ................................................... 26
Figure 24 - Example of vertical displacement with detected transitions..................................... 26
Figure 25 – Error corrector for vertical displacement ................................................................. 27
Figure 26 - Example of vertical displacement processed for sonification .................................. 27
Figure 27 – Filtering and de-normalization of the step distance data ......................................... 28
Figure 28 – Maximum feet depth detection ................................................................................ 28
Figure 29 - Example of maximum foot depth detection ............................................................. 29
Figure 30 - Average foot distance values calculation ................................................................. 29
Figure 31 - Filtering and de-normalization of the torso tilt data ................................................. 30
Figure 32 - Tilt angle calculation ................................................................................................ 30
Figure 33 - Tilting patch.............................................................................................................. 31
Figure 34 - Sonification patch ..................................................................................................... 32
Figure 35 – Vertical displacement sonification patch. The vertical displacement values are
averaged and sent to the twang patch. ......................................................................................... 33
Figure 36 - Frequency adjustment on the twang patch ............................................................... 34
Figure 37 - Volume adjustment of the twang patch .................................................................... 35
Figure 38 - Tilt sonification patch. The tilt angle determines the cutting frequency of the low
pass filter. .................................................................................................................................... 36
Figure 39 - Step distance sonification patch ............................................................................... 37
Figure 40 - Overall performance sonification patch. If all the required conditions match for a
while, an auditory icon is played to tell the runner to keep on doing like that. ........................... 38
Figure 41 - Statistics patch .......................................................................................................... 39
Figure 42 - Test in the Sport Physiology Laboratory, from the Swedish Sports Confederation . 41
Figure 43 - Kinect tracking a runner (edited from
http://www.truefitness.com/galleries/37/cs550-treadmill-images) ............................................. 42
Tables
Table 1 - Vertical displacement test parameters ......................................................................... 43
Table 2 - Tilt test parameters ....................................................................................................... 44
Table 3 - Step distance walking test parameters ......................................................................... 45
Table 4 - Step distance running test parameters .......................................................................... 45
Table 5 - VD + tilt test parameters .............................................................................................. 45
Table 6 - Complete test parameters ............................................................................................. 46
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
1
Introduction
Background
In the last years, auditory display has become quite popular together with visualization in order
to present data or send feedback in different contexts. Psychological studies have concluded that
it is really intuitive for the human brain to react to sounds; therefore, the concept of sonification
has started to be used in sports and other activities that require a good body coordination and
technique. Another positive aspect about sonification is that in most sports, the sportsperson
must focus his sight on many different elements of the sport he is practicing, which makes it
almost impossible for him to pay attention at the same time to visual feedback. However, audio
feedback is way less intrusive in that sense, and most sports can be performed normally even if
the runner must be able to hear certain sounds.Sonification has already been applied in a wide
variety of fields such as: physical exercise, games, physiotherapy, etc.[1]. On the other hand,
many runners are eager to find a good technique which allows them to improve their
performance and reduce the chances of injuries at the same time; or they simply learnt by
themselves and developed some bad running habit which later became impossible to change. By
reading books or just taking some advice it is not so easy to correct your own technique, since it
requires a lot of outsight and not just focusing on a single part of the body but the whole as a
pack. That is why runners have recently been looking at new technologies that help them enjoy
running even more. A remarkable example is the new product developed by Nike and Microsoft
called “Nike+ Kinect Training”, which through the help of a motion capture sensor (the same
used in this project), the sportsperson is assisted in his training through visual and audio
feedback [21]. On the other hand, new philosophies and techniques have appeared in running,
claiming to have found the “perfect” way to run. However, there is a lot of controversy
concerning that, since experts do not seem to agree about these philosophies. Many say that they
are mere commercial products, with a clear aim focused on making money and being different
than their rivals. The two main ones are Pose running and Chi running, which share many
characteristics.In any case, there are some clear facts that will be presented later in detail, that
can simply be analysed by physics and that have been accepted by many different professionals
[2][3].
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
2
Goals
In this project, a method is presented for aiding a long-distance runner to correct and improve
the technique in real time. By tracking and analysingthe runner’s movements and sending audio
feedback to him,a much more intuitive way to correct the pose can be achieved; thus,the runner
can adjust the pose in real time. Normally, the tracking of a runner is performed in some kind of
intrusive manner, whether placing sensors to the runner’s body or by changing somehow the
normal running procedure. In this project, the runner is tracked with Kinect, a motion sensing
camera by Microsoft which can track more than 14 joints of a human body. By using this, the
runner can run as always on a treadmill and it is still possible to track the body movements and
extract all the desired meaningful data.
Thesis content
This report is structured in the following way. After the introduction, an overview of the
previous work done is presented. Since this project assembles extremely different topics such as
psychology, biomechanics or engineering, the related work has been separated into topics.
Then, a problem formulation is stated, setting the scope and goal of the project. After this, a
series of introductory chapters explain all the necessary background about the motion sensor
used and the biomechanics of running. After this, the methodology used in this project is
explained. How the Kinect Audio-Runner program is structured, how each single problem has
been faced and how all the different parts of the software interact together. Finally, the results of
this project are shown, explaining the tests carried in the laboratory and later on, some
conclusions and useful thoughts for future work are drawn.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
3
Related work
Sonification and sports engineering
At the Music, Speech and Hearing department of Kungliga Tekniska högskolan, sonification
has been applied in different projects. Firstly, sonification was used for improving a runner’s
technique by analysing the vertical displacement of a runner with a smartphone’s accelerometer
[4]. Another project, focused on the improvement of performance in rowing [5]. Different
sensors were placed in the boat and the oars in order to compute different information like the
horizontal acceleration and displacement and send audio feedback to the rowers in order to
improve their technique. On the other hand, there are other projects that have worked with
sonification with runners. A study by Barrass and colleagues [6] studied the preferences by
casual runners for different types of audio feedback. This was achieved by asking runners to
carry a probe logging their preferences among six sonification models. In another study, music
was used in three types of sonification of the degree of motoric synchronization in active music
listening [7]. The interesting contribution of this study was that the sonification itself was music
that changed in different ways depending on how coordinated the movements were. Thomas
Hermann, Andy Hunt and John G. Neuhoff edited a handbook with contributions from several
authors about an introduction to sonification and auditory display. The book gathers information
concerning sonification from a wide range of fields and explains different techniques and useful
applications for it [1]. Recently, Nike and Microsoft have developed a videogame for Xbox 360
which helps people to train at home with the aid of Kinect. The game gives different challenges
which the user must complete by performing certain movements that are tracked and analysed
by Kinect [21].
Biomechanics
In the field of biomechanics and running analysis, Branko Skof and Stanko Stuhec carried a
thorough analysis of Jolanda Ceplak’s running motion[8]. She is an elite runner, holder of a
world record for 800m. The angles, displacements, speeds and accelerations of many of her
limbs and joints were captured, plotted and studied. Finally, some conclusions are extracted
about what determines her good technique and makes her run that fast. Jessica Gonowon
analyzed The Physics of Efficient Running in a web project [9]. Different aspects on how
physics affected the running technique were studied, such as forces, pendulums, the center of
mass or the pelvic rotation. Tom F. Novacheck wrote a thorough analysis about the
Biomechanics of running [10]. He summarizes most of the literature available so far about the
running gait, and compares it to walking and sprinting. Moreover, he also analyses kinetic and
kinematic data, including times of each phase, position of the different limbs or EMG in order
to know the activity of the muscles involved. Steve Magness posted a long article in his “The
Science of Running” website about how to run with proper biomechanics [11].
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
4
He describes with a high amount of detail the process of running and gives lots of
recommendations on what to take into account when trying to improve your technique. In an
instructional video made by the Okemos Playmakers running store, some key elements of the
running technique are explained in a very intuitive way. The elements that have been taken into
account are extremely relevant for this project [12].
Psychology of perception
In psychology, it has been proved that there is a strong relationship between sounds and actions,
and it can be used to ease and help people in the process of learning something. Sonification is
basically using this in order to use audio as information. In [13], E. Kohler et. al. studied the
neuronal reactions of a set of monkeys after observing certain actions involving a sound. They
concluded that the same neurons from the monkeys discharged when they performed the action
or they heard the related sound. This enhances the idea of this project of using only sound
feedback as information. When the runner starts performing the training, he will hear sounds
that will relate to the motion of his body. After a while, once he becomes familiar with them, he
will learn how to adjust his pose in order to make them sound in an attractive way (correct
technique). After some training, if the sound feedback is removed, the runner will still be able to
run as he was taught since he will be able to anticipate the sounds that he would hear just like if
he was doing the training.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
5
Problem formulation
In order to carry this project, some boundaries and requirements have been specified to set its
scope. The goal is to give audio feedback to both casual and professional runners to adjust their
pose and technique in real time. There have been similar projects with sonification with
running, but they never gathered all the features included in this project. They are the following
ones:
The motion capture and body tracking are performed with a non-intrusive method
which is explained later thoroughly. Neither sensors nor any especial equipment must
be worn by the runner.
The audio feedback takes into account more than one parameter of the runner, so the
runner must interpret the multiple changes in the feedback in order to know what to
change.
Due to constraints on the distance where the tracking can be performed, the runner must
run on a treadmill. The technique is trained and learnt on the treadmill and later applied
when running on a real track.
The motion of the arms is not taken into account. Since the runner is tracked from
behind, when the arms swing in front of the chest they remain hidden. Thus, the
information provided is not reliable, plus the arms are not as important as the rest of the
analysed parameters.
The training is not based on any specific threshold values of the three analysed
elements. The program gives total flexibility to the runner and trainer to set the desired
thresholds to achieve. Therefore, it can be applied to achieve any desired running
technique or philosophy.
The training is built on background theory of long-distance running energy economy.
This means that it is aimed at average speed running, such as marathon running. It
would not apply for sprinting, where energy consumption is not the main concern and
other parameters have to be taken into account.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
6
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
7
Kinect and motion tracking Before going into the details of the project, an explanation about the motion tracking system
that has been used will be given. Then, the methodology for tracking the motion of the runner’s
body will be assessed.
Kinect functionalities
Kinect (see figure 1) is a motion sensing camera, designed by Microsoft for the Xbox 360
console. It features an RGB camera, a 3D depth sensor, a motor to adjust the tilt of the camera, a
multi-array microphone and a LED which can be controlled. After some time, Microsoft
released the Kinect Software Development Kit (SDK), which allowed users to develop their
own applications in C++/CLi, C# or Visual Basic .NET. Later on, open source drivers appeared,
and PrimeSense released a motion tracking middleware called NITE. OpenNI packed the
binaries, the middleware and the hardware drivers into an installation pack, which is
multiplatform (Linux, Windows and Mac) unlike the Microsoft SDK, which was only for
Windows.
Figure 1 - Microsoft's Kinect (http://xboxconsole.blogspot.se)
The Kinect sends the tracked skeleton, RGB image and depth map (see figure 2)through the
USB cable to the computer. The different messages are treated by the OpenNI framework in
order to produce meaningful data. However, in this project, a program called OSCeleton [14]
has been used; which works as a proxy that broadcasts the skeleton data from the Kinect
middleware as OSC messages to a certain port. These messages are the ones directly read by the
software designed for this project, which will analyse the runner’s pose and send the audio
feedback. The input interface of an application called OSCeleton has been used in order to
process these messages and record and plot them.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
8
Figure 2 - Depth image from Kinect
Body tracking
For this project, only the tracked skeleton information has been used. It is structured in 14 joints
(head, neck, torso, right and left shoulders, right and left elbows, right and left hands, right and
left hips, right and left knees and right and left feet) as it can be seen in figure 3. Each of these
contains its 3D position (X, Y, Z) with the Kinect as the origin of the 3D space (0, 0, 0).Once
the Kinect starts sending messages, first it will send a message when a new user (person in its
view range) is detected. After this, it will send another message when the skeleton starts being
tracked, meaning that it will start sending its coordinates soon. Finally, it will send a message
when a user is lost.
Figure 3 - Tracked skeleton with Kinect
The Kinect is able to track each of the skeleton’s joints if they are directly visible. This means
that if any joint is hidden behind an object or even the body itself, the Kinect will lose track of it
and will mostly produce a random or determined position. This has presented some troubles
when implementing the program as it will be explained in detail later. Moreover, more than one
user can be tracked at the same time; but this possibility has not been used in this project, hence
it will not be further analysed.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
9
Biomechanics and physics in
running In order to know what are the characteristics of a proper long-distance running technique,
answers must be found in biomechanics and physics. There are many philosophies or techniques
that are aimed to be sold as a commercial product, by selling a training pack with an infallible
method which claims to be the “perfect running technique”. These kinds of approaches do not
have a unanimous response among the elite running community, however, there are some facts
explained but physics that are widely accepted [2]. The aim of this project is not to cover all the
aspects of a good technique, but to focus on three of them and try to correct them in an intuitive
and pleasant way.
The running cycle
First of all, a little introduction to what the running cycle is will be presented, in order to
understand its different phases. The running cycle is formed by four different phases: stance,
float, swing and float again. They are depicted in Figure 4.
Figure 4 - The running
cycle(http://csmres.jmu.edu/biology/Bio490/Biomechanic%20Webposter/background.htm)
In each one of these phases there is a key element to take into account and to correct. In the
following chapter the characteristics of a good running technique that have been used for this
project are explained. These are the criterions that are applied when determining the audio
feedback that the runner receives.
What determines a good technique?
When running, ideally all the used energy must be spent in moving forward. When negative
horizontal accelerations are produced, the runner is slowed down. Since energy is also spent in
lifting the runner’s body, it is extremely important to have a pose that minimizes this energy
consumption in order to invest it on moving forward [9].
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
10
All these factors are explained by physics, and three very important ones have been chosen for
this project as the parameters to be analysed and sonified. The reason for choosing these three
and not others is that they are extremely key elements of the running cycle and at the same time
they can successfully be tracked by the Kinect. There are other interesting elements that would
be interesting to track, such as the swing of the arms. However, as it has been explained before,
the arms are hidden behind the chest thus making it impossible for the Kinect to track them.
The three parameters that are analysed are:
The tilt angle of the upper body
The angle of the body of a runner is a key aspect on how the forces are distributed in the
horizontal axis. The more the runner leans forward the more weight is put in front of the centre
of gravity. For the case of the human body, the centre of gravity is exactly the same as the
centre of mass, so from now on these two terms will be used to refer to the same thing. It is
obvious that in reality it does not happen this way, but if the weight of the torso is put before the
centre of gravity the upper body is pulled by gravity and it tends to fall, thus producing a
movement forwards (see force vectors depicted in figure 5). The runner must try to use this in
his favorin order to reduce energy consumption. If the upper body were tilted backwards, the
resulting effect would be the opposite: gravity would pull the runner backwards thus slowing
him down[9]. It is important to state that the tilt of the body must come from the ankle; it is not
enough to just run with the legs straight and tilt the torso. On this project the analysed angle
comes from the torso due technical reasons from the tracking, but unless the runner runs in a
very awkward way the angle of the torso should be the same as the angle of the whole body.
Figure 5 - Runner leaning forwardand theproduced resulting forces(edited from
http://wakatennis2011.blogspot.se)
The further the runner leans forward the faster he will be able to run. As the distance to the
centre of gravity increases, the force produced by gravity also increases. This resulting
magnitude can be calculated using equation 1:
𝐹𝑔𝑟𝑎𝑣𝑖𝑡𝑦 =𝑔 ∙ 𝑀 ∙ 𝑚
𝑑 (1)
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
11
Where: g is the gravitational constant (9.81 𝑚/𝑠2 on Earth), M is the mass of the Earth, m is the
mass of the runner and d is the distance between them.
This statements stands since the more the runner leans forward, the closer his centre of gravity
becomes to the floor (smaller d) and therefore he is pulled with a greater force [9].
The distance between the landing foot and the projection to the ground
of the centre of gravity
Another extremely important aspect on the running cycle, closely related with the previous one,
is the distance between the landing foot and the projection to the ground of the centre of gravity
during the stance phase. If the foot lands in front of the centre of mass, it represents an opposing
force to the forward movement, thus slowing down the runner. Landing behind the centre of
mass can be quite awkward and can make the runner lose balance. The best thing to do is to
land the foot right under the centre of mass or as close as possible to it. This makes the whole
stance phase much natural and helps to relax the ankle and calf muscles, since they do not have
to hold and stop the runner’s weight for that long [9][12]. The legs must flow under the torso of
the runner, following the motion and inertia.
The vertical displacement of the runner’s centre of gravity
A key aspect when running, is trying to minimize the vertical displacement of the runner. The
energy must be spent in moving forward, not upward. However, the goal is not to run with a
completely null vertical displacement. Every time the foot lands on the ground, a force is
produced when the leg extends, thus pushing the runner forward. As if it were a cannonball,
there is a trajectory that allows reaching the furthest point possible. This means that the vertical
and horizontal component of the stride must be optimized for every gait in order to achieve the
longest possible distance [3].If the vertical displacement is reduced, this also means that the
impact suffered by the feet and legs is mitigated thus decreasing the chances of getting injured.
As it was shown in a previous study from the same department in KTH as the one from this
project, there is a direct relationship between the vertical displacement of the runner’s centre of
gravity and the energy consumption [4]. Therefore, this represents an extremely important
element in running economy.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
12
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
13
Methodology
Installing the drivers
First of all, in order to get Kinect working on a PC, a series of drivers and middleware must be
installed. They have been developed by PrimeSense and they consist on OpenNI and NITE. It is
possible to install them separately from the OpenNI website, but for this project an auto installer
was used which included all of them in the same installer [20]. This version is called
“BrekelOpenNIKinect Auto Installer – Developer Editionv1.5.4.0.exe”and after installing it the
Kinect is ready to be used. Moreover, a software developed by the same people was
installed;the “BrekelKinect Setup v0.50.exe”. It gives a visual interface where the depth image
from the Kinect, the RGB image and the tracked skeleton can be seen at the same time. It also
provides you with functionalities for recording audio and tracked movements. Nonetheless,
these were not used for this project and this software was mainly used to make sure that the
Kinect was correctly placed and the runner did not go out from the screen while running during
the initial set-up.
Pure data program
In this section the different steps followed in the development of the program called Kinect
Audio-Runner will be shown in a structured way. This project has been developed in Pure data
(Pd) [15], a visual programming language aimed at creating multimedia works. Pd is structured
in patches and sub-patches, which separate the different parts of a program and help to structure
and interconnect these different parts. The patches will be explained following their hierarchy.
Many of the patches used in this project have not been developed by me, since they belong to a
project called OSCeleton which has been used and further expanded with the motion
recognition and sonification algorithms. Before executing the Pd code and starting to track the
runner, the OSCeleton proxy must be executed in order to send the Kinect messages to the port
7110 of the computer. Once OSCeleton has been executed, it will start looking for users. In
order to correctly track a user, the runner must adopt the psi pose, as it is shown in Figure 6.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
14
Figure 6 - OSCeleton calibration pose (http://www.kinecthacks.com/guides/bvh-motion-capture-guide/)
Once the runner is detected, he will be able to start running normally and all the skeleton
information will be correctly received by the Pd code. The Pd code is composed by different
patches, structured in the following hierarchy depicted in Figure 7.
Figure 7–Kinect audio-runner program structure.
Interface
This patch is the interface which the user uses in order to interact with Kinect Audio-Runner. It
consists on different buttons, toggles and sliders which control the different sections of the
program and display some information in a graphical way. It also displays the instructions in
different steps in order to carry on with the sonification training for a runner.
Inte
rfac
e
Audiorun
sonification
vd_sonification twang
step_sonification
tilt_sonification
stats
overall
draw-skeleton
vertical_disp
tilting
step_dist
connection
joint
record data
results
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
15
The different steps are:
1. Running analysis: The runner runs for a desired time and the three different parameters
of his running technique are analysed and stored. No audio feedback will be given
during this test period. The test button must be pressed again in order to stop the test.
2. Setting thresholds: Once the mean value of the three different parameters has been
computed, a threshold of improvement can be set in order to perform the sonification
training. The lower the threshold is for the vertical displacement and the step distance,
the stricter the sonification will become. For the tilt angle case, the higher the threshold
is set, the more demanding will be the training. When changing this value all the
sonification thresholds are adjusted in consequence.
3. Start the sonification training: Once the thresholds have been set, the runner can start
the training. It is possible to select which of the three elements want to be trained in
order to train any combination of them. The music will start as well as the vertical
displacement audio feedback and the good technique auditory icon. While this mode is
selected, the runner will receive audio feedback depending on these movements and the
previously specified thresholds.
4. Display results: After concluding the training, a series of results are displayed in the
interface for the runner to check on his performance and in order to see how much he
improved.
Moreover, it also allows displaying or hiding the tracked skeleton, checking the three analysed
parameters with sliders, to play, stop or adjust the volume of the music file being played and to
reset all the variables in order to sonify again. The user interface is shown in figure 8.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
16
Figure 8 - User interface with its different buttons and sliders. It consists in four steps: 1- Analysis of the
current running technique, 2- Setting the desired thresholds for the training, 3- Training with audio
feedback and 4- Display of the results.
Audiorun
This patch is responsible to gather all the other patches and sub-patches. In some ways it is
similar to the Interface but without buttons and toggles and presented in way it would not be
understood by the end user.
However, its main purpose is to take the incoming OSC messages broadcasted by OSCeleton
from the Kinect and send them to the draw-skeleton patch and to the record data patch in case
the movements of the runner want to be recorded for further use. The OSC messages are
received through the port 7110. Once a new user has been discovered and the body is already
being tracked, the received messages have the following structure:
Address pattern: "/joint"
Type tag: "sifff"
s: Joint name, check out the full list of joints below.
i: The ID of the user.
f: X coordinate of joint in interval [0.0, 1.0]
f: Y coordinate of joint in interval [0.0, 1.0]
f: Z coordinate of joint in interval [0.0, 7.0]
For example:
/joint r_foot 1 0.577044 0.812729 2.79039
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
17
It can be seen that the coordinates, for the X and Y values, go from 0.0 to 1.0 and for the Z
value from 0.0 to 7.0. The reason for this is that the space where the Kinect can track a user is
much deeper than wide or tall. A user will be able to move roughly around 6 m2, between a
distance of 0.7-6 meters, whilst the horizontal field at minimum viewing distance would be
around 0.8 meters and for the vertical field around 0.6 meters [16]. That is why for the Z
component it is logical to have a wider range of values.
The Pd code responsible for this is shown in Figure 9.
Figure 9 - Incoming Kinect OSC messages. A message is forwarded to different patches depending on its
header.
The route function routes the /joint and /new_skel messages to different paths. The /joint
messages are sent to the draw-skeleton patch, where the different nodes are plotted and also
where the three parameters to analyse the technique are computed. The /new_skel messages are
used to create a new skeleton and plot it together with the previous one.
On the other hand, it is also responsible for plotting the runner’s movements on a separate
screen using a 3D space representation from Pd called Gem. Different parameters are specified
when creating the Gem window, such as: the lighting, perspective, title of the window, rotation
of the different joints, etc. The section responsible for this is shown in figure 10.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
18
Figure 10 - Skeleton plot and visualisation settings.
In addition, the audiorun patch also takes the incoming audio and adjusts the volume. It is
possible to use music from the input line of the computer’s sound card (for instance an iPod) or
a WAV file stored in the computer.
After this, the two audio channels are sent to the sonification patch where the music is filtered
and processed in order to give the necessary audio feedback. Once the music has been
processed, it is sent to the output line of the computer in order to be played. Moreover, it is also
possible to record the output sound into a WAV file in order to listen to it after the training. The
block is shown in figure 11.
Figure 11- Input and output of the audio feedback. A local music file can be used or the line in of the
sound card can be directly read.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
19
Once the program is loaded, a sound is played as an introduction. In Figure 12 it can be seen
how it is executed.
Figure 12–Sound played once the program is started.
Every time a new test or sonification training is started, all the stored values in the program
must be erased in order to start from scratch. These orders must be centralised in order to be
easily applied throughout the code as it is shown in figure 13. Once a new test is started, all the
values must be cleared. In case of wanting to run a new sonification training, only certain values
will be cleared, and the average ones which set the thresholds will be kept.
Figure 13 - Reset connections. Depending on what wants to be repeated different values will need to be
cleared.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
20
Record data
This patch is used in order to record the incoming messages from Kinect and also it allows
playing them once they have been stored in a .txt file.The messages are edited before storing
them, adding a time stamp to each of them thus fulfilling the format asked by the qlist function.
The messages are stored in the .txt file with the following format:
29.0249 osceleton /joint head 1 0.544386 0.0754202 3.33552;
0 osceleton /joint neck 1 0.547206 0.194634 3.27663;
0 osceleton /joint l_shoulder 1 0.606797 0.203392 3.32487;
0 osceleton /joint l_elbow 1 0.606797 0.366595 3.32487;
0 osceleton /joint l_hand 1 0.606798 0.529799 3.32487;
0 osceleton /joint r_shoulder 1 0.487615 0.185877 3.22838;
0 osceleton /joint r_elbow 1 0.487352 0.349077 3.22998;
0 osceleton /joint r_hand 1 0.503657 0.510319 3.2492;
0 osceleton /joint torso 1 0.545305 0.31582 3.23652;
0 osceleton /joint l_hip 1 0.58308 0.442837 3.22854;
0 osceleton /joint l_knee 1 0.576513 0.675919 3.05872;
0 osceleton /joint l_foot 1 0.559341 0.908464 2.93515;
0 osceleton /joint r_hip 1 0.503727 0.431176 3.1643;
0 osceleton /joint r_knee 1 0.499874 0.673481 3.31297;
0 osceleton /joint r_foot 1 0.494751 0.878755 3.4167;
The first number means the delay with respect the previous sample in milliseconds. Then it is
followed by “osceleton”, in order to be received by the atoms waiting for the symbol
“osceleton”. After this, comes the normal message originally sent by the Kinect.
The qlist function allows reading .txt files chosen from a path, and they can be played using the
time stamps (see figure 14). It also allows stopping and rewinding the set of messages. This part
was already included in the OSCeleton program and it has been really useful in order to test the
program outside the lab. Runners were recorded during the lab tests and those recordings were
later used to test the algorithms before running the proper tests again in the lab.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
21
Figure 14 - Record and play data Pd patch. The OSC messages are changed to fit the format of the qlist
function.
Skeleton and calculations
The draw-skeleton patch is responsible for plotting the skeleton using Gem and for computing
the three different parameters of the running technique that are analysed during the sonification
training. A 3D object is created for each joint by the joint patch, which is plotted in the 3D
space. Moreover, a line is drawn between the different joints using the connection patch. All
these patches that plot the skeleton come from the previously mentioned OSCeleton project. So
the detailed explanations will be focused on the calculations of the running technique
parameters, which are the ones created in this project.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
22
The original OSCeleton code with the three new blocks added is shown in figure 15.
Figure 15 - draw-skeleton Pd patch. It creates the skeleton in the 3D space and also computes the three
analysed parameters of the training.
Vertical displacement
The vertical_disp patch takes values of the joint of the torso in order to analyse the vertical
displacement of the runner. That joint is the closest one to the runner’s centre of gravity;
therefore it is the most reasonable body joint to us for this calculation. First the joints from the
torso are filtered at the input of this patch and then the vertical (Y) component is taken and
filtered with a certain cut-off frequency. The reason for doing this is to reduce some of the
tracking noise produced by the Kinect. Using a low pass filter allows to eliminate the high
frequency movements which are closely related to noise. A cut-off frequency of 5 Hz has been
used for all the tests, but it can be easily changed.
After that, since the values of the coordinates are between 0.0 and 1.0 for the X and Y
components and between 0.0 and 7.0 for the Y component, they have to be de-normalized [17].
The de-normalization functions applied are the following ones:
x = 1280 – x∙2560
y = 960 - y∙1920
z = z∙1280
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
23
All the resulting distances are in millimeters.
The code responsible for doing this is shown in figure 16.
Figure 16 - Filtering and de-normalization for the vertical displacement of the CG
After this, in order to calculate the vertical displacement of the runner, the difference between
the maximum and minimum values of the torso joint must be computed. Since these may vary at
every step, they must be computed in a way that they update themselves automatically.
Moreover, the algorithm must be sure to be looking at an absolute minimum value of a period,
since noise can alter the original shape of the motion and can create relative minimum values
that do not coincide with the absolute one. An example of this can be seen in figure 17, where
the sinus-like shape of the vertical displacement can be seen, but it still has some noise after
being filtered with a 5Hz low pass filter.
Figure 17 - Example of absolute vertical displacement with noise
The solution to this problem is to use a hysteresis window which leaves a margin of time to
check if that value is really an absolute minimum value. For this case, four different values in
different time intervals have been taken and all of them are checked in order to be sure that that
value fulfills the requirements (see figure 18).
0
20
40
60
80
100
120
140
160
1 16 31 46 61 76 91 106 121 136 151 166 181 196 211
Ve
rtic
al d
isp
lace
me
nt
(mm
)
Samples
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
24
Figure 18 - Finding minimum values of the vertical displacement
This method is widely used in this project with both maximum and minimum values, so it will
be presented more in detail by means of pseudo-code. A combination of five different points is
searched in the incoming values. Once a match is found that fulfills the requirements, that value
is taken as a maximum or minimum. In order to make the pseudo-code easier to understand, the
five points shown in figure 19 will be used.
Figure 19 - Peak detection points
For detecting maximum values:
if 1 is smaller than 2 and 2 is smaller than 3 and 4 is smaller than 3
and 5 is smaller than 4
3 is a maximum value
For detecting minimum values:
If 1 is bigger than 2 and 2 is bigger than 3 and 4 is bigger than 3
and 5 is bigger than 4
3 is a minimum value
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
25
Once the minimum value of a period is known, it is subtracted from the rest of the incoming
values so all the positions are relative to the minimum vertical displacement. After this, the
same operation carried in order to find the minimum values is carried for the maximum values.
The incoming data looks like in figure 20.
Figure 20 – Example of relative vertical displacement with noise
Therefore, if the peaks are extracted, the maximum displacement for each period is known.
When a maximum value is found, the value is held until another maximum value is found (see
figure 21).
Figure 21–Peak detection for the vertical displacement
After detecting the maximum values and holding them, the output looks like in Figure 22.
Figure 22 - Example of vertical displacement with step maximum values
0
20
40
60
80
100
1 9
17
25
33
41
49
57
65
73
81
89
97
10
5
11
3
12
1
12
9
13
7
14
5
15
3
16
1
16
9
17
7
18
5
19
3
20
1
20
9
21
7
Ve
rtic
al d
isp
lace
me
nt
(mm
)
Samples
0
20
40
60
80
100
1 81
52
22
93
64
35
05
76
47
17
88
59
29
91
06
11
31
20
12
71
34
14
11
48
15
51
62
16
91
76
18
31
90
19
72
04
21
12
18
Ve
rtic
al d
isp
lace
me
nt
(mm
)
Samples
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
26
After having the maximum values, these still must be treated a bit in order to make them more
reliable. First of all, there are small transitions between the values that are held, so the single
values need to be taken in order to process them in a more comfortable way. In order to do this,
a transition is detected using the following condition shown in Figure 23.
Figure 23 - Transition detection for the vertical displacement
The output for this module is shown in Figure 24.
Figure 24 - Example of vertical displacement with detected transitions
It can be seen that there is still some noise present in the signal, like in the peak of 200 mm. But
since all the values will be averaged using windows in the sonification patch, this kind of errors
will be mitigated. After this, these values are inserted to the following module, which avoids the
insertion of noise into the data by reducing extreme values which differ too much from the rest.
For instance, the two peaks that can be seen in Figure 15 of around 200 mm will be reduced
since they clearly represent noise and not the movement of the runner. The total number of
samples has been reduced since only the transitions are accepted. These samples are processed
with the blocks presented in Figure 25. The aim of these functions is to avoid huge differences
between vertical displacement values. Since noise produces sometimes extreme values, they are
cancelled by producing just a small increase or decrease in the next incoming value, depending
on if the value is higher or lower.
0
50
100
150
200
250
1 7
13
19
25
31
37
43
49
55
61
67
73
79
85
91
97
10
3
10
9
11
5
12
1
12
7
13
3
13
9
14
5
15
1
Ve
rtic
al d
isp
lace
me
nt
(mm
)
Samples
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
27
Figure 25 – Errorcorrector for vertical displacement
Therefore, the values sent to the sonification patch which have already been adjusted will look
like in Figure 26.
Figure 26 - Example of vertical displacement processed for sonification
Step distance
The step_dist patch calculates the distance between the landing foot and the centre of gravity of
the runner and averages the distance for both feet every cycle. First of all, the coordinates for
the right and left feet and the torso are separated. Then, the values for the Z position of the feet
and the Z position of the torso are low pass filtered as always in order to reduce incoming noise
from the Kinect tracking. After this, the values are de-normalized as it has been done previously
with the vertical displacement (see figure 27).
0
20
40
60
80
100
120
140
1 7
13
19
25
31
37
43
49
55
61
67
73
79
85
91
97
10
3
10
9
11
5
12
1
12
7
13
3
13
9
14
5
Ve
rtic
al d
isp
lace
me
nt
(mm
)
Samples
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
28
Figure 27 – Filtering and de-normalization of the step distance data
Initially, the intention was to calculate the step distance by checking when the foot was at a very
close distance to the floor, and then compare the depths of the landing foot and the CG.
However, this proved to be extremely noisy, since the Kinect has a maximum error of around 3-
4 cm, which made that every time the foot swung down, a wide range of values were interpreted
as foot contact. In order to solve this and use a more robust method, the maximum depth of the
feet has been computed. This value of course is slightly superior as the foot contact, since the
foot always swings back a little bit before touching the floor, but it is completely proportional to
the step distance and therefore is still totally useful for this application.
A maximum peak detector has been applied to the difference between the depth of the feet and
the depth of the CG. The maximum depth values of both feet are averaged and sent to the
sonification patch. Moreover, the total average value is computed in order to be used for the test
part and threshold selection. The corresponding code can be seen in figure 28.
Figure 28–Maximum feet depth detection
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
29
The difference between the foot depth and the CG depth with the peak detection applied looks
like in figure 29.
Figure 29 - Example of maximum foot depth detection
The average of both feet and the total average value are computed as it is shown in figure 30.
Figure 30 - Average foot distance values calculation
-800
-600
-400
-200
0
200
1
25
49
73
97
12
1
14
5
16
9
19
3
21
7
24
1
26
5
28
9
31
3
33
7
36
1
38
5
40
9
43
3
45
7
48
1
50
5
52
9
55
3
57
7
60
1
62
5
64
9
Dis
tan
ce (
mm
)
Samples
Step distance - CGStep distance
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
30
Tilt of the torso
The tilting patch calculates the tilt of the upper body with respect the floor. The coordinates of
the torso and neck joints are taken and they are filtered and de-normalized as it has done for the
previous patches (see figure 31).
Figure 31 - Filtering and de-normalization of the torso tilt data
Once the values of the height and depth of the torso and neck joints have been de-normalized
and filtered, the difference in height and depth is computed. These two values represent two
sides of a triangle, and the angle that they form is the angle between the vertical projection from
the floor and the runner’s back (see figure 32). With these two values, the tangent of the angle is
calculated, and from this, the inverse tangent is applied in order to know the resulting angle in
radians. Then this angle is converted to degrees and sent to the sonification patch.
Figure 32 - Tilt angle calculation
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
31
As it has been done in the step distance patch, the overall average angle is computed in order to
use it for running test. The code responsible for this is depicted in figure 33.
Figure 33 - Tilting patch
Sonification
The sonification patch takes the incoming audio signal chosen for the training and changes it
accordingly to the received data produced by the runner’s movements in order to create the
audio feedback. First of all, the patch has four main different sonifications: the step distance, the
vertical displacement, the tilt of the upper body and the overall performance. These four parts
work together sharing certain information and especially they use the same time reference,
which is the frequency of the incoming messages For each one of them a sub-patch has been
created in order to ease the navigation and structure of the program. Moreover, a sub-patch for
the calculation of statistics of the runner’s performance has been included too.
In Figure 34 the structure of the sonification patch is presented. On the left there is a timer based
on the input samples coming from the Kinect. It was developed due to delays and non-stability
of the Pd metronomes when running the whole program. The time is referenced depending on
the incoming Pd messages, which makes the synchronization easier. It is used in order to signal
when to restart the current average value used for the sonification. By default it is set to 100
samples, which means around 4 seconds (since most samples are sampled at 25 Hz). This means
that every 100 samples will be averaged and used as the current value for the sonification,
which helps to reduce the processing time of the program. If the cutting frequency of the filters
were to be changed in average 25 times per second it would require a much higher processing
time than if it is performed around every 4 seconds. The cut-off frequency for the low pass filter
can go from 100 to 20000 Hz. For the high pass filter, it goes from 5 to 5000 Hz.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
32
The tilt and step distance sonification patches are connected to their respective filters and the
vertical displacement one is connected to the twang patch which generates the bouncing sound.
Moreover, if the specific button for each one of the three sonified parameters is not pressed, the
values are blocked before reaching the filter in order not to apply them.
Figure 34 - Sonification patch
Each sub-patch will be analysed separately, focusing on the element that it is sonifying.
Sonification of the vertical displacement
The vd_sonification sub-patch (see figure 35) receives the computed values of the vertical
displacement from the sub-patch inside the draw-skeleton patch and filters the incoming music
file accordingly. On the left side of the patch, the total average value of the vertical
displacement is computed. This value is used for the test that the runner does before starting the
actual training. With this average value a threshold will be set which will determine the training.
When the test button is pressed and the sonification one is not, then the average value is stored
for setting the thresholds. Once the sonification starts and the test button is not pressed and the
sonification one is, the average value is not modified. Moreover, every time that the retest
button or another test is run, the average value is erased in order to compute it once again from
scratch.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
33
Figure 35–Vertical displacement sonification patch. The vertical displacement values are averaged and
sent to the twang patch.
On the right side of the patch, the incoming values are averaged every 100 samples using the
time flags sent from the sonification patch. This averaged value is held until the next flag
comes, and it is sent to a transition detector.
It is actually needed because the transitions between average values are not instantaneous, so
there are some values in between which need to be ignored.
Besides this, a condition is also set for the value to be sent to the twang patch only if the
sonification button is pressed, the test button is not pressed and the sonification of the vertical
displacement has been chosen on the interface. If all these conditions are fulfilled, the value is
sent to the twang patch which will produce the bouncing sound feedback as it is explained in the
next section. Otherwise, a zero value is sent which will be stopped by the spigot box.
Twang
This patch has been taken from the code examples from the “Designing sound” textbook,
included in a compilation in The MIT Press website [18]. When an inlet is banged, a boingy
sound is produced. The volume, frequency and vibrato of it can be adjusted through different
inputs. Since this patch has not been developed for this project, all further explanations will be
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
34
avoided and they will only be focused on explaining how the incoming vertical displacement
values are scaled in order to produce the audio feedback.
First of all, the frequency of the boingy sound is adjusted depending on the values of the vertical
displacement. The higher the vertical displacement is the higher will be the frequency of the
sound that the runner will hear. This helps to intensify the feeling that he is bouncing too much.
In order to do this, a function has been created in order to scale the frequency.
This function takes into account the threshold that has been set for the training. A constant value
is multiplied by the current vertical displacement value received and the threshold. This will
mean that if the runner goes over that threshold the constant value will be increased, and if he
goes under it, the constant value will be reduced. For values different to zero, the frequency will
be determined by formula 2.
𝑓 = 600 𝑣𝑑
𝑣𝑑𝑡 (2)
Where 𝑣𝑑is the current vertical displacement and 𝑣𝑑𝑡 is the threshold value for the vertical
displacement.
On the top left of Figure 36, it can be seen that when a message arrives to the inlet, it triggers
the boingy sound.
Figure 36 - Frequency adjustment on the twang patch
The same procedure done with the frequency is applied to the volume. Before all the samples
reach the sound output, they are multiplied by a value between 0 and 1 in order to adjust the
volume. In this case, the function is divided into two different ranges of values. If the current
vertical displacement value is over the specified threshold, the sound will be played at
maximum volume (1). For the values lower than the threshold, a function is applied where the
bigger is the difference between these two values, the smaller the volume will get. This is done
by dividing the input value by the subtraction of itself with the threshold. This kind of function
helps to make the transition over the threshold really obvious for the runner. As soon as he gets
close to the threshold the volume will increase a lot and vice versa. If more intermediate values
were used, it could be quite confusing for the runner, since he is supposed to listen to many
different sorts of audio feedback at the same time. Therefore, if the response of the audio
feedback becomes obvious and easy to understand, the chances for it to be useful to the runner
will clearly increase.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
35
Formula 3 determines the volume of the auditory icon.
𝑣𝑜𝑙𝑢𝑚𝑒 =
𝑣𝑑𝑡 − 𝑣𝑑 < 0 ; 1
𝑣𝑑𝑡 − 𝑣𝑑 > 0 ; 𝑣𝑑
𝑣𝑑𝑡 − 𝑣𝑑 ∙ 200 (3)
Where 𝑣𝑑 is the current vertical displacement and 𝑣𝑑𝑡 is the threshold set for the vertical
displacement.
The part of the program that applies this formula is shown in figure 37.
Figure 37 - Volume adjustment of the twang patch
With these two parameters sonified, the boingy sound is played over the music file in a periodic
fashion, aiding the runner to understand his vertical displacement.
Sonification of the tilt
The tilt of the upper body is sonified by using a low pass filter that cuts the high frequencies of
the song when the runner does not tilt his body enough compared to the specified threshold.
The tilt sonification averages the incoming values in a periodic way as it is done with the
vertical displacement sonification patch. For every averaged value, a function is applied to it in
order to turn the angle in degrees into a value in hertz which will change the cutting frequency
of the filter. This function uses the same idea as for the one used for the volume of the boingy
sound. If the angle is over the threshold, the cutting frequency is set to 20000 Hz, which is the
maximum frequency that a human ear can hear. However, if the angle is under the threshold, the
maximum frequency (20000 Hz) will be reduced depending on the difference between the
current angle and the specified threshold. This type of function will help to make the transitions
really abrupt. Therefore, as soon as the runner has an angle under the threshold, he will hear that
the music changes dramatically.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
36
In 4 the formula responsible for converting the angle into a cutting frequency for the filter can
be seen.
𝑓𝑐 =
𝛼𝑡 − 𝛼 < 0 ; 20 ∙ 103
𝛼𝑡 − 𝛼 > 0 ;2000
𝛼𝑡 − 𝛼
(4)
Where 𝛼 is the current tilt angle of the body and 𝛼𝑡 is the threshold for the tilt angle of the
body.
The sonification patch for the tilt can be seen in figure 38.
Figure 38 - Tilt sonification patch. The tilt angle determines the cutting frequency of the low pass filter.
Sonification of the step distance
The step distance between the foot and the projection of the centre of gravity is sonified using a
high pass filter that cuts the low frequencies of the song whenever the runner performs over the
specified threshold.
Once again, for this sub-patch (see figure 39) the values are also averaged periodically, but this
time a counter has been used which helps to produce a new average value when four values of
foot distance have been received. However, the applied function to turn the step distance values
into hertz works in the exact same way too.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
37
Figure 39 - Step distance sonification patch
In 5 the formula that converts the distance in mm from the step into the cutting frequency of the
filter is shown.
𝑓𝑐 =
𝑠𝑑𝑡 − 𝑠𝑑 + 10 < 0 ; 5 ∙ 103
𝑠𝑑𝑡 − 𝑠𝑑 + 10 > 0 ;3 ∙ 103
𝑠𝑑𝑡 − 𝑠𝑑 + 10
(5)
Where 𝑠𝑑 is the current step distance and 𝑠𝑑𝑡 is the threshold for step distance of the training.
If the step distance surpasses the threshold for more than 1 cm, the cutting frequency will be set
to 5000 Hz, which is the maximum frequency for this filter. This will mean that all frequencies
under 5000 Hz (mid-low and low frequencies) will be removed from the song. Once the step
distance goes under the threshold, a constant value is divided by the difference between this
current value and the threshold. This will produce the same effect as for the other two
sonification patches. As soon as the runner goes under the threshold, the sound will improve
dramatically. However, if he gets close to it, the sound will be quickly filtered and only high
frequencies will remain on the song.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
38
Overall
The overall patch (see figure 40) is responsible for telling the runner if he has been performing
correctly according to the specified threshold for a certain amount of time. Depending on which
of the three elements have been chosen for the sonification training, if the sonification button is
pressed and the chosen elements are fulfilling the specified thresholds, a counting will start. If
10000 samples are received (a little bit more than 10 seconds) under these conditions, a sound
will be played over the music to tell the runner than he is performing in the correct way.
Figure 40 - Overall performance sonification patch. If all the required conditions match for a while, an
auditory icon is played to tell the runner to keep on doing like that.
This is intended to ease the training for the runner. Once the runner is performing in the right
way, he will hear the music correctly with no artefacts or anomalies; but after doing it right for a
while, this auditory icon will tell him that he is on the right track and he just must keep up the
current pose to eventually memorise it.
Stats
This sub-patch calculates the percentage of time that the runner is fulfilling the desired
thresholds. However, an error threshold can be set in order to take into account also values that
approach the desired threshold. For example, if an error threshold of 10% is chosen, even if the
runner performs 10% more over the chosen sonification threshold it still count as if he were
performing correctly. In order to do it, the current values of the vertical displacement, the step
distance and the tilt are compared to the specified thresholds, and if they fulfil them they are
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
39
counted as successful samples. Then, these samples are divided over the total amount of
samples, thus having the ratio of correct samples.
All these calculations are sent to the results patch, where they are displayed in a more intuitive
way. The patch is shown in figure 41.
Figure 41 - Statistics patch
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
40
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
41
Results In this chapter, the different experiments and tests that have been performed with runners will
be exposed, as well as the set-up used. All the tests have been done in the Sport Physiology
Laboratory in Elite Performance Centre of the Swedish Sports Confederation, in Lidingö
(Stockholm).
Set-up
For all the tests, the following set-up has been used (also shown in figure 42):
The Kinect is mounted on a tripod with the help of a tripod mount. It must be perfectly
aligned to the runner’s axis. This means that it must be completely horizontal with
respect the floor and forming a 90º angle when facing the treadmill. This will assure
that the coordinates fulfil the program’s requirements. Moreover, it must be placed at
the rear of the treadmill, at a distance of approximately 2 meters from the runner (the
runner will move forward and backward while running anyway). However, for a totally
correct placement it is better to check first the RGB image of the Kinect in order to be
sure that the runner fits completely in the screen.
The Kinect is connected to a power source and to a laptop by means of a USB cable.
The laptop has stereo 40W speakers connected to it and it is placed on a table next to
the Kinect. The speakers are aimed at the runner at a considerably high volume in order
to be heard over the treadmill noise.
The room where the tests are run must be lit with artificial light, and sunlight from the
windows must be avoided. Covering the windows with blinds is the best solution [19].
Figure 42 - Test in the Sport Physiology Laboratory, from the Swedish Sports Confederation
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
42
Once everything is set up, the Pd program can be started by opening the Interface patch. After
this, it is recommended to press the skeleton button and then to open the OSCeleton application
in order to start tracking the runner. To do so, he must stand in front of the Kinect facing it, in
the same spot he will run. Moreover, he must adopt the psi posed as it has been previously
explained until OSCeleton says that the user is being tracked correctly. In order to check that the
tracking works fine, the user’s skeleton is shown in the skeleton window.
After this, the runner must start running in the same way as he normally does, in order to
analyse his current technique. However, it is highly recommendable to start running with the
legs slightly more separated than normal for the Kinect to track the movements better. After 5-
10 seconds of proper tracking, the runner can run as always and the Kinect will still track the
legs correctly. Once the runner is already running, the test button must be pressed and all
average values for the three sonified parameters will be computed. After a certain amount of
time (determined by the person who runs the test) which should be enough to have a meaningful
set of values, the test button can be pressed again in order to finish the test. Then, the average
values are displayed on the interface; and using a slider, they can be modified in order to set the
desired thresholds for the three analysed parameters that the runner should achieve during the
training.
Once the test part has concluded, the runner can start the training and he will hear the audio
feedback that will react depending on his position compared to the desired thresholds. The set-
up looks like in figure 43.
Figure 43 - Kinect tracking a runner(edited from http://www.truefitness.com/galleries/37/cs550-
treadmill-images)
During the training the runner should try to focus on making the music sound well. This means
that the audio feedback will tell him that he is running in the desired way when he will not hear
any artefacts or changes in the music. In order to finish the training, the sonification button must
be pressed again, which will end the audio feedback and will stop the computation of results.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
43
After this, the results button can be pressed, which will show the percentage of time the runner
has been complying with every one of the analysed elements. Furthermore, the overall
percentage of time that he has been fulfilling the three of them at the same time is also
displayed.
Perception tests
A series of tests were carried, for checking if the runner was able to understand the audio
feedback to the movements of his body and get to know where the threshold was in order to
change the sound in a controlled manner. The runners were asked to be able to go over and
under the threshold whenever they wanted in order to change the music and auditory icons at
will. The goal was not to actually test if Kinect Audio-Runner was able to improve someone’s
technique but to see if it could successfully interact with a runner. Each one of the three sonified
elements were tested separately and then in a combined way. The vertical displacement and the
torso tilt were tested by three different casual runners and the step distance by one (due to some
problems on the implementation on the early tests). Each one of the tests will be commented
separately.
Vertical displacement perception
For this test, the runners were asked to run for a while on the treadmill in order to measure their
average vertical displacement using the test mode of the program. Then, the threshold was set
slightly underneath their average vertical displacement and they were asked to sense at which
point the threshold was and to be able to run for as long as they wanted both under and over the
threshold. It did not prove difficult for the runners, except for some very specific situations
where the tracking did not work as it was supposed to. At the beginning, they had to adapt to the
fact that the boingy sound takes some time to adapt to the current vertical displacement of the
runner, since the values are averaged in order to make it more robust. However, all of them
were very pleased with the way it worked and were perfectly capable of controlling the boingy
sound at will. An example of a recorded test is shown in the demonstration video.
The parameters and thresholds used were:
Table 1 - Vertical displacement test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
8 km/h 10 cm - -
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
44
Tilt perception
The tests for the tilt angle were the most successful ones. The accuracy of the reaction of the
sound was really good and the runners had absolutely no problems when trying to make the
music sound well or not.
As with the vertical displacement, the reaction was not extremely fast, since the values were
averaged and it took a few seconds for the sound to adapt, but that did not prove as a problem.
An important thing to mention is that the runners had to be told not to tilt only their upper body;
the tilt should come from the ankle [12]. If a large angle was set, the only way to achieve it
correctly was increasing the speed. At low speeds it was not possible for the runners to tilt that
much in a correct way, they could only do it by tilting only the torso. An example of a recorded
test can also be found in the demonstration video.
The parameters and thresholds used were:
Table 2 - Tilt test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
8 km/h - - 15º
Step distance perception
In order to analyse the sonification of the step distance, two different tests were carried by the
third runner. First, he had to walk on the treadmill trying to make strides of different lengths. He
was asked to be able to go over or under the set threshold whenever he wanted. The results
turned to be successful, and the runner was able to make the music change (cutting the high
frequencies) whenever he made long strides over the determined threshold (6cm). Compared to
the other two sonifications, it took a slightly longer time for the audio feedback to adapt due to
averaging computations of the step distances. However, it hardly ever took more than 3-4
seconds to adapt, which did not produce any troubles to the runner. After the walking test, he
was asked to do the same but running.
For the running case, the distance between the vertical projection of the CG and the landing foot
was much smaller than for the walking case. This is due to the fact that the runner tilts forward
in order to run, thus making the distance to the landing foot much shorter. When the runner was
asked to walk normally the step distance was quite similar to the when he ran, but he was able
to make much longer strides while walking which could easily surpass any threshold used when
running. Therefore, the thresholds for the step distance were different than the ones from the
walking test as it can be seen in the tables 3 and 4.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
45
Table 3 - Step distance walking test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
4,5 km/h - 6 cm -
Table 4 - Step distance running test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
9 km/h - 4 cm -
Combined perception: Vertical displacement + tilt
This test was run after the runners had tried both of the sonifications individually. All the three
runners coincided in the way of facing it and they tried to deal with the sonified elements one by
one. Since it is quite hard to tilt with a higher vertical displacement, they first tried to reduce
their vertical displacement by making the boingy sound disappear while the music could not be
heard correctly. After finding the threshold and getting to know at which maximum vertical
displacement they were allowed to run, they focused on tilting forward until the music became
normal and the high frequencies could be heard again. Again, like for the case of the individual
tests, the thresholds were set slightly underneath their average values acquired during the test
part. The tilt angle of the runners was set at 15º, an intermediate angle that made it easy for
them to make the music fade if they run quite straight and when they tilted and ran correctly the
music started to sound correctly really fast. For the vertical displacement, the values changed a
little bit more. For two runners, the vertical displacement was set at 9 cm. If they intended to run
with a notorious bouncing, the boingy sound appeared easily, but if they focused on maintaining
a low vertical displacement without becoming awkward they managed really easily to make the
boingy sound disappear. A recorded example of this type of test is shown in the demonstration
video too.
The parameters and thresholds used were:
Table 5 - VD + tilt test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
10 km/h 9 cm - 15º
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
46
Combined perception: Vertical displacement + tilt + step distance
After successfully testing the combination of the vertical displacement and the tilt, all the three
parameters where analysed at the same time. For that, the third runner was asked to wilfully
produce any combination of parameters (being over or under the threshold) in order to check if
the audio feedback responded correctly and adjusted to the runner’s pose. Once more, it proved
easier for him to deal with them one at a time.
In the end, he managed to control the audio as he wanted, despite that sometimes the audio
feedback reacted with a delay of around 3-4 seconds. Some combinations proved to be quite
difficult and awkward to achieve, since they were not natural positions that a runner would
attempt, but still he managed to produce them. The results can be seen in the video too, where
the whole experiment was recorded.
The parameters used for this experiment can be seen in table 47.
Table 6 - Complete test parameters
Treadmill speed Max.vertical displacement Max. step distance Min. torso tilt
9 km/h 10 cm 5 cm 15º
Video
As it has been previously mentioned, a video which shows the basic functionalities of the
program, as well as the different steps to follow and the perception tests has been developed.
Since the project contains a high amount of audio-visual content, it is much more intuitive and
logical to show the results through a video rather than just describing how the music or the
training sound or look like. The video has been uploaded to Vimeo for everybody to check. The
url to the video is the following one:
https://vimeo.com/42772997
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
47
Conclusions and further work
Conclusions
The tests with real runners confirmed that the initial goals of the project have been achieved. A
program has been developed which allows runners to analyse their current technique and change
certain elements (the vertical displacement of the CG, the tilt of the torso and the distance
between the projection of the CG to the floor and the landing foot). The feedback to the runner
is provided with audio consisting in both real-time filtered music and auditory icons. The runner
is able to identify the mistakes on his/her pose by just listening to the audio feedback. The
response from the runners has been good and they have stated that they enjoyed the training as
well as the whole concept of the project.
The response of the audio feedback has proved to be robust in most cases. The code is able to
deal with different situations and noisy data from the Kinect tracking. However, still some
problems may occur from time to time due to the difficulties for Kinect to track the runner from
behind. Kinect was initially designed to track a person from the front, and the user is supposed
to perform clear slow movements towards the camera. On the other hand, for this project, the
runner had his arms and legs very close to each other and to the body. The legs move quite fast
so sometimes it becomes quite hard for the Kinect to correctly track them. The best solution to
this problem is to reboot the tracking software (OSCeleton) and start tracking the user again.
Another “trick” to make the tracking more robust is to ask the runner to initially run with the
legs slightly more separated than normal and after a few seconds, when the legs have been
tracked for a while, start running normally again. Nonetheless, it is just a matter of time that a
better version of Kinect is released, which would perfectly work with the developed software
from this project.
Further work
There is an aspect of this project that would be really interesting to work on in the future, which
has been left out due to time reasons. The program should be tested with a subject who had a
real running problem: analyse the technique, understand which elements of the running gait
could be improved, and set a target position as a goal to achieve. Then, plan a set of trainings
using this software and carefully analyse the results and changes in his technique. Finally, check
again in an outdoor court in order to see if the treadmill training has stuck in the runner’s mind
and he is capable of applying the acquired knowledge without receiving the audio feedback.
Moreover, it would be interesting to analyse more elements in the training. For instance, try to
find a proper way to track the arms and analyse their swing, and how it synchronises with the
movement of the legs. Actually, many other elements could be added to this program in order to
make the technique analysis even more complete.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
48
It would be a matter of finding a way to correctly analyse them by using a robust tracking, and
being able to give them an audio feedback that does not interfere with the already developed
ones. Notwithstanding, a too high amount of information could become too vexing to handle for
the runner; especially if he is asked to interpret many different audio feedbacks at the same
time. On the other hand, it would be interesting to carry out a broader study on how audio
feedback is perceived from a large number of runners, and see if it could be improved in some
aspects in order to adapt it to the taste of different people.
Another remarkable concept that could be easily developed from this project would be to use
Kinect Audio-Runner with a different aim, such as rehabilitation or aiding old people to walk.
This could be easily implemented by investigating which are the interesting parameters for the
user; and use the current ones plus any new one which might be needed. For instance, for old
people who need to train to walk with longer steps, the step distance sonification could be used,
as it has already been shown in the results and the video.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
49
Bibliography
[1] Thomas Hermann, Andy Hunt& John G. Neuhoff. 2011. The Sonification Handbook.
Logos Verlag. ISBN 978-3-8325-2819-5.
[2] Ross Tucker. 2007. Running Technique part I: The philosophy of how we run.
http://www.sportsscientists.com/2007/09/running-technique-part-i-philosophy-of.html.
Latest visit May 2012.
[3] Ross Tucker. 2007. Running Technique part II: Biomechanics of running and Pose.
http://www.sportsscientists.com/2007/09/running-technique-part-ii-biomechanics.html.
Latest visit May 2012.
[4] Eriksson, M., & Bresin, R. (2010). Improving running mechanics by use of interactive
sonification. In Bresin, R., Hermann, T., & Hunt, A. (Eds.), Proceedings of the Interaction
Sonification workshop (ISon) 2010 (pp. 95-98). Stockholm, Sweden: KTH Royal Institute
of Technology.
[5] GaëlDubus. 2011. Evaluation of four models for the sonification of elite rowing. J
Multimodal User Interfaces (2012) 5:143–156.
[6] Stephen Barrass, Nina Schaffert & Tim Barrass. Probing Preferences between Six Designs
of Interactive Sonifications for Recreational Sports, Health and Fitness.In Bresin, R.,
Hermann, T. & Hunt, A. (Eds.), Proceedings of the Interaction Sonification workshop
(ISon) 2010 (pp. 23-29). Stockholm, Sweden: KTH Royal Institute of Technology.
[7] Varni, G., Dubus, G., Oksanen, S., Volpe, G., Fabiani, M., Bresin, R., Välimäki, V.,
&Kleimola, J. (2012).Interactive sonification of synchronisation of motoric behaviour in
social active listening of music with mobile devices. Journal on Multimodal User
Interfaces, 5(3), 157-173.
[8] Branko Skof & Stanko Stuhec. Kinematic analysis of Jolanda Ceplak’s running technique.
http://www.coachr.org/kinematic_analysis_of_jolanda_ce.htm. Last visit April 2012.
[9] Jessica Gonowon. 2007. The Physics of Efficient Running. http://ffden-
2.phys.uaf.edu/212_spring2007.web.dir/jessica_gonowon/gonowon_page1.html. Last
visited April 2012.
[10] Tom F. Novacheck. 1997. The biomechanics of running. Gait and Posture 7 (1998) 77–
95.
[11] Steve Magness. 2010. How to run: running with proper biomechanics.
http://www.scienceofrunning.com/2010/08/how-to-run-running-with-proper.html. Last
visited March 2012.
[12] Okemos Playmakers running store. 2009. Good Form Running.
http://www.youtube.com/watch?v=Tx6x2cD6Y8Q&list=PLAF9B61E2EB3A9230&index
=1&feature=plpp_video. Last visited May 2012.
[13] Evelyne Kohler et al. 2002.Hearing sounds, understanding actions: Action representation
in mirror neurons. Science 297, 846 (2002). DOI: 10.1126/science.1070311.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
50
[14] Sensebloom. 2011. OSCeleton. https://github.com/Sensebloom/OSCeleton#readme. Last
visited May 2012.
[15] Pure data community website.http://puredata.info/. Last visited May 2012.
[16] Wikipedia. 2012. Kinect.http://en.wikipedia.org/wiki/Kinect. Last visited April 2012.
[17] OSCeleton Google Group. 2011. Kinect distances in mm.
http://groups.google.com/group/osceleton/browse_thread/thread/9cdcd25584a4be54/f8027f
65e08199cb?lnk=gst&q=distance+mm#f8027f65e08199cb. Last visited March 2012.
[18] Andy Farnell. Code examples for “Designing sound” textbook. 2010. MIT Press.
http://mitpress.mit.edu/designingsound/boing.asp. Last visited May 20120.
[19] Kinect support.Lighting. http://support.xbox.com/en-GB/kinect/setup-and-
playspace/lighting. Last visited May 2012.
[20] Brekel.Brekel Kinect. http://www.brekel.com/?page_id=160. Last visited June 2012.
[21] Nike. Introducing Nike+ Kinect Training. http://nikeinc.com/news/introducing-nike-
kinect-training. Last visited June 2012.
Kinect Audio-Runner: Audio feedback for improving performance in long-distance running
Jordi Bolíbar
51
Appendix
List of used terms
CG: Centre of gravity
EMG: Electromyography
OSC: Open Sound Control
Pd: Pure data
SDK: Software Development Kit
VD: Vertical displacement
WAV: Waveform Audio File Format
TRITA-CSC-E 2012:062 ISRN-KTH/CSC/E--12/062-SE
ISSN-1653-5715
www.kth.se