The Science of Fun

The Science of FunData-driven Game DevelopmentDr. Alexandra urcan, Deep Silver GamesLab ManagerRuan Pearce-Authers, Senior Online Services Engineer

About us

Alex

Ruan

Dambuster Studios

We make video games.

We are one of Deep Silvers development studios, along with Volition (USA) and Fishlabs (Germany).

You can find us on Canal Street, near Broadmarsh.

Homefront: The Revolution - is an open-world first person shooter where you must lead the Resistance movement in guerrilla warfare against a superior military force.

Homefront "Freedom fighters" trailer

6

Data-driven game development

What is data-driven game development?Including data in the game development processThis data can be generated from playtests with both internal and external playersIt can be used to make more informed decisions during game developmentAt Dambuster, we collect data in our research lab, Deep Silver GamesLab

GamesLab: research and playtesting facility

Fun is the ultimate goal of the game development process

Fun is a subjective experience, and for each player it could come from something else:easy game,hard game,collaborative,individual,etc.

What is fun?

10

Million dollar question: How do you measure fun?

UX

UX Research Traditionally, its the process of understanding the impact of design (e.g. websites) on an audienceExample methods:InterviewsSurveysObservationsFocus groupsHeuristic evaluation / expert reviewAdapted to video games, but it doesnt scale well and its not easy to automatePro: can answer the why questions

Telemetry pipeline

What is telemetry?Data collected from running gamesUsed for both development and retail buildsGives us quantitative behavioural data

Example telemetry:Discrete events player started a level, killed an enemy, died, etcContinuous data streams player position, health, etc

Sending telemetryGame code is instrumented to send events appropriatelyAs events are emitted, we send them to a remote serverEvents are sent in the exact order theyre emittedThis is very useful later on!Think of this as a stream of dataThe first event in a telemetry stream is purely metadataPlatform, user ID, etc

Telemetry event examples

{ "type": "telem_start", "t": 1484132366, "v": 1, "data": { "Account": { "User": "UKPC4020", "Platform": "editor" }, "Build": { "Game": "OnlineSample", "Type": "dev" }, "Hardware": { "OS": "windows" } }}

Telemetry event examples{ "type": "player_spawn", "t": 1484132366, "v": 1, "data": { "Name": "PlayerBlueprint_C_0", "Pos": { "X": 326.24, "Y": 3.55, "Z": 268.38 } }}

Telemetry event busA lightweight HTTPS/WebSocket server receives all these eventsTheyre forwarded into Kafka, a log databaseSupports many consumers, with non-destructive readsWe retain data based on storage capacity and lifetimesCan replay events if we find processing bugs, or need to extract new data

Telemetry stream processingWere now all set to start making use of the data!Processors listen for one or more specific event typesThey have local memory to combine eventsCan use RAM directly, or a Redis instanceOutput summarised/combined events to:Separate queues for further processingPermanent data storage

Example telemetry stream

Example telemetry processor in JavaScript

Permanent telemetry storageWe use MongoDB as a defaultRich aggregation framework for queriesScales well from an ops perspectiveBut theres no one-size-fits-all solution to data storageSpecial cases include: time-series dataThese databases are then used by analysis scripts and dashboards

R is a functional language and environment for statistical computing and graphics.

https://www.r-project.org/Telemetry analysis

Telemetry architecture recap

Telemetry use cases

Telemetry for designers- Where do players die? -

Telemetry for designers- Can the player see the enemy whos shooting them? -

Player cant see the enemy:not_in_FoVrendered / not_rendered

Enemy is occluded:in_FoVnot_rendered

Player could be seeing the enemy:in_FoVrendered

28

Telemetry for designers- Do players become more accurate shooters over time? -

lm(accuracy ~ time) => Model explains 0.004% of variance. Time does not predict accuracy on its own. Other predictors?

29

Player accuracy doesnt improve over time.

Reasons why that might be:Not enough practice?Too long delay between play sessions?Other variables might be affecting accuracy (e.g. ammo count: when players have loads of ammo they can afford to not be as accurate as when they are running out of ammo)?

30

Telemetry for detecting subtle issues

Telemetry for business intelligence- Timeline of mission popularity -

Telemetry for online services- What is the relationship between ping and distance? -Distance is only weakly correlated with session ping (correlation coefficient = 0.28).With every 1 km increase in distance, the average ping only increases with 0.01 ms.

The value of the ping does not change with distance. So, a good matchmaking algorithm should not depend solely on geographical distance.

Biometrics introduction

Why use biometrics at all?It nicely complements the quantitative behavioural data provided by telemetry and the qualitative player opinions provided by UX methodsIts the only tool we have that measures unconscious player behaviourCant be faked player or researcher subjectivity does not affect it

Eye-trackingTraditionally used in academic research to study human visual attentionAdopted more and more in the games industry because it can answer questions about player behaviour that cannot be answered through any other means It is non-invasive eye-tracking hardware snaps onto the PC monitor and tracks eye movements from a distance - the player will not be inconvenienced in any way

Skin conductanceGalvanic skin response (GSR) aka Skin conductance (SC)aka Electrodermal activity (EDA)

Measures changes in skin electrical conductivityReliable indicator of emotional stateNot under conscious control

StimulusActivate bodys autonomic nervous systemSweatSkin moisture content increasesSkins electrical conductivity increasesEven if a person is not visibly sweating, sweat is still building up underneath the skin, which means that GSR can still be measured.

How GSR works

ValenceArousal

Can be measured with facial recognition.Can be measured with skin conductance.

Emotions

Biometrics implementation

Eye-tracking technical specsWorks by reflecting near-infrared light off the corneaThe hardware gives us (x,y) screen coordinates

This raw data is entirely useless without game world contextCant be included in telemetry directlyLets see what can actually be done with this!

Eye-tracking telemetry (3D)This tells us which objects in the world are being looked atDeproject those (x,y) coordinates into a world position and directionCast a sphere through the world, see what we hitCustomisable radius for accuracyWhen the hit object changes, log a started_looking event to telemetry with object details

Eye-tracking debug visualisation

Eye-tracking telemetry (2D)This tells us how the player uses the UI/HUDThe game needs to provide a 2D map of all interesting elementse.g. Circle for a minimap, small rectangle for a health barPerform intersection testing on these elements, log events when this changes

GSR technical specsGSR data is recorded in microSiemens Our sensors operate at 10hz (we get one reading every 100ms)Roughly speaking: sweat means more current goes through

The data is just a single floating-point valueSuitable for direct inclusion in the telemetry streamMeans we can check response to any in-game events

Pacing volumesGSR data cant be used in absolute termsDesigners add markup objects to the world, called pacing volumesThese define expectations for emotional intensity at given areas in the gamee.g. calm, moderate, intenseWe can then check how well our expectations match up with reality

GSR analysis algorithmRaw GSR data needs to be analysed before it can be interpreted

An algorithm in R outputs a series of significant peaks coupled with the events that caused them

Ledalab algorithm in Matlab (http://www.ledalab.de/) by Benedek & Kaernbach (2010). We translated it to R.

Biometrics use cases

Example eye-tracking use case

Example GSR use case

Other GSR use casesCan be used to answer questions like:Are various areas of the game more or less engaging for the player?Does dying elicit a similar emotional response as wining the mission?Do players panic when they run out of ammo?Does removing certain game elements make the game less engaging/more boring?

Dark Souls 3Homefront: The Revolution

GSR overlaid on game-play: GSR tracking Dark Souls 3

52

Tools recap

Get involved!Sign up for playtestshttps://gameslab.dsdambuster.com/

Contact us @AleTurcan and @returnString