Automating Content Analysis of Video Games

T. Bullen and M. KatchabawDepartment of Computer ScienceThe University of Western Ontario

N. Dyer-WithefordFaculty of Information and Media StudiesThe University of Western Ontario

Outline

1. Introduction2. Automating Content Analysis 3. Prototype Implementation4. Experiences and Discussion5. Concluding Remarks

Introduction

Content analyses of video games involve coding, enumerating, and statistically analyzing various elements and characteristics of games– This includes violence, offensive language, sexual

content, gender and racial inclusiveness, and so on While content analysis has its limitations, it is

invaluable in providing a quantitative assessment of games to go with more qualitative analyses– It can be an important tool to many people dealing

with various aspects of games and the games industry

Introduction

Problems arise, however, when one tries to apply traditional content analysis processes, for example from film or television, to games– Processes are manual and are consequently time

consuming and labour-intensive– This tends to result in significantly reduced play times

or limiting analyses to only a very few games– Traditional analyses also tend not to consider

interactivity and non-linearity that occurs in games– The rapid rate at which games are released and the

industry evolves makes keeping up difficult

Introduction

In the end, with the limited time and resources often available, it is exceedingly difficult to perform thorough content analyses on even areasonable portion of games

To address these problems, our current work examines automating the process of content analysis for video games– Through automation, it is hoped that time and

resources can be used more efficiently andeffectively to permit more thorough studies

Automating Content Analysis

To automate content analysis, we take advantage of the fact that, unlike other forms of media, video games are software executing onsome kind of computing device

This can permit two forms of automation:– Partial automation: software executing along side the

game monitors game execution and collects and reports the data normally collected manually

– Full automation: further software elements take the role of the player and generate gameplay experiences without the need for a human player

Automating Content Analysis:Instrumentation

Game ApplicationCode

GameObject 1

GameObject 2

GameObject 3

GameObject n

Sensor 1

Sensor 2

Sensor n

Coordinator

Prototype Implementation

As a proof of concept, we have used our instrumentation framework to instrument Epic’s Unreal Engine to enable automated content analyses of Unreal-based games– Unreal is a popular engine amongst professional and

amateur developers, providing numerous possible games for content analysis experiments

Instrumentation was implemented using the UnrealScript language– Source level access to the engine was not available

Prototype Implementation

Game Info

Game Rules

Mutators

GameObjectSensor

Coordinator

Prototype Implementation

Sensors have been developed to collect a wide variety of data useful for content analyses:– Death of characters, weapon use by characters, use

of offensive language, gender and racial diversity in characters, and a variety of other game statistics

– Data can be reported throughout a game or only as summaries at the end of games

Sensors can be configured at run-time to tailor the data collected to the needs of the content analyses being conducted

Experiences and Discussion

To validate our prototype implementation, we conducted several content analysis experiments on Unreal Tournament 2004– This game is one of the flagship titles driven

by the Unreal Engine It is a fairly popular First Person Shooter that

has numerous gameplay options– Several different game types and rule sets– Individual and team-based games– Single player, multiplayer, and spectator modes

Experiences and Discussion:Deathmatch Game

------------Level Info------------Level Name: RrajigarGame Type: DeathMatchTotal Players: 14AI Players: 13Human Players: 1Spectators: 0Male Players: 13Female Players: 1Level Loaded: 0:26:45Game Finished: 0:30:29Gameplay Elapsed (Seconds): 240.88AI Dialog: 28Human Dialog: 27--------------------------------------------All Player Stats--------Total Deaths: 47Total Suicides: 1Total Kills: 46Total AI Deaths: 46Total Human Deaths: 1Total Deaths Caused By AIs: 22Total Deaths Caused By Humans: 25Total Female Deaths: 12

Total Male Deaths: 35Total Deaths Caused By Females: 6Total Deaths Caused By Males: 41------------------------------------------Local Player Stats--------Player Deaths: 1Player Suicides: 0Player Killed: 1Deaths Caused By Player: 25Player Killed By AI: 1Player Killed By Human: 0Player Killed By Male: 1Player Killed By Female: 0AI Deaths Caused By Player: 25Human Deaths Caused By Player: 0Female Deaths Caused By Player: 7Male Deaths Caused By Player: 18Deaths Witnessed By Player: 29-----------------------------------------------Expletives-----------ass: 2----------------------------------

Experiences and Discussion:Onslaught Game

------------Level Info------------Level Name: Arctic StrongholdGame Type: OnslaughtTotal Players: 12AI Players: 11Human Players: 1Spectators: 0Male Players: 9Female Players: 3Level Loaded: 23:16:43Game Finished: 23:32:3Gameplay Elapsed (Seconds): 964.18AI Dialog: 216Human Dialog: 31--------------------------------------------All Player Stats--------Total Deaths: 142Total Suicides: 5Total Kills: 137Total AI Deaths: 138Total Human Deaths: 4Total Deaths Caused By AIs: 119Total Deaths Caused By Humans: 23Total Female Deaths: 26

Total Male Deaths: 116Total Deaths Caused By Females: 8Total Deaths Caused By Males: 134------------------------------------------Local Player Stats--------Player Deaths: 4Player Killed: 4Deaths Caused By Player: 23Player Killed By AI: 4Player Killed By Human: 0Player Killed By Male: 4Player Killed By Female: 0AI Deaths Caused By Player: 23Human Deaths Caused By Player: 0Female Deaths Caused By Player: 9Male Deaths Caused By Player: 14Deaths Witnessed By Player: 43----------------------------------------------Team Info-------------Female Allies: 1Male Allies: 4Friendly Fire Deaths: 5Allies Killed By Player: 0Player Killed By Ally: 0

Experiences and Discussion

Quality of data– Data collected through automation matched manual

results, and in some cases was better Quantity of data

– We found that we could collect massive amounts of data with no visible impact on gameplay, even when data was reported throughout a game

Partial versus fully automated analyses– We found that results could be very different– Which is ultimately better?

Concluding Remarks

Content analysis plays several important roles to the video games industry, but is unfortunately an arduous task to complete in a thorough fashion

Our current work addresses this issue by providing an automated approach to content analysis based on software instrumentation

Initial experimentation with a prototype implementation of this approach demonstrates its usefulness and shows great promise

Concluding Remarks

Directions for future work include the following:– Conduct further experimentation and more detailed

content analyses of Unreal Tournament 2004, and combine qualitative analyses with our results

– Expand experimentation to other Unreal-based games– Investigate instrumentation of other popular game

engines and conduct further analyses this way– Create sensors for measuring other content metrics– Further explore the issue of partial versus fully

automated content analyses