An empirical comparison of first-person shooter ... · In ﬁrst-person shooter (FPS) games, the...

Contents lists available at ScienceDirect

Entertainment Computing

journal homepage: www.elsevier.com/locate/entcom

An empirical comparison of first-person shooter information displays: HUDs,diegetic displays, and spatial representations

Margaree Peacocke1, Robert J. Teather⁎,2, Jacques Carette3, I. Scott MacKenzie4,Victoria McArthur5

G-ScalE Lab, Dept. of Computing & Software, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4L8, Canada

A R T I C L E I N F O

Keywords:First-person shooterVideo gamesInformation displaysDiegeticHead-up displaysUser interfaces

A B S T R A C T

We present four experiments comparing player performance between several information displays used in first-person shooter (FPS) games. Broadly, these information displays included heads-up displays (HUDs), and al-ternatives such as spatial representations, and diegetic (in-game) indicators. Each experiment isolated a specifictask common to FPS games: (1) monitoring ammunition, (2) monitoring health, (3) matching the weapon to thesituation, and (4) navigating the environment. Correspondingly, each experiment studied a different informationtype, specifically ammunition (ammo) levels, health levels, current weapon, and navigation aids, while com-paring HUDs to alternatives. The goal was to expose player performance differences between different classes ofdisplays, and types of information displays (e.g., numeric, iconic, etc.). Results suggest that no one display type –HUDs or alternatives – are universally best; each performed well, depending on the type of information. Forammo, player performance was best with diegetic/spatial displays; for health information, players performedsignificantly better with a HUD. For weapon displays, results were best when showing a redundant HUD icon anda diegetic/spatial display (the actual weapon). Finally, for navigation, a spatial “navigation line” (showing thepath) was best, but HUD-based mini-maps offered competitive player performance. We discuss implications forthe design of first-person shooter games.

1. Introduction

In first-person shooter (FPS) games, the player sees the world fromthe eyes of a gun-wielding avatar, completing missions and shootingenemies. FPS games are wildly popular. The NPD Group reports that 3of the top 10 bestselling games of 2015 were FPS games [33]. They arealso highly profitable. For example, Activision’s Call of Duty: ModernWarfare 3 earned $400 million within 24 h of release and $1 billionwithin 16 days [14]. Player engagement is crucial to their success.

FPS games are interesting platforms for human-computer interac-tion (HCI) research. Due to the genre’s success and large user base,improvements to FPS user interfaces (UIs) affects many users.Consequently, there is a large body of research on FPS games[21,18,10,20,2,15,8,12,32,36]. Previous work in this area generallyfocuses on input-related issues, for example, aiming [21,32] or inputdevices [19,36]. While these input-related tasks are undoubtedly

critical in FPS UIs, information displays within FPS games have beencomparatively underexplored. We thus focus on the output-related taskof effectively displaying and conveying game information to the player.

Feedback is long recognized as crucial in user interface design[27,28]. When displaying game information, “feedback is crucial forplayer learning and satisfaction with the game” [28]. Most games, untilrecently, have traditionally used heads-up displays (HUDs) to presentthe player with relevant information. These HUDs usually display cri-tical information (e.g., health or ammo levels) as numbers, icons, ormeters displayed around the edge of the screen. Schaffer [30] arguesthat since HUDs on the periphery of the display occupy little gamespace, they are not likely to distract from gameplay. Fig. 1 depicts ex-ample HUDs from Call of Duty: Strike Team, Tom Clancy's Rainbow Six:Vegas, and Call of Duty: Ghosts.

Game designers increasingly seek to produce more immersive ex-periences. Immersion occurs when players “voluntarily adopt the game

https://doi.org/10.1016/j.entcom.2018.01.003Received 28 April 2017; Received in revised form 28 November 2017; Accepted 10 January 2018

⁎ Corresponding author at: 230 G Azrieli Pavilion, School of Information Technology, Carleton University, 1125 Colonel By Drive, Ottawa, Ontario K1S 5B6, Canada.

1 Present address: D2L, 151 Charles St. W. Suite 400, Waterloo, ON N2G 1H6, Canada.2 Present address: School of Information Technology, Carleton University, 1125 Colonel By Dr, Ottawa, ON K1S 5B6, Canada.3 Present address: Dept. of Computing & Software, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4L8, Canada.4 Present address: Department of Electrical Engineering and Computer Science, York University, 4700 Keele St, Toronto, ON M3J 1P3, Canada.5 Present address: Institute of Communication, Culture, Information and Technology, University of Toronto Mississauga, 3359 Mississauga Road, Mississauga, ON L5L 1C6, Canada.

E-mail address: [email protected] (R.J. Teather).

Entertainment Computing 26 (2018) 41–58

Available online 16 January 20181875-9521/ © 2018 Elsevier B.V. All rights reserved.

T

http://www.sciencedirect.com/science/journal/18759521

https://www.elsevier.com/locate/entcom

https://doi.org/10.1016/j.entcom.2018.01.003


mailto:[email protected]


http://crossmark.crossref.org/dialog/?doi=10.1016/j.entcom.2018.01.003&domain=pdf

world as a primary world and reason from the character’s point of view”([10], p. 69). HUDs may compromise immersion as they are not part ofthe game world, but rather overlaid on it. A comparatively recent trendin games is the use of alternative displays that may better preserve, oreven enhance, player immersion over HUDs [34]. The alternatives in-clude geometric/spatial representations, meta perceptions, and diegeticelements [10]. Fagerholt and Lorentzon categorized game displays ac-cording to whether they are presented in the game world (spatiality)and whether they are part of the game fiction (diegesis). This suggests atwo-dimensional model, as seen in Fig. 2.

Traditional HUDs fall within the upper left quadrant (Fig. 2, sincethey are neither part of the fictional game world nor are rendered(graphically) in the 3D game space. Conversely, diegetic displays arerendered in the game world, but are also a part of the game fiction. Theterm diegetic, borrowed from film, refers to elements within the gameworld that are part of the game world. In film, any element that can beperceived by the characters of the story is said to be diegetic [3]. Bycontrast, elements such as the musical score or subtitles are non-die-getic, since they are not perceived by the characters. By extension, ingaming, a diegetic UI element is part of the game world and is visibleand/or audible to the characters in the game world while simulta-neously providing information to the player [10]. Information dis-played in the game and recognized in the game fiction is considereddiegetic. In contrast, both spatial representations and meta re-presentations are typically non-diegetic, and involve non-traditional UIelements or on-screen placements. Spatial representations are elementswithin the game's 3D space. Meta perceptions are not displayed in thegame world, Instead, they are displayed in a fashion similar to HUDs,despite representing something in the game fiction. Common examplesinclude “blood spatter” or cracked glass when the player takes damage.We collectively refer to these non-HUD displays as “alternative” dis-plays (i.e., alternatives to HUDs).

These classes of displays are equivalent in the type of informationthey present the player; their comparative and objective effectiveness inpresenting information is the primary motivation for our research. Forexample, an ammunition counter could be displayed as a component ofa weapon, making the information seemingly visible to both the playerand avatar in a singular format. In this case, the ammunition counterwould be visible within the game space and is part of the game fiction,and hence would be considered a diegetic display. Alternatively, thesame information could be presented as a label, counter, or icons on the

HUD. However, it is unclear which display type provides the player themost efficient way to quickly intake the information. For example, al-though HUDs often present information in a clear and concise way, thisis not necessarily universal. Consider ammunition presented as anumber compared to a meter (e.g., Fig. 1c). The number presents theinformation more clearly. Moreover, in addition to potential immersionbenefits, alternative displays may also offer player performance ad-vantages over HUDs. The information presented by diegetic displays,for instance, can be displayed centrally, decreasing the need to glanceat the screen edges. Developers of several FPS (as well as third-personshooter) games have employed diegetic displays, typically to enhanceimmersion [17]. Some of the better-known examples include Metro2033 and Dead Space, although many games use variants of these. SeeFig. 3 for some common examples.

Understanding the relative performance considerations of alter-native displays may be important in the design of small-screen games,e.g., on mobile devices, now a bigger market than either the console orPC markets [24]. These small screens drive us to ask about efficient useof screen real estate. Alternative displays may help since they requireless screen real-estate than HUDs – they are embedded in the game,rather than covering parts of it. When screen real estate is at a premium,developers must consider the most effective way to convey informationto the player. Consider, for example, porting a PC game that makesheavy use of HUDs to a small-screen device. If diegetic options offercomparable user performance, then changing the HUDs to diegeticdisplays on the mobile app may be preferable to cluttering the interfacewith HUDs. However, to make such a decision, the performance dif-ferences between these display types must first be understood.

Our research focuses on the effectiveness – in terms of quantitativeuser performance – of in-game information displays. There is littlequantitative research on the performance offered by diegetic displays.Most work in this area is qualitative [10,20,12] or focuses on immer-sion [2,16]. While past research makes a convincing case that diegeticdisplays are more coherent with an escapist philosophy of game en-joyment, it is unclear if this view is consistent with the more utilitarianview of performance-based enjoyment. We note that player enjoyment,performance, and immersion are not necessarily aligned. For example,players may enjoy using a diegetic display, even if it offers demon-strably worse performance than an equivalent HUD. That said, if adisplay is sufficiently ineffective, it may negatively impact player ex-perience – much like any other bad user interface. We thus argue that

Fig. 1. Example HUD displays. (a) Call of Duty:Strike Team, depicting controls (soft buttons, left-side), health (variation of bar), and ammunition as anumber and bar (b) Tom Clancy's Rainbow Six:Vegas, depicting ammunition numerically (c) Call ofDuty: Ghosts depicting ammunition both numeri-cally and as a bar/meter.

Fig. 2. A classification of game UI elements, based ondiegesis (if the UI exists in the fictional game world)and spatial orientation (if the UI element is visualizedas part of the 3D game space). Figure based on that ofFagerholt and Lorentzon ([10], p. 51), but simplifiedto include only display types found in our experi-ments.

M. Peacocke et al. Entertainment Computing 26 (2018) 41–58

42

research in this area must also examine performance trade-offs betweenthe different display types. Although we are also interested in the re-lationship between player experience and display type, our currentwork focuses exclusively on player performance.

In this context, our primary research question is thus empirical:How do various information displays affect in-game success with the“micro-tasks” that make up a game experience? Our work presents whatis, to our knowledge, the first experimental comparison of the perfor-mance offered by HUDs compared to alternatives such as diegetic dis-plays or spatial representations. We first present an analysis of recentFPS games to identify the most important information displayed duringgameplay. This analysis and previous work [29] identified four types ofinformation common to modern FPS games: health, ammunition level,the player’s current weapon, and navigation aid. This led us to conductfour experiments comparing the information displays commonly usedfor each information type. These experiments are not intended as aseries; rather, they represent a cross-section of tasks common to mostFPS games, isolated to offer greater experimental control. In each ex-periment, we included both HUD and alternative display options. Our“diegetic” display options do not technically qualify as such, as definedby Fagerholt and Lorentzon [10]. After all, our experimental platform –a custom-developed game – presents no in-game fiction. In the absenceof in-game fiction, diegetic displays and spatial representations essen-tially collapse into the same class of display, as do HUDs and meta-

perceptions. In terms of game mechanics, and in isolation from enjoy-ment or immersion, diegetic displays and spatial representations shouldoffer comparable player performance. Consequently, the displays in-cluded in our study align with commercial games, as found in our initialanalysis. They include HUDs, meta-perceptions, spatial representations,and diegetic displays. Our analysis further revealed that there is often amix of HUDs and alternative displays within the same game. Thisvariety is reflected in the options studied herein. Our study used acustom-developed FPS game, which offers better experimental controlthan commercial games [25,31], and avoids participant biases towardsexisting games [10]. The displays selected are discussed further in theMethodology section for each experiment.

We solicited participants who regularly play FPS games, sinceskilled gamers can quickly assess their status, while novice playerscannot [8]. Hence expert gamers should be skilled enough to elicitdifferences between the conditions studied. In contrast, novice parti-cipants require training to get to this level of skill, and thus may notexpose differences between the experimental conditions. From an ex-periment design point of view, this decision makes sense. Novice par-ticipants introduce a greater degree of variability in performancemeasures. Statistically, this means novice participants are less likely toproduce statistically significantly differences, despite potentially largedifferences in the conditions studied [22].

Fig. 3. In-game displays. (a) Call of Duty: Modern Warfare 2 makes use of a meta-perception (blood splatter) to represent health information. (b) Dead Space displays the health meter(cyan bar mounted on player's back) diegetically. The in-game inventory is also presented like an augmented reality display floating in front of the player. (c) Watch Dogs uses a spatialnavigation display which is visually similar to Dead Space’s diegetic navigation aid. Unlike Dead Space, this navigation aid is not part of the game fiction (it is visible to the player, not thecharacter) and hence is not considered diegetic.


43

2. Related work

There is considerable research on FPS games in the HCI literature[19,26,32,36]. The general themes are aiming [32], targeting in thecontext of pointing [21,23,36], or metrics for empirical evaluation ofFPS UIs [19]. Other work focused on developing or evaluating inputdevices [18,26,15] or immersion [5,10,2,16]. To date, performancecomparisons of HUD and alternative displays have received little at-tention.

Research on FPS information displays is primarily qualitative[10,20,12]. Past results indicate that participants support the use ofdiegetic displays to enhance immersion, as long as it does not impactperformance. Similarly, if information is clearly communicated, playerstend not to care how it is displayed [20,12]. Applicable game designheuristics also exist. Federoff states that “the interface should be as non-intrusive as possible” [11] and that “a player should always be able toidentify their score/status in a game” [11]. We argue that empiricalstudies on the effectiveness of these displays are needed to complementexisting qualitative work and to improve design heuristics.

While there is little empirical work on player performance withdiegetic and non-diegetic game elements, several studies have focusedon the immersive qualities of diegetic displays. For example, Babu [2]compared immersion levels in two games with diegetic displays (Metro2033 and Dead Space) and two games with HUDs (Bioshock and ResidentEvil 5). Immersion was assessed through self-reporting on a 5-pointLikert scale and was not significantly different between display types.Participants instead suggested that graphics and storyline had astronger impact on immersion. Recent work by Iacovides et al. [16]revealed that diegetic displays can indeed enhance immersion, but theeffect is stronger for expert gamers.

Galloway [13] introduced the terms diegetic and non-diegetic to thestudy of video games. The terms originated in literary and film theory.He defines game diegesis as “the game’s total world of narrative action”[13], and non-diegetic as “gamic elements that are inside the totalgamic apparatus yet outside the portion of the apparatus that con-stitutes a pretend world of character and story” [13]. He concludes thatHUDs are non-diegetic elements.

Fagerholt and Lorentzon [10] also explore diegesis, developing adescriptive model categorizing FPS UI elements on whether the elementexists in the fictional world and is a part of the game space. They re-commend considering the game’s fiction when deciding if informationshould be displayed diegetically, arguing that game coherence isparamount. For example, diegetic options make sense in a game likeDead Space, as the futuristic setting allows designers to cast diegeticdisplays as future technologies such as augmented reality displays orholograms. Ultimately, Fagerholt and Lorentzon suggest using diegeticdisplays whenever appropriate and where game cohesion can be re-tained. However, the merit of this suggestion is questionable in theabsence of empirical results assessing the potential performance impactof such a design choice. For FPS games, we believe that player per-formance is ultimately the most important factor.

Fragoso [12] conducted a qualitative study of the effects of diegeticdisplays on player immersion. Participants played EA’s Battlefield 3,which is considered more immersive than other games due to theminimal use of a HUD. For example, the game employs “blood splatter”as health status – as the player takes damage, the screen becomes in-creasingly occluded by blood – rather than using more common healthbars or numeric displays. Participants reported that the lack of mean-ingful feedback was disruptive, due to the vagueness of the displays.The authors conclude that effective feedback is more important thanrealism. They further report that HUDs were less disruptive than theirdiegetic counterparts. These sentiments are echoed by Llanos andJørgensen [20] who report that while players liked the aesthetic ofdiegetic displays, they preferred clear communication. However, theyalso note that players were annoyed when excessive information isdisplayed on HUDs. More recent work by Caroux et al. [6] suggest that

differences in player experience between HUDs and diegetic options aredependent on player expertise and game genre.

Zammitto [35] conducted a visual analysis of Valve’s Half Life 2 toassess if visualization design principles were applied to presenting gameinformation. She notes that the game applied two principles to the HUDammunition display: silhouette and colour coding. These were im-plemented by (i) showing a bullet icon when the player should reloadtheir weapon and (ii) changing the ammunition indicator from yellowto red when ammunition was low. Red is appropriate because it con-notes “danger” in Western cultures. A similar approach was used in thegame’s health indicators. Overall, Zammitto concluded that informationvisualization is not well used in video games.

Bowman et al. [4] share this sentiment, noting that because datavisualization in games is new, it is relatively underutilized. They ana-lyzed visualization in games and proposed a design framework. Their“primary purpose” dimension classifies critical game information asStatus, noting that “visual representations are often chosen in lieu of asimple number … because the game designers feel that visualization ismore immersive and easier to read quickly” ([4], p. 1961). They re-commend considering the target audience to ensure that “the visuali-zation is in spirit with the game’s atmosphere and integrated within thegame” ([4], p. 1962). This is consistent with the recommendation ofFagerholt and Lorentzon [10]. The consensus is that FPS players valuecohesion in games and that proper data visualizations improve players’situational awareness.

3. Analysis of current games

Before presenting our experiments comparing several informationdisplay types, we first explain how we determined which classes ofinformation were most critical (i.e., ammunition, health, currentweapon, navigation aid) and specifically which display types are mostcommonly used in modern shooter games (i.e., diegetic displays, HUDs,etc.). We analyzed several recent and popular shooter games (not ex-clusively FPS) across multiple platforms. The games were chosen be-cause of sales and awards, and because they are available for large andsmall screen platforms. As discussed earlier, the latter necessitates UIchanges between display sizes, which may favour the use of in-gamedisplays (e.g., diegetic, spatial). The purpose of this analysis was tolearn what types of information were consistently displayed in shootergames, and what types of displays were most commonly used – not onlyin terms of the display class (e.g., HUDs vs. diegetic), but also theirpresentation (e.g., numbers vs. icons, etc.). Our intent was to focus onthe most critical information, to inform the design of our experiments.The games analyzed included Activisions’ Call of Duty: Strike Team, Callof Duty: Black Ops, Call of Duty: Ghosts, Ubisoft’s Tom Clancy’s RainbowSix: Vegas, Bioware’s Mass Effect 3, Mass Effect Infiltrator and EA’s DeadSpace. The analysis involved playing these games, watching gameplayvideos, and reading publicly available game reviews.

We found that four types of information were common to everygame analyzed. These included player health, ammunition level, cur-rent weapon, and navigation aid. We thus conclude that these are themost important information displays in FPS games. The displaymethods used for health, ammo, weapon, and navigation are shown inTable 1. Games using alternative displays are shaded, with the alter-native (non-HUD) option set in boldface. Note that navigation aid ismostly displayed as a mini-map in multiplayer mode, but as a naviga-tion arrow in single-player campaign mode. Our analysis focused ex-clusively on single-player campaign modes.

Numeric displays show a numeric count, typically on a HUD (seeFig. 1b). They are useful for displaying “amounts of things for whichyou would normally use digits in the real world” [1], such as ammu-nition. Numeric displays are especially useful for large quantities.

Bars (see Fig. 1c) are also useful for large quantities [1]. These areoften presented as a meter that is full at the maximum quantity, andempties as appropriate. The primary benefit is that bars can be


44

interpreted at a glance.Icon bars (Fig. 1c), or “small multiples” [1], are best for small-in-

teger numeric data. Icons are thus useful for indicating the quantities ofaround five items or less. Players have difficulty taking in greater thanfive items at a glance, and thus have greater difficulty remembering thenumber. However, Adams suggests using graphical indicators ratherthan text or numbers because they are easier to read at a glance [1].Our analysis indicates that bar and numeric displays are commonlyused together. This offers players the ability to both read at a glance andreceive more detailed information as desired.

Our analysis reveals some consistency in alternative health displays(favouring “blood spatter” – a meta perception) and weapon displays(typically diegetically displaying the weapon in front of the player,often with redundant HUD-based icons showing the current weapon).However, there is little consistency in ammunition displays. As seen inTable 1, there is great variety in the presentation of ammo HUDs, andeven some diegetic options. Since alternative displays are relativelynew (compared to HUD-based displays), design standards have yet toevolve and it is important to develop best practices early. EA’s DeadSpace (see Fig. 3b) has been praised for its lack of a HUD, relying in-stead on diegetic displays. In Dead Space, ammo is displayed using anumeric count positioned directly above the weapon, and health as abar physically mounted on the player’s back, which is coherent with itsfuturistic theming.

4. Common methodology

The following Sections 5–8 present four user studies designed toevaluate different UI display options in an instrumented FPS game. Thestudies were designed to compare different UI options (traditionalHUDs vs. alternative displays) for ammunition, health, weapons, andnavigation respectively. The design of each HUD and alternative UIdisplay for each experiment was motivated by the analysis of com-mercial FPS games presented in the previous section. Section 4 providesdetails common to the four experiments here. Later sections provideexperiment-specific details.

4.1. Apparatus

The experiments were conducted on a 3.4 GHz quad-core i7-basedPC, with 8 GB of RAM running Windows 7. A 75-in. Samsung Series 77100 Smart TV (1920×1080 pixel resolution) was used for the display.The display ran in game mode to minimize latency. Participants wereseated approximately 15 ft. from the screen, which allowed viewing ofthe entire display without excessive gaze shifts. The setup is shown inFig. 4a.

A custom game was developed using Unity Technologies’ Unity 4.5engine. The game presented a first-person view to the player, consistentwith the appearance of a typical FPS game. It supported typical controlsoffered by FPS games, such as turning, moving, and shooting. However,

Table 1Analysis of current game displays for health, remaining ammunition, and current weapon. Alternative (non-HUD) options are set in boldface font. Displays with at least one alternative(e.g., diegetic, spatial, etc.) option are highlighted.

Fig. 4. Experimental setup and hardware apparatus. (a) A participant performing the experiment. (b) Annotated Xbox One controller.


45

since we used experimental software and not a “true” game, there wasno fiction, and no in-game objectives beyond those set for each ex-periment (e.g., shoot enemies, navigate a maze, etc.). Moreover, certaincontrols were enabled/disabled as necessary in each of the four ex-periments. For example, movement was disabled in three of the fourexperiments. Further details of the game appear in the following ex-periment Sections 5–8. Participants used a Microsoft Xbox One con-troller to play the game. See Fig. 4b. Viewpoint rotation/aiming wascontrolled by the right analog stick in all experiments. In experimentsthat included shooting, the right trigger was used to shoot. The game(during the ammo experiment) is depicted in Fig. 5.

4.2. Procedure

Upon arrival, participants were greeted and the purpose of the ex-periment was explained. Participants gave informed consent beforeproceeding. They were instructed on the mechanics of each display andthe controls. They were then allowed to begin. Upon finishing all trials,participants completed a questionnaire asking about prior experiencewith FPS games and soliciting subjective feedback.

5. Experiment 1: Ammunition displays

The purpose of this experiment was to assess user performancedifferences in a simulated ammo monitoring task. Participants werepresented with five different ammunition displays – including HUD andspatial/diegetic displays – and tasked with monitoring ammo whileshooting enemies.

5.1. Participants

Twenty paid participants (16 male) took part in the study. Agesranged from 18 to 38 years (mean 22.35, SD 4.31). Half reported thattheir preferred system was a console and the other half reported PC. Allparticipants were regular gamers, playing between 1 and 10 h per week.Sixteen participants reported playing FPS games every week.

5.2. Apparatus

The player’s ammunition level was displayed using one of the fiveammunition displays shown in Fig. 6. The ammunition displays in-cluded bar-on-HUD (BH), number-on-HUD (NH), icons-on-HUD (IH),number-in-game (NG), and icons-in-game (IG). Each presented thesame information, but visualized it differently. The three HUD-basedoptions were similar to those described earlier, presenting ammo aseither a number, a “meter”, or as a set of icons displayed at the lowerright corner of the display. The alternative options (NG and IG) were

modeled after diegetic displays found in games like Microsoft Studios’Halo 4 and Dead Space. In these games, futuristic weapons include anammo numeric counter built into the gun, or bullets visualized throughthe gun (e.g., Metro 2033).

The game was set in a simulated warehouse. There were 25 enemysoldiers initially positioned in a semi-circle around the player (Fig. 5.The enemies walked slowly towards the player. The player had a riflewhich fired one shot per trigger press. Enemies died when shot, anddisappeared. The right trigger button shot, the x-button reloaded. Re-loading was only possible upon running out of ammunition (i.e., usingall shots in the clip).

The software automatically recorded the number of clips used, hits,misses, enemies remaining, shots before reload, and time before reload.For each shot, the time was recorded along with the remaining am-munition and whether the shot hit or missed an enemy.

5.3. Procedure

Participants were instructed to play the game to the best of theirability, shooting all enemy soldiers as quickly and accurately as pos-sible. They were informed that they had unlimited ammunition, buteach clip only had a certain number of shots. Consequently, participantshad to reload when out of ammo. They were then instructed on thecontrols and were allowed to begin. A trial ended when all enemieswere killed.

On starting the trial and after reloading, each clip had a randomnumber of rounds. The number of rounds ranged from 7 to 16 (decidedonce per trial). Using a random number of shots per clip requiredparticipants pay greater attention to their ammunition level. Thishelped prevent participants from mentally tracking ammunition, andthus was expected to help elicit differences between the test conditions.Upon running out of ammunition, participants manually reloaded (andcould not reload prior to running out). This task was representative ofreal games: Ammunition level becomes crucial when it is low in a battlesituation. The task requires participants to be highly aware of theirammunition level.

Participants completed 15 trials for each of the five ammunitiondisplays, completing 75 trials in total. After each trial participants couldtake a break before continuing. Each trial took between 30 and 45 s. Intotal, the experiment took approximately 1 h.

Upon finishing all trials, participants completed a questionnaireabout prior experience with FPS games. The questionnaire also askedparticipants about their perceived effectiveness of each ammunitiondisplay.

Fig. 5. The custom FPS game (showing the ammoexperiment task).


46

5.4. Design

The study employed a 5×15 within-subjects design. The in-dependent variables and levels were as follows:

Ammunition Display: BH, NH, IH, NG, IGTrial: 1, 2, 3, … 15

The ammunition displays are depicted in Fig. 6 and described inSection 5.2. Ammunition display ordering was counterbalanced ac-cording to a Latin square.

The dependent variables were the number of shots before reload(count) and the time before reload (seconds). Shots before reload wasthe average number of shots fired from the time the participant ran outof ammunition until the reload button was pushed. Time before reloadwas the time between running out of ammunition and pushing the re-load button.

In total, participants completed 20× 5 × 15=1500 trials.

5.5. Results

5.5.1. Shots before reloadAchieving a low score for shots before reload required the partici-

pant to notice they had no ammunition left. A high score thus indicatesdecreased awareness of ammunition levels. Average shots before reloadis summarized for each ammunition display in Fig. 7.

The main effect of ammunition display on shots before reload wasstatistically significant (F4,19 = 9.22, p < .0001). A Tukey-Kramer posthoc analysis revealed that the difference between number-in-game (NG)and all other ammunition displays was statistically significant(p < .05). The rest of the ammunition displays were not significantlydifferent from each other. The main effect for trial on shots before re-load was not significant (F14,19 = 0.94, ns), nor was the interactioneffect between ammunition display and trial (F56,19 = 1.03, p > .05).

The worst performing ammunition displays were the HUD options(BH, NH, IH). All three had comparable scores (slightly over 1 each)and were not significantly different from one another. Although icons-in-game (IG) performed slightly better, the difference was not sig-nificant. The best performing option was number-in-game (NG), which

had 0.68 shots fired before reload. Number-in-game resulted in about35% fewer shots before reload than the worst performer, icons-on-HUD.

Participants noted that number-in-game (NG) was easy to see, as theammunition count was almost directly where they were looking whileaiming. The HUD-based displays were in the bottom right corner, re-quiring more glancing. These ammunition displays performed very si-milarly, suggesting a relationship between performance and displaylocation. We speculate that positioning the HUD in a different location(e.g., another corner of the screen) is unlikely to yield a substantialperformance difference, unless they are placed closer to the screencentre. However, it appears that “counting” the icons takes enough timethat, in the icons-in-game (IG), their central location was not sufficientto yield a statistically significant performance boost. This suggests thatvisualization may have a greater influence than position.

5.5.2. Time before reloadLike shots before reload, higher scores were worse: The greater time

before reload was, the lower the awareness of the ammunition level.Lower scores indicate a more immediate awareness of low ammo levels.Average time before reload for each ammunition display is depicted inFig. 8.

There was a significant main effect of ammunition display on timebefore reload (F4,19 = 4.26, p < .005). A Tukey-Kramer analysis in-dicated a significant difference between number-in-game (NG) and allother ammunition displays. The main effect for trial was not significant(F14,19 = 1.61, p > .05), nor was the interaction between ammunitiondisplay and trial (F56,19 = 0.92, ns).

As with shots before reload, the icons-on-HUD (IH) ammunitiondisplay performed worst, and the number-in-game (NG) ammunitiondisplay performed best. NG offered the lowest time before reload, withan average of 1.0 s, approximately 20% lower than the next best per-forming ammunition display, number-on-HUD (NH). The most sub-stantial difference was between icons-on-HUD (IH) and number-in-game (NG) ammunition displays. NG was about 26% faster than IH.

The results are rather consistent for both dependent variables. Itappears the central location of the number-in-game ammunition displayallows for better performance than the other displays. This is mostlikely because it reduces the amount of gaze shifting or glancing re-quired. Again, IG seems to require enough additional mental processing

Fig. 6. The five ammo display conditions for Experiment 1. (a) Bar-on-HUD (BH), (b) Number-on-HUD (NH), (c) Icons-on-HUD (IH), (d) Number-in-game (NG), (e) Icons-in-game (IG).

Fig. 7. Shots before reload by ammunition display. Lower scores are better. Error barsshow±1 SD.

Fig. 8. Time before reload for each ammunition display. Lower scores are better. Errorbars show±1 SD.


47

to outweigh this positional advantage, however.

5.5.3. QuestionnaireParticipants completed a questionnaire soliciting their feedback on

the ammunition displays studied. They were asked to rate their pre-ference towards each ammunition display on a 5-point Likert scale.Specifically, they were asked “Did each of the ammunition displays helpor hinder your gameplay?” with response options ranging from “Reallyhindered” to “Really helped”. Fig. 9 depicts the percentage of partici-pants for each response level.

Overall, the number-in-game (NG) ammunition display was con-sidered the most effective, with 80% of participants reporting theyfound it helpful or really helpful. Opinions toward icons-in-game (IG),icons-on-HUD (IH), and number-on-HUD (NH) ammunition displayswere mixed. The bar-on-HUD (BH) was thought to hinder gameplay by45% of participants. A Friedman non-parametric test deemed the dif-ferences statistically significant (χ2 = 11.56, p < .05, df=4). A posthoc analysis revealed significant differences between number-in-game(NG) and bar-on-HUD (BH), number-in-game (NG) and number-on-HUD (NH), and number-in-game (NG) and icons-in-game (IG).Participants tended to correctly identify the most effective display,exhibiting higher subjective preference towards it.

5.6. Summary

Overall, the diegetic number-in-game (NG) display offered sig-nificantly better performance than all other displays studied.Participants were also aware of the performance difference, as theyranked the display significantly better in terms of subjective perceivedeffectiveness. This may be due to the central placement of the display(near the screen centre), made possible by the fact that it was displayedspatially rather than as a HUD. Previous work [7] revealed differencesin player performance due to HUD position, hence we included HUDposition in the next experiment to investigate further.

6. Experiment 2: Health displays

The purpose of this experiment was to evaluate which of severalHUD and diegetic/spatial options presented health information mosteffectively to the player. The task involved monitoring health levelswhile being shot at by enemies. Participants had to notice when theirhealth level fell below a certain threshold.

6.1. Participants

Twenty-four paid participants (21 male, 3 female) took part in thestudy. Ages ranged from 19 to 53 years (mean 25.4, SD 7.2). All wereregular gamers, with 58% playing video games for more than 10 h perweek, and 86% playing first-person shooter games every week. None ofthe participants had participated in the previous (ammunition) ex-periment.

6.2. Apparatus

The game was set within a turret with five enemies surrounding theparticipant’s character in a semi-circle. Movement was disabled. Theright trigger button shot and the left bumper button “escaped” whenhealth was low (upon reaching 20% or lower). The player initially had100 health points. Enemies shot at the player, causing 5 points of da-mage with each hit. The player would “die” (ending the trial un-successfully) upon reaching 0% health.

The participant’s health level was displayed using one of 12 healthdisplays. Each display presented the same information, but visualized itdifferently. Nine were HUD-based, derived from all combinations ofthree visualizations (icons, bar, and number) and three display posi-tions (top, bottom, left). The bar-based options presented health as ameter decreasing towards the left as the player took damage. The iconsshowed hearts, similar to games like The Legend of Zelda. The numberoptions presented health as a percentage of their total. Fig. 10 depictsthe three different HUD visualizations and Fig. 11 depicts the threedisplay positions. The nine HUD-based options were coded thus:

• bar-on-HUD-bottom (BHB)

• bar-on-HUD-left (BHL)

• bar-on-HUD-top (BHT)

• icons-on-HUD-bottom (IHB)

• icons-on-HUD-left (IHL)

• icons-on-HUD-top (IHT)

• number-on-HUD-bottom (NHB)

• number-on-HUD-left (NHL)

• number-on-HUD-top (NHT)

The bar and number visualizations were selected since they com-monly appear in FPS games, while icons are common in games of othergenres, but may have potential in FPS games as well.

Three HUD-alternative options were also studied. Two were spatial/diegetic variants of the HUDs described above, icons-in-game (IG), andbar-in-game (BG). They operated the same as the HUD-based variants,but were displayed on the 3D model of the character rather than as aHUD, similar to the number-in-game option in the ammo experiment.The third was blood splatter (S), a commonly used meta-perceptionwhich increasingly occludes the participant’s view with simulatedblood as they take more damage. These are depicted in Fig. 12.

6.3. Procedure

Participants were instructed to shoot enemies while the enemiesshot at them. Each time they hit an enemy, their score increased. The

Fig. 9. Participant perceived effectiveness of each ammunition display.

Fig. 10. Health display visualizations including (a)icons (b) bar and (c) numbers.


48

point value of hitting an enemy increased with lower health. This en-couraged participants to stay as long as possible without reaching 0health (i.e., dying) and to get as high a score as possible. They wereinformed that once their health reached 20% or lower, they could“escape” by pressing the escape button, which transported them to asafe location and successfully completed the trial. The primary goal wasto escape, while the secondary goal was to get a high score. The overallhigh score only updated if they escaped before running out of health.The data recorded included participant escape before 0 health, healthremaining (zero if they did not escape), and score. Each shot was re-corded along with hit or miss and, if it hit an enemy, how many pointswere earned.

Participants completed 20 trials for each of the 12 health displays,240 trials in total. After each trial participants could take a break. Eachtrial took approximately 5 s, or about 25min for the entire experiment.

6.4. Design


Health Display: BHB, BHL, BHT, IHB, IHL, IHT, NHB, NHL, NHT, BG,IG, STrial: 1, 2, 3, …, 20

The ordering of health display was randomized in order to offsetpotential learning effects. The dependent variables were escape per-centage (%) and health when escaped (%). Escape percentage wascalculated as the percentage of trials where participants were able toescape (i.e., without dying). Health when escaped was the health re-maining (as a percentage of their initial 100 health points) when par-ticipants successfully escaped, avoiding dying in a given trial.

Fig. 11. Health display positions used with each vi-sualization. (a) bottom (b) top (c) left. Note: only barvisualizations are shown here. Number and iconsappear in the same locations, but use the visualiza-tions shown in Fig. 10.

Fig. 12. Alternative displays: (a) bar-in-game, (b) icons-in-game, and (c) splatter (at reaching 0% health).


49

Participants completed 24×12 × 20=5760 individual trials.

6.5. Results

6.5.1. Escape percentageA high escape percentage indicates better performance, as partici-

pants could see and understand the health display well, escaping fromthe trial before dying with greater reliability. The best performinghealth display was the HUD-based number-on-HUD-top (NHT), with76%. The worst performer was the bar-on-HUD-top (BHT), with 46%,followed closely by bar-on-HUD-bottom (BHB) and bar-on-HUD-left(BHL), both at 47%. See Fig. 13.

A repeated measures ANOVA revealed a significant main effect forhealth display on escape percentage (F11,23 = 12.91, p < .0001). AFisher LSD pair-wise analysis revealed that the number-on-HUD dis-plays (NHB, NHL, and NHT) all performed significantly better than allother health displays except for icons-in-game (IG). Icons-on-HUD-bottom (IHB) performed significantly better than all bar-on-HUD var-iants.

In general, the number-based options did well, likely because it waseasier to determine when health was between 0% and 20%. The bar andicon options were more ambiguous, although icons were better thanbars. Position appeared to matter less than visualization; as seen inFig. 13, results “cluster” by visualization rather than location.

6.5.2. Health when escapedScores for health when escaped are always between 0% and 20%,

since participants could only escape upon reaching 20% or lower health(and died upon hitting 0% health). Lower scores are better, since theywere tasked with getting the highest score possible while still escaping.Participants were rewarded for staying longer as they were awardedhigher scores when their health was low.

Icons-on-HUD-top (IHT) had the best result, with an average healthof 6.54% upon escape. Since each enemy shot cost 5 health points, thismeans participants tended to escape with slightly more than one shotleft of health until they would have died in a given trial. This wasfollowed closely by bar-on-HUD-top (BHT), with an average healthwhen escaped of 6.65%. Icons-in-game (IG) had the highest healthwhen escaped, with an average of 12.88%. This was approximately49% higher than with the icons-on-HUD-top (IHT) health display. Allnumber options were close to 10% (i.e., around 2 enemy shots fromdying). See Fig. 14.

There was a significant main effect for health display on healthwhen escaped (F11,23 = 16.01, p < .0001). In summary, participantsescaped with significantly lower health remaining with the bar and icondisplays on the HUD. All other health displays had significantly lowerhealth when escaped than the icons-in-game (IG).

6.5.3. Subjective resultsParticipants were queried the effectiveness of health display on a 5-

point Likert scale, ranging from “Really hindered” performance, to“Really helped” performance. Results are summarized in Fig. 15, andsuggest strong preference towards the HUD-based options, particularlythe Bar-on-HUD variants. The results were found to be statisticallysignificant using a Friedman non-parametric test (χ2 = 15.134,p < .01, df=5). A post hoc analysis revealed significantly differentpreference levels between bar-on-HUD (BH) and bar-in-game (BG),icons-in-game (IG), and spatter (S), as well as between number-on-HUD(NH) and bar-in-game (BG), icons-in-game (IG), and spatter (S). Thisresult shows that the bar and number on HUD were thought to be muchmore helpful than the alternative displays (BG, IG, and S).

6.6. Summary

Overall, the HUD-based variants offered the best performance, re-gardless of placement on the screen. In terms of trial success rate, nu-meric options were best. Only the diegetic icon-in-game option cameclose. Subjectively, participants preferred the bar-on-HUD display, de-spite its relatively worse performance.

7. Experiment 3: Weapon displays

The purpose of this experiment was to compare different options forpresenting the currently selected weapon. We note that all modern (andmost pre-modern) FPS games show the current weapon diegetically infront of the player. However, many include redundant icons, text, orvarious other HUD representations. Players were tasked with trying tomatch their weapon to specific enemies (a colour-matching task).

7.1. Participants

The same 24 participants that completed the health display ex-periment completed this experiment, either right before, or right after.The ordering of the experiments was alternated to eliminate learning/practice effects. Half of the participants completed the health experi-ment first, followed by this experiment. The other half completed thetwo experiments in the reverse order.

7.2. Apparatus

The experiment was set in the same warehouse scenario as in theammunition experiment. Eight groups of three enemies appeared insuccession, each group appearing after the last was destroyed by theparticipant. Enemy groups were coloured red, green, blue, or yellow,and all enemies in a single group were the same colour. See Fig. 16.

Fig. 13. Average escape percentage by health display.Higher scores are better. Error bars show±1 SD.


50

Player movement was disabled, but the participant could look aroundand aim using the right analog stick, and shoot using the right triggerbutton. Each direction pad button mapped to switching to a differentweapon colour (i.e., up→ red, right→ green, down→ blue, left→yellow; see Fig. 4b). This direct mapping was used instead of pressing

buttons to cycle through weapons, as the latter introduced additionaldelay in contrast with the immediacy of pressing a direction button.

The UI displayed the participant’s current weapon (colour) usingone of six weapon displays: name-on-HUD-right (NHR) name-on-HUD-left (NHL), icon-on-HUD-right (IHR), icon-on-HUD-left (IHL), in-front(IF) and name-on-gun (NG). The “left” and “right” options indicated onwhich side of the bottom of the screen a HUD-based option was dis-played. As with the previous experiments, each weapon display pre-sented the same information, but visualized it differently, either asHUDs via text or an icon, or spatially (i.e., rendered directly on theweapon in the game itself). The displays are seen in Fig. 17. Note thatthe spatial “in-front” option (which positions the weapon display infront of the player) is consistently used in all modern FPS games. Therest of the weapon displays and positions are found in recent FPSgames.

7.3. Procedure

Participants were instructed to match the weapon colour to theenemy colour and shoot as quickly as possible to get a high score.

Fig. 14. Health when escaped by health display.Lower scores are better. Error bars show±1 SD.

Fig. 15. Participant perceived effectiveness for each health display, showing percentageof participants giving each response.

Fig. 16. Weapon display experiment, showing overview of the task and scene.


51

Shooting an enemy with a gun of the wrong colour had no effect. Weused colour matching, rather than different weapons, since matchingspecific weapons to enemy types would require additional training.Colour matching is more straightforward. Upon killing an enemy group,a new randomly coloured group appeared. A participant could get a“kill streak”, gaining extra points, by killing additional enemies within750ms of the previous kill. The high-score goal encouraged speed and,in turn, necessitated quick recognition of the current weapon. A trialfinished when all enemy groups were destroyed. Data recorded in-cluded number of shots with the wrong colour weapon and time toswitch to the correct weapon.

Participants completed 8 trials for each of the six weapon displays,for 48 trials in total. After each trial participants could take a breakbefore continuing. Each trial took about 25 s, or about 25min for theentire experiment.

7.4. Design

The study employed a 6× 8 within-subjects design. The in-dependent variables and levels were as follows:

Weapon Display: NHR, NHL, IHR, IHL, IF, NGTrial: 1, 2, 3, …, 8

The ordering of weapon display was counterbalanced with a Latinsquare.

The dependent variables were shots with wrong weapon (%), andweapon switch time (seconds). Shots with wrong weapon is the per-centage of shots fired with a non-matching weapon. Weapon switchtime is the time to switch from an incorrect to correct weapon colour.

Overall, participants completed 6×8 × 24=1728 trials.

7.5. Results

7.5.1. Shots with wrong weaponFewer shots with wrong weapon scores are better, as this indicates

participants could more quickly identify when using a non-matchingweapon on an enemy. Icons-on-HUD-right (IHR) performed best, with2.12% of shots with wrong weapon. This was closely followed by theweapon in-front (IF) display, with 2.20%. The name-on-gun (NG) dis-play performed worst, with 2.93% of shots with wrong weapon. SeeFig. 18. A repeated measures ANOVA revealed that the main effect fortrial was significant (F7,23 = 2.44, p < .05), while the main effect forweapon display was not (F5,23 = 1.74, p > .05). The interaction effectwas not significant (F35,23 = 0.82, ns).

7.5.2. Weapon switch timeWeapon switch time indicates how quickly participants recognized

they were using the wrong weapon and changed. Lower times arebetter. The icon-on-HUD-right (IHR) again performed best, at anaverage of 1.01 s to switch to the correct weapon. This was closelyfollowed by name-on-HUD-left (NHL), in-front (IF), then name-on-

HUD-right (NHR). The name-on-gun (NG) weapon display performedworst, with an average of 1.16 s to switch to the correct weapon. Thiswas approximately 11% worse than with the icons-on-HUD-right (IHR)display. See Fig. 19.

The main effect for trial on weapon switch time was significant(F7.23 = 17.33, p < .0001), while the main effect for weapon displayon weapon switch time was not significant (F5,23 = 2.03, p > .05). Theinteraction effect between weapon display and trial was significant(F35,23 = 1.73, p < .01). A Fisher LSD pair-wise analysis revealed asignificant difference between NHR trial 1, IHR trial 1, and NG trial 1,and the remaining trials.

7.5.3. Subjective resultsParticipants ranked their preference towards each weapon display

on a 5-point Likert scale, ranging from “Really hindered” (1) to “Reallyhelped” (5). Results are summarized in Fig. 20. Participants found IFand IH most helpful. The results were statistically significant using aFriedman non-parametric test (χ2 = 40.854, p < .0005, df=3). Post-hoc analysis revealed the significant pairwise differences between allcondition pairs except for name-on-HUD and name-on-gun. Participantsperformed well with most preferred weapon displays – icon-on-HUDand in-front. This suggests that both of these weapon displays are goodchoices, and also could be good to use in combination.

7.6. Summary

Results of this experiment revealed little difference between thedisplays studied. Overall, participants got better over the 8 trials, butnone of the displays were significantly different in terms of perfor-mance. Participants strongly preferred the “in-front” display, althoughicon-on-HUD was also positively assessed.

8. Experiment 4: Navigation aid displays

The final task we studied was navigation through the environment.Players were tasked with moving through a maze, with several differentnavigation aids to assist them, ranging from diegetic compasses, mini-map HUDs, or a “navigation line” modeled after the diegetic way-finding aid found in Dead Space, which showed the player the exactpath through the maze. While we expected this latter option to performbest, we included it primarily as a “ground truth” or control condition.

8.1. Participants

Twenty-one of the participants who completed the preceding ex-periments also took part in this experiment. Any participants whocompleted multiple experiments did so on different days, such that anequal number did each possible ordering of experiments. In total, thisexperiment included 24 paid participants (22 male, 2 female). Agesranged from 19 to 53 years (mean 25.3, SD 7.3). All participants wereregular gamers, with 46% reporting they play video games more than10 h per week.

Fig. 17. Weapon displays used in experiment. (a) Icon-on-HUD, (b) Name-on-HUD, (c) Name-on-gun (NG), (d) In-front (IF). The Icon-on HUD and Name-on HUD weapon displays weretested in two display positions, at the bottom right (IHR, NHR) and the bottom left (IHL, NHL).


52

8.2. Apparatus

The left analog stick controlled movement. See Fig. 4b. All othercontrols were disabled; the task only required navigating the maze.Navigating the maze required reaching four waypoints (represented bylarge red spheres floating slightly above the floor) located in the maze,

all separated by equidistant paths. The four waypoints were positionedalong the “optimal” path through the maze. Participants needed toreach all four waypoints in order to complete a trial, i.e., the lastwaypoint was positioned at the very end of the maze. The next way-point sphere would appear only after the previous waypoint wasreached. Participants simply had to walk through a waypoint to reachit. The maze had a grid of “tiles” – 1× 1m squares that made up themap. Distance was judged based on the number of tiles crossed by theparticipant – hence the number of tiles between each waypoint was thesame. The software recorded the number of tiles the participant crossedin reaching each waypoint and the time to get to each waypoint fromeither the start or the previous waypoint.

The experiment included five navigation displays: mini-map still(MMS), mini-map rotate (MMR), wayfinding arrow (WA), light pillar(LP), compass (C), and navigation line (NL). Each presented the sameinformation, but visualized it differently. See Fig. 21, which also depictsthe environment used in the experimental task.

Mini-maps and the wayfinding arrow are HUD elements commonlyused in recent FPS games (see Table 1). The mini-maps showed a top-down view of the participant position, the maze, and the position of thenext waypoint when it came into the mini-map range. With MMS, northwas always up on the mini-map; with MMR, the movement direction

Fig. 18. Shots with wrong weapon by weapon dis-play. Lower scores are better. Error bars show±1 SD.

Fig. 19. Weapon switch time by weapon display.Lower scores are better. Error bars show±1 SD.

Fig. 20. Subjective preferences for each weapon display, showing percentage of partici-pants giving each response.


53

was up. The mini-map rotated such that the player’s movement direc-tion (the green arrow in Fig. 21c) always pointed up. The wayfindingarrow (HUD) and compass (diegetic) always pointed to the next way-point. Such displays are sometimes used in FPS games. The navigationline (Fig. 21a) traced a path (visualized as a semi-transparent blue line)along the floor of the maze directly from the player’s position to thenext waypoint, turning corners as necessary. Since it was visualized inthe game world, it qualifies as a spatial element (as per Fig. 2). Thenavigation line was based on the navigation aid in the popular third-person shooter game Dead Space. Since this display shows the playerexactly how to navigate the maze to all waypoints, we included it as acontrol condition, as it was expected to offer optimal performance. Fi-nally, the light pillar is another spatial option based on navigation aidsused in some adventure titles, such as the Legend of Zelda. It emittedlight straight up from the position of the next waypoint, allowing na-vigation by a physical landmark, similar to navigating a real city usingphysical features such as tall buildings. We are unaware of any pre-cedent for such a display in existing FPS games, but included it to assessits potential.

8.3. Procedure

The environment used an outdoor maze (see Fig. 21a). Participantsnavigated to the end of the maze as quickly and accurately as possible.Accuracy meant minimizing wrong turns and avoiding wandering intounnecessary parts of the maze – in other words, minimizing the numberof tiles they crossed while completing the maze.

The same maze was used with all navigation displays, and wascompleted twice with each. For the first trial, participants navigated themaze in one direction, and in the second trial, they proceeded in thereverse direction. They were not informed that both trials took place inthe same maze. This was intended to avoid learning effects that couldoccur by using the same maze in the same direction in both trials.However, it also ensured that the complexity and distance were equal inboth trials, unlike using a different maze. To prevent participants get-ting lost and wandering endlessly, a 6-min time limit was used for eachtrial. Trials were marked as “incomplete” if the participant did notreach the maze’s end within the time limit.

Participants completed two trials for each of the six navigationdisplays, completing 12 trials in total. After each trial, participantscould take a break before continuing. Each trial took approximately 3

min, for a total experiment time of approximately 36min.

8.4. Design


Navigation Display: MMS, MMR, WA, LP, C, NLTrial: 1, 2

Navigation display order was counterbalanced with a Latin square.The dependent variables were step error rate (path efficiency ratio),waypoint time (in seconds), and incomplete trials (%). Step error ratewas calculated by dividing the actual number of tiles the participantcrossed in navigating to the next waypoint by the actual distance be-tween waypoints, 60 tiles. Waypoint time is the time to find eachwaypoint, measured from the start of the maze (for the first waypoint)or the previous waypoint (for subsequent waypoints). Incomplete trialsindicated the percentage of trials where the participant ran out of timebefore finding all waypoints.

Overall, the experiment included 12 participants× 6 navigationdisplays× 2 trials per navigation display= 144 trials in total.

8.5. Results

8.5.1. Waypoint timeWaypoint time indicated how long it took (on average, in seconds),

to find the next waypoint. Lower scores are better. Note that this de-pendent variable excludes incomplete trials – if the participant ran outof time, their waypoint times for that trial were not included in theaverage. Unsurprisingly, the navigation line (NL) performed best, withan average time of 21.6 s to reach each waypoint. The compass (C)performed worst, with an average time of 56.4 s to reach each way-point. This was about 62% worse than with the navigation line (NL).See Fig. 22.

There was a significant main effect of navigation display on time(F5,23 = 26.77, p < .0001). A Fisher LSD pair-wise analysis revealedthat the mini-map displays (MMR and MMS) and the navigation line(NL) were significantly faster than when using the wayfinding arrow(WA), light pillar (LP), and compass (C).

The main effect for trial was significant (F1,23 = 47.06, p < .0001),

Fig. 21. Navigation displays. (a) Task overview, showing navigation line display (b) mini-map still (c) mini-map rotate (d) compass (e) wayfinding arrow (f) light pillar.


54

as was the interaction between navigation display and trial (F5,23 =9.90, p < .0001). Participants generally completed the first trial fasterthan the second trial. Despite being the same map, participants oftenfound completing the maze in reverse direction more difficult than thefirst-trial maze, taking on average about 9 s longer to complete themaze in the second trial (reverse order). This is perhaps because the endof the maze (with respect to the first trial) was more complex than thestart, hence the reverse order began with a more complicated start.

8.5.2. Step error rateStep error rate yields a ratio of path efficiency, where the best result

is 1.0 (i.e., a participant who did not deviate from the optimal path).Scores higher than 1.0 indicate worse performance. See Fig. 23.

The results for step error rate and waypoint time were similar. Thenavigation line (NL) display (again) performed best, with an averagestep error rate of 1.03 to reach each waypoint. This is not unexpected,as the display shows participants exactly which way to go. The compass(C) again performed worst, with an average step error rate of 2.43 toreach each waypoint – in other words, participants traversed onaverage ≈2.5×more distance in the maze as required, clearly be-coming lost more often than with other displays. The variability wasalso high, suggesting the compass was highly ineffective for many

participants.There was a significant main effect for navigation display on step

error rate (F5,23 = 36.32, p < .0001). A Fisher LSD post hoc test re-vealed that the navigation line (NL) had significantly lower step errorrate than all other navigation displays except for the mini-map rotate(MMR). This is a promising result; the MMR display offered comparableperformance to the somewhat unrealistic and arguably unfair naviga-tion line. The compass (C) had significantly higher step error rate thanall other navigation displays except for the wayfinding arrow (WA).

The main effect for trial was significant (F1,23 = 132.24,p < .0001), as was the interaction between navigation display and trial(F5,23 = 5.27, p < .0005). Like time, participants performed worse inthe second trial, with an average step error rate (over all navigationdisplays) of 1.91 vs. 1.48 in the first trial.

8.5.3. Incomplete trialsTrials were marked “incomplete” if a participant took longer than 6

min to find all four waypoints, reaching the end of the maze. When thisoccurred it was typically because a participant was hopelessly lost andcould not find their way through the maze. The frequency of incompletetrials is expressed as a percentage of all trials in Fig. 24. A lower per-centage indicates a better performance. The navigation line (NL)

Fig. 22. Time by navigation display. Lower scores arebetter. Error bars show±1 SD.

Fig. 23. Step error rate by navigation display. Lowerscores are better. Error bars show±1 SD.


55

performed best – only 2% of trials with this navigation display wereincomplete. The compass performed worst, with 42% of trials in-complete.

There was a significant main effect for navigation display on in-complete trials (F5,23 = 18.29, p < .0001). A Fisher LSD pair-wise testindicated that both mini-maps and the navigation line (NL) all yieldedsignificantly fewer incomplete trials than the wayfinding arrow (WA),light pillar (LP), and compass (C). The wayfinding arrow (WA) andcompass (C) both yielded significantly more incomplete trials than theother navigation displays. It is interesting to note again that displayeffectiveness had little to do with whether the display was renderedspatially (e.g., within the game world) or not. Both spatial and tradi-tional HUDs are represented in the best and worst performers in thisexperiment.

8.5.4. Subjective resultsParticipants ranked their perceived effectiveness of each navigation

display on a 5-point Likert scale with responses ranging from “Reallyhindered” (1) to “Really helped” (5). As seen in Fig. 25, participantsfound NL most effective, followed by MMR and MMS. A Friedman non-parametric test found the results to be statistically significant (χ2 =71.94, p < .001, df=5). A post hoc analysis revealed significancebetween the following:

• MMS and WA, C, LP, and NL

• MMR and WA, C, and NL

• WA and MMS, MMR, LP, and NL

• C and MMS, MMR, LP, and NL

• LP and MMS, WA, C, and NL

• NL and MMS, MMR, WA, C, and LP

These results indicate that the navigation line received a sig-nificantly more positive result than any of the other navigation dis-plays. They also indicate that the mini-maps (MMS and MMR) received

statistically similar results, and that mini-map rotate (MMR) and lightpillar (LP) also received statistically similar results.

8.6. Summary

Results of this study indicate that, unsurprisingly, the navigationline, functionally equivalent to the diegetic navigation aid in DeadSpace, offered the best performance in terms of completion time andpath efficiency. However, the HUD-based mini-map options also didwell, coming close to the navigation line, and significantly better thanother HUD, diegetic, or spatial options. Notably, rotating the mini-map(i.e., forward is always up) worked better, despite participant pre-ference for the non-rotating variant.

9. Discussion

Globally, our results prevent us from being categorical as to whetherHUDs or alternatives offer better performance. In some cases, alter-native displays were best, while in other cases, HUDs were best.

Results of the ammunition experiment (#1) indicate that thenumber-in-game ammunition display was best in terms of how long ittook participants to recognize they were out of ammo. We expected thisis due in part to the placement of the display. Since it was co-locatedwith the player’s gun, no additional glancing to HUD elements wasrequired. We note that without eye tracking data, we cannot be certain;this is merely our suspicion based on the primary differences betweenthese displays. In particular, participants were able to effectively tracktheir ammunition while otherwise playing the game normally. Thissuspicion motivated us to include display position as a condition in thesecond experiment on health displays.

The health display experiment (#2) revealed that number-basedoptions reliably allowed participants to detect when their health waslow, successfully escaping the scenario before reaching 0 health. Thisresult is similar to the ammo experiment, where, as noted above, the“number-in-game” diegetic option performed best. In contrast to theammo experiment, alternative display options placed near the centre ofthe screen offered mediocre performance compared to HUDs. In thehealth experiment, display position seemed less important than displayvisualization. While there were large differences between types of dis-plays, positioning HUDS at the top, bottom, or side of the screen mat-tered less. We were surprised by this as we expected a central position(i.e., as presented by the diegetic options, and similar to the ammostudy) to reduce glancing to HUDs located on the periphery of thescreen. Unlike the ammo experiment, HUDs performed better than al-ternative displays. Participants also indicated that they found thesemore effective, likely due to the comparative precision. In particular,numeric display options were well suited to identifying when healthwas in a certain percentage range. Alternative methods – including lessprecise iconic and bar HUD options – simply did not offer the samedegree of precision as numbers.

Results of the weapon display experiment (#3) are more mixed. Thediegetic in-front display performed well, and was strongly preferred byparticipants. This was expected, as it not only centralizes the informa-tion, but utilizes an arguably immersive display for the weapon (i.e.,showing the weapon directly in front of the player as though they areholding it). In practice, all modern FPS games display the currentweapon this way – but they also frequently show a redundant icon orthe name of the weapon. To our surprise, and despite its redundantnature, such an icon display (icon-on-HUD) did very well, offering thebest performance of all displays in the experiment. This was unexpected– the icon presented the same information as the weapon in front, yetnecessitated extra glancing to effectively use it. In contrast, the weapon-in-front display was visible at all times. This may have improved per-formance because the redundant icon display provided participants asecond place to look and assess their current weapon, thus potentiallysaving time while glancing around the screen for enemies. Our findings

Fig. 24. Incomplete trials by navigation display. Lower scores are better.

Fig. 25. Subjective preferences for each navigation display, showing percentage of par-ticipants giving each response.


56

also suggest that the bottom-right corner is the best location for dis-playing HUD-based weapon information. This might be due to partici-pant familiarity with this option, as this location is commonly used forsuch icons in commercial games, as revealed in Section 3. However, wedid not exhaustively evaluate the effect of display position in this ex-periment. This result lends merit to the idea of combining diegetic andHUD-based options in other situations. Testing other combinations ofHUDs and diegetic displays is an opportunity for future work, poten-tially in all kinds of displays (health, ammo, etc.). The HUD-alternativedisplay option, “name-on-gun”, performed worse with participantsnoting that they did not like it. This could be due to the viewing angleof the text printed on the gun, or to the unusual nature of such a dis-play.

Results of the navigation aid experiment (#4) revealed that thespatial navigation display option offered the best performance for thistask, which is unsurprising, given the relationship between the in-formation presented and the task itself. The spatial (HUD-alternative)navigation line (NL) offered high performance due to the detailed pathinformation it provided. Like the mini-map options, the navigation lineprovided information about direction and distance, making it easy toturn through the maze. In contrast, the worst performing display – thecompass (C) – the only diegetic display type included in this test, of-fered less rich path information (only direction). This suggests the re-lationship (at least for navigation aids) between alternative displaytypes is limited. Instead, performance is determined by how well anoption suits the task. In this study, the navigation line (NL) was the bestdisplay for a maze. This may not be the case in other environments(e.g., an open environment). This is something we may wish to explorein future work.

9.1. Limitations and future work

The primary limitation of these experiments is that each displaytype was studied in isolation. This is appropriate from an experimentalcontrol point of view, and thus enhances the internal validity of theresults. However, it decreases the generality of the results. Most gamesshow multiple displays at once (e.g., see Fig. 1, usually combinations ofhealth, ammunition, weapon, and navigation displays. Sometimes,these even include combinations of diegetic displays and HUDs.Studying a single display in isolation is not fully representative of thismore complex task of monitoring multiple displays simultaneously.However, we expect that, with multiple displays present, those thatindividually demonstrated better performance are likely to offer betterperformance together. Hence, we believe studying multiple displaytypes in isolation is worthwhile to maintain high internal validity andto “chip away” at the more complex problem of monitoring multipledisplays at once. Future work will focus on this goal. For example, acompetitive “death match” style experiment using different combina-tions of displays could provide great insight into how competitive playis influenced by diegetic displays, HUDs, spatial representations, andmeta-perceptions.

As noted earlier, without eye tracking data, many explanations ofour results are somewhat speculative. Eye tracking could enable moreinsightful explanations – for example, that it was indeed the fact thatthe fixation point of the user’s eyes influenced performance with onedisplay or another. This is a consideration for a future study.

Finally, we note again that our experiments exclusively used “ex-pert” FPS gamers. There are two limitations here. First, expertise wasself-assessed by participants; it is difficult to objectively quantify ex-pertise, but on average, participant performance seemed fairly con-sistent. Second, these results likely do not apply to novice gamers. Asdescribed in the introduction, from an experimental control point ofview, this was likely the right choice, as it increases the likelihood ofdetecting significant differences due to inherent differences in the displaysstudied. That said, it decreases the generality (external validity) of ourresults.

10. Conclusions

Our results suggest that neither traditional HUDs nor alternativedisplay types (e.g., diegetic, spatial, etc.) are best suited to FPS games,but rather specific properties of a given display (i.e., word, icon,number, bar, etc.) and, to a lesser extent, its position (in-game, bottommiddle, top left, etc.) relative to the task at hand have an effect on playerperformance. In other words, a proper design methodology would firstidentify the most important information for player success in com-pleting game tasks, then choose the best method to display that in-formation effectively, and finally understand the best place to put theinformation display. This of course assumes that the game designer istrying to optimize player efficiency; if the designer is instead trying tooptimize “difficulty” (as apparently the designers of Dark Souls appearto strive for [9], then of course one could also use our results in reverse.For example, to make a game harder (e.g., via difficulty modes), adeveloper could use less effective information displays, or vice versa.This could also be adjusted to the skill level of the player, similar to asuggestion by Iacovides et al. [16].

Our results do not support existing recommendations to use alter-native displays whenever possible [10]. Instead, our results indicatethat HUD alternatives are not inherently the best option when con-sidering player performance and preference. The game world and tasksshould be the key factors in selecting an information display method,and not immersion or following current trends. For example, while it iscommonly used, splatter as an indicator of health should be avoided, asit hampers players much more than aesthetic considerations can justify.Our results also suggest a strong relationship between performance andperceived performance (preference) – participants fairly consistentlyidentified the better display options presented to them. In general,participants were well aware of which displays let them to performbest, and preferred these over less effective display types.

Finally, we speculate that the over-emphasis on diegetic and alter-native displays in the literature might be due to a certain onlooker effect,where theories of game design are arrived at by “looking at” a game (asa reviewer would, but also as spectators), rather than through the lensof the player’s experience. For someone who has the leisure to lookaround the game world at their own pace, of course HUD alternativeswill be much more satisfying; our results show that from the players’perspective, this may not matter at all, and in some cases, their abilityto perform well in the game might even be hampered by such a design.Of course, there are game genres, such as e-sports, where one couldargue that spectators are as important as players – and in these games,aesthetic considerations may weigh in as importantly as player effec-tiveness. Our results suggest that the game designer should pay closeattention to their intended audience, and tune the game UI to optimizethe audience's experience.

Acknowledgements

This work was supported by the Canada Foundation for Innovation,and the Natural Sciences and Engineering Research Council of Canada.

References

[1] E. Adams, Fundamentals of Game Design, 2nd ed., California, New Riders, 2010.[2] J. Babu, Video game HUDs: information presentation and spatial immersion.

Rochester, NY, Golisano College of Computing and Information Sciences, RochesterInstitute of Technology, Master of Science, 2012.

[3] D. Bordwell, K. Thompson, Film Art, McGraw Hill, New York, 1993.[4] B. Bowman, N. Elmqvist, T. Jankun-Kelly, Toward visualization for games: theory,

design space, and patterns, IEEE Trans. Visual Comput. Graph. 18 (11) (2012)1956–1968.

[5] E. Brown, P. Cairns, A grounded investigation of game immersion, ExtendedAbstracts of the 2004 Conference on Human factors in Computing Systems, Austria,ACM, 2004.

[6] L. Caroux, K. Isbister, Influence of head-up displays’ characteristics on user ex-perience in video games, Int. J. Hum Comput Stud. 87 (2016) 65–79.

[7] L. Caroux, L. Le Bigot, N. Vibert, Maximizing players’ anticipation by applying the


57

http://refhub.elsevier.com/S1875-9521(17)30043-5/h0005








proximity-compatibility principle to the design of video games, Human Factors 53(2) (2011) 103–117.

[8] D. Conroy, P. Wyeth, D. Johnson, Understand player threat responses in FPS games,in: Proceedings of the 9th Australasian Conference on Interactive EntertainmentMatters of Life and Death, Melbourne, ACM, 2013.

[9] A. Donaldson, Dark Souls 3: Miyazaki explains the difference between “difficult”and “unreasonable”, Designer Interview, 2016, URL:< https://www.vg247.com/2016/03/02/dark-souls-3-miyazaki-bloodborne-interview/ > (Accessed November26, 2017).

[10] E. Fagerholt, M. Lorentzon, Beyond the HUD – user interfaces for increased playerimmersion in FPS games, Dept. Comp. Sci and Eng. Göteborg, Sweden, ChalmersUniv. of Tech. M.S., 2009.

[11] M.A. Federoff, Heuristics and usability guidelines for the creation and evaluation offun in video games, Dept. of Telecommunications, Indiana University, M.S., 2002.

[12] S. Fragoso, Interface design strategies and disruptions of gameplay: notes from aqualitative study with first-person gamers, Human-Computer Interaction.Applications and Services. M. Kurosu, Springer International Publishing, 2014, vol.8512, pp. 593–603.

[13] A.R. Galloway, Gaming: Essays on Algorithmic Culture, University of MinnesotaPress, Minneapolis, 2006.

[14] D.J. Hill, Call of Duty Video Games Reaches $1 Billion In Sales In 16 Days, FasterThan Cameron’s Avatar, Retrieved December 9, 2014, from<http://singularityhub.com/2012/01/19/call-of-duty-video-game-reaches-1-billion-in-sales-in-16-days-faster-than-camerons-avatar/ > .

[15] T. Hynninen, First-person shooter controls on touchscreen devices: a heuristicevaluation of three games on the iPod touch. Department of Computer Sciences.Tampere, Finland, University of Tampere. M.Sc. Thesis, 2012, 64 pages.

[16] I. Iacovides, A. Cox, R. Kennedy, P. Cairns, C. Jennett, Removing the HUD: theimpact of non-diegetic game elements and expertise on player involvement, in:Proceedings of the ACM symposium on computer-human interaction in play – CHIPlay 2015, ACM, London, United Kingdom, New York, 2015, p. 13–22.

[17] D. Ignacio, Crafting destruction: The evolution of the Dead Space user interface,Game Developer's Conference, 2013. URL:< https://www.youtube.com/watch?v=pXGWJRV1Zoc> (Accessed November 26, 2017).

[18] P. Isokoski, B. Martin, Performance of input devices in FPS target acquisition, in:Proceedings of the international conference on advances in computer entertainmenttechnology, Austria, ACM, 2007.

[19] C. Klochek, I.S. MacKenzie, Performance measures of game controllers in a three-dimensional environment, in: Proceedings of Graphics Interface 2006, CanadianInformation Processing Society, Toronto, 2006.

[20] S.C. Llanos, K. Jørgensen, Do players prefer integrated user interfaces? A qualitativestudy of game UI design issues, in: Proceedings of the 2011 DiGRA internationalconference: think design play, Hilversum, Netherlands, DiGRA, 2011.

[21] J. Looser, A. Cockburn, J. Savage, On the validity of using First-Person Shooters forFitts' law studies, Proceedings of the British HCI Conference, Springer, 2005.

[22] I.S. MacKenzie, Human-Computer Interaction: An Empirical Research Perspective,Morgan Kaufman, 2012.

[23] V. McArthur, S.J. Castellucci, I.S. MacKenzie, An empirical comparison of“Wiimote” gun attachments for pointing tasks, Proceedings of the ACM Symposiumon Engineering Interactive Computing Systems – EICS 2009, ACM Press, 2009, pp.203–208.

[24] E. McDonald, The global games market will reach $108.9 billion in 2017 withmobile taking 42%. Newzoo Global Games Market Report, April 2017.URL:<https://newzoo.com/insights/articles/the-global-games-market-will-reach-108-9-billion-in-2017-with-mobile-taking-42/ > (Accessed November 26, 2017).

[25] R.P. McMahan, E.D. Ragan, A. Leal, R.J. Beaton, D.A. Bowman, Considerations forthe use of commercial video games in controlled experiments, Entertain. Comput. 2(1) (2011) 3–9.

[26] D. Natapov, I.S. MacKenzie, Gameplay evaluation of the trackball controller, in:Proceedings of the international academic conference on the future of game designand technology – FuturePlay 2010, ACM, New York, 2010.

[27] D.A. Norman, The Design of Everyday Things, Basic Books, New York, 2002.[28] R.J. Pagulayan, K. Keeker, D. Wixon, R.L. Romero, T. Fuller, User-centered design in

games, in: J. Jacko, A. Sears (Eds.), Human-Computer Interaction in InteractiveSystems, Lawrence Erlbaum Associates Inc, 2002, pp. 883–906.

[29] M. Peacocke, R.J. Teather, J. Carette, I.S. MacKenzie, Evaluating the effectivenessof HUDs and diegetic ammo displays in first-person shooter games, in: Proceedingsof the IEEE Consumer Electronics Society Games, Entertainment, and MediaConference – GEM 2015, IEEE, Toronto, 2015, p. 138–145.

[30] N. Schaffer, Heuristics for usability in games white paper, Retrieved April 3, 2015,from<https://gamesqa.files.wordpress.com/2008/03/heuristics_noahschafferwhitepaper.pdf> .

[31] R.J. Teather, I.S. MacKenzie, Comparing order of control for tilt and touch games,in: Proceedings of the 2014 conference on interactive entertainment, ACM,Newcastle, NSW, Australia, 2014, p. 1–10.

[32] R. Vicencio-Moreira, R.L. Mandryk, C. Gutwin, S. Bateman, 2014. The effectiveness(or lack thereof) of aim-assist techniques in first-person shooter games, in:Proceedings of the 32nd annual ACM conference on Human factors in computingsystems, ACM, New York, NY.

[33] R. Weber, NPD: 2015 video game sales flat compared to 2014, Retrieved March 9,2016, from<http://www.gamesindustry.biz/articles/2016-01-14-npd> .

[34] G. Wilson, Off with their HUDs!: Rethinking the Heads-up display in console gamedesign, 2006 (Retrieved November 12, 2014)< http://www.gamasutra.com/view/feature/130948/off_with_their_huds_rethinking_.php> .

[35] V. Zammitto, Visualization techniques in video games, in: Proceedings of ElectronicInformation, the Visual Arts and Beyond, London, UK, BCS, 2008.

[36] A. Zaranek, B. Ramoul, H.F. Yu, Y. Yao, R.J. Teather, Performance of moderngaming input devices in first-person shooter target acquisition, CHI'14 ExtendedAbstracts on Human Factors in Computing Systems, ACM, 2014.


58



https://www.vg247.com/2016/03/02/dark-souls-3-miyazaki-bloodborne-interview/

https://www.vg247.com/2016/03/02/dark-souls-3-miyazaki-bloodborne-interview/



http://singularityhub.com/2012/01/19/call-of-duty-video-game-reaches-1-billion-in-sales-in-16-days-faster-than-camerons-avatar/



https://www.youtube.com/watch?v=pXGWJRV1Zoc

https://www.youtube.com/watch?v=pXGWJRV1Zoc









https://newzoo.com/insights/articles/the-global-games-market-will-reach-108-9-billion-in-2017-with-mobile-taking-42/

https://newzoo.com/insights/articles/the-global-games-market-will-reach-108-9-billion-in-2017-with-mobile-taking-42/








https://gamesqa.files.wordpress.com/2008/03/heuristics_noahschafferwhitepaper.pdf

https://gamesqa.files.wordpress.com/2008/03/heuristics_noahschafferwhitepaper.pdf

http://www.gamesindustry.biz/articles/2016-01-14-npd

http://www.gamasutra.com/view/feature/130948/off_with_their_huds_rethinking_.php

http://www.gamasutra.com/view/feature/130948/off_with_their_huds_rethinking_.php

An empirical comparison of first-person shooter ... · In ﬁrst-person shooter (FPS) games, the...

Documents

Transcript of An empirical comparison of first-person shooter ... · In ﬁrst-person shooter (FPS) games, the...