Integrating learning analytics

into a game authoring tool

Ivan J. Perez-Colado, Victor M. Perez-Colado, Manuel Freire-Moran,Ivan Martinez-Ortiz, Baltasar Fernandez-Manjon

Dept. of Software Engineering and Artificial Intelligence, Facultad de Informática,Universidad Complutense de Madrid Madrid, Spain

ICWL 2017: 16th International Conference on Web-based Learning

Introduction: Game learning analytics

● Game analytics (GA): collecting and analyzing videogame user interactions to

generate a better insight of the game experience

● Learning analytics (LA): analysis of user’s interactions with educational purposes

● Game learning analytics (GLA) = GA + LA → better evidence-based

use of the games in education

● GLA is a complex ad-hoc process specific for each game and

each analytics system.

GLA role in serious games lifecycle

● Game learning designers to validate their designs

● Provide formative and summative evaluation

ANALYTICS MODEL

Simplifying GLA in games

● Applying GLA would be greatly simplified by having built-in

support within the game authoring tools.

● Linking the game model with the analytics model and

automatically generating traces → Reduces costs

● We are providing integrated GLA support into uAdventure, our

game authoring tool.

xAPI Serious Games application profile

New standard interactions model

developed and implemented in

Experience API (xAPI) by UCM with ADL

(Ángel Serrano et al, 2017).

The model allows tracking of all

in-game interactions as xAPI traces

(e.g. level started or completed,

interactions with NPC or game items,

options selected, score increased)

https://www.adlnet.gov/serious-games-cop

Game Learning Analytics infrastructure

Game trackers

Cloud GLA server

infrastructure

Realising an Applied Gaming Eco-System

Systematization of Analytics Dashboards

As long as traces follow xAPI format, these analysis

and visualizations do not require further

configuration!

Also possible to extend with game-dependent

analysis and visualizations for specific games and

game characteristics.

An Analytics model for uAdventure

● uAdventure (uA) is a serious game authoring tool

○ built on top of the Unity game platform

● xAPI profile for SG is focused on:

○ Completables: level, quest, race...

○ Alternatives: menus, questions, path,

dialogue choices...

○ Meaningful Variables: score, health, or

game-dependant variables.

○ Custom Interactions: accessed into an area,

used an item, or interacted with a character.

Meaningful

Variables

Completables

Alternatives

Custom

Interactions

An Analytics model for uAdventure

● xAPI traces are composed of: Actor, Verb, Object and Result.○ For example: “Dan has completed the level 1 winning in 2 minutes”

○ in xAPI for serious games:{

"actor": { "name": "Dan" },

"verb": {

"id": "http://adlnet.gov/expapi/verbs/completed"

},

"object": {

"id": "1",

"definition": {

"type": "https://w3id.org/xapi/seriousgames/activity-types/level"

}

},

"timestamp": "2017-09-18T14:55:56.688Z",

"result": {

"success": true,

"extensions": {

"time": 120

}

}

}

Session Events

● Analytics servers need to identify players:

○ Logged players: Can be connected with an LMS

○ Pseudonymous players: Can continue a game in multiple sessions

○ Anonymous players

● Authentication and Authorization are provided by the GLA

server.

● uAdventure supports Login (For non-anonymous use), with an

small login form.

User Interaction Events

● Raw input events: mouse tracking, clicks and keyboard inputs○ Can be grouped to extract meaning.

● Game progress or interaction with scene elements are much

more valuable

● Automatically uAdventure does:○ Send “Interacted” SG-xAPI traces whenever a player interacts with an in-

game element.

○ Identifies: type of interaction (such as “examine” or “use”)

Scene and Cutscene changes

● Scenes: provide backdrops within which the player moves and interacts

with items and other characters.

● Automatically uAdventure does:

Send “Interacted” SG-xAPI traces whenever a player interacts with an in-game

element. And Identifies the type of interaction (such as “examine” or “use”)

● What uAdventure lets the user is to change what that scene is:

Meaningful Variables

The game state is represented by variables and flags.

● Therefore changes in variables can be relevant for the analysis.

However, xAPI is a action based standard.

✘ A change in a variable itself is not an action, but a consequence.

✓ The change is appended (as extension) to the corresponding origin action.

Puzzle = false Puzzle = true

/w

&

Puzzle = true

Alternatives

An alternative represent a selection or choose between different options in the game.

Mainly in uAdventure we can directly identify:

● Choices in Dialogs:

They can be manually marked as

correct or incorrect and have an ID.

● Exit selection in Scenes:

Scenes can be redefined

as an Alternative of exits.

✘Not exiting = Incorrect

✓ Exiting = Correct

Different exits as choicesQuestion scene spec.

Different responses as choices

Completables

They might be finished at some point... Right?

Completables

A completable is a task to be solved in the game => It defines the game progress.

● Game, stages, missions, quests, minigames, etc. are examples of completables.

In uAdventure these concepts are embedded into the game story.

● A superstructure that groups the actions and milestones is supports the process of

identifying when this progress happens and what parts have been completed.

○ This is managed in the uAdventure Analytics tab for Completables

■ All the completables in a game define the general progress.

Start

End

Main

Progress

Completable progress popup

Variables

identify the

score

Milestone editor popup

The completable progress popup can be used to

associate a progress for each milestone that has been

completed inside of a completable.

Completables

The milestone editor popup

shows the author different

choices to identify the

condition that fires the

milestone and configurations.

If condition is chosen, game

variables can be used freely.

Also, completables can be completed in parallel or even be repeated several times.

Mapping game events to SG-xAPI

● Complete summary of every trace uAdventure sends

● From game Events to xAPI traces

● Utility from LA point of view:

○ The closer to the interaction with game events

○ The lesser meaningful than the other ones connected to game progression

Map

pin

g g

am

e e

ven

ts to S

G-x

AP

I

Discussion and Conclusions

● Few serious games formally evaluated but we propose a new

evidence-driven model, mapping in-game actions to events.

● The uAdventure analytics model describes how player

interactions are automatically captured and transformed into

events that the GLA system can collect and analyze.

● uAdventure + GLA presents multiple benefits to game

developers

● Automatically integrate free support for LA, and can take

advantage of several generic analyses and visualizations in SG

xAPI-aware GLA platforms.

References

● M. Freire, Á. Serrano-Laguna, B. M. Iglesias, I. Martínez-Ortiz, P. Moreno-Ger, and B.

Fernández-Manjón, “Game Learning Analytics: Learning Analytics for Serious Games,” in

Learning, Design, and Technology, Cham: Springer International Publishing, 2016, pp. 1–29.

● I. Perez Colado, V. Perez Colado, I. Martínez-Ortiz, M. Freire, and B. Fernandez-Manjon,

“uAdventure: The eAdventure reboot - Combining the experience of commercial gaming tools and

tailored educational tools,” IEEE Glob. Eng. Educ. Conf., no. April, pp. 1754–1761, 2017.

● J. Torrente, A. del Blanco, E. J. Marchiori, P. Moreno-Ger, and B. Fernandez-Manjon, “<e-

Adventure>: Introducing educational games in the learning process,” in IEEE EDUCON 2010

Conference, 2010, pp. 1121–1126.

● A. Serrano-Laguna, I. Martinez-Ortiz, J. Haag, D. Regan, A. Johnson, and B. Fernandez-

Manjon, “Applying standards to systematize learning analytics in serious games,” Comput. Stand.

Interfaces, vol. 50, no. September, pp. 116–123, 2016.

● “xAPI - ADL Net @ www.adlnet.gov.” .

● Á. Serrano-laguna, J. Torrente, P. Moreno-ger, and B. Fernández-manjón, “Tracing a little for big

Improvements : Application of Learning Analytics and Videogames for Student Assessment,”

Procedia Comput. Sci., 2012.

● C. Alonso-Fernandez, A. Calvo, M. Freire, I. Martinez-Ortiz, and B. Fernandez-Manjon,

“Systematizing game learning analytics for serious games,” 2017 IEEE Glob. Eng. Educ. Conf.,

no. April, pp. 1106–1113, 2017.

Want to be involved in uAdventure?

check our Github repository

https://github.com/e-ucm/uadventure

Any questions?- Mail: balta@fdi.ucm.es

- Twitter: @BaltaFM

- GScholar: https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en

- ResearchGate: https://www.researchgate.net/profile/Baltasar_Fernandez-Manjon

- SlideShare: https://www.slideshare.net/BaltasarFernandezManjon

ICWL 2017