Games thatunderstandand adapt

Julian Togelius

As you play a game, you learn more about the game

To play the game well, you need to understand it

You need to adapt your behaviour to play the game successfully

Why not theother way around?

As you play a game, the game learns more about you

For the game to play you well, it needs to understand you

The game needs to adapt its mechanics in order for the playing to be successful

As you play a game, the game learns more about you

For the game to play you well, it needs to understand you

The game needs to adapt its mechanics in order for the playing to be successful

Whatever that m

eans

Games should understand us

•We spend hours upon hours concentrating on the game......and acting in it!

•There’s no reason for the game not to record and analyze everything we do

•The game should know how we play better than we do ourselves

We need adaptation

•The gaming demographic is becoming ever more diverse - no longer just pimpled teenagers

•Preferences diverge and abilities diverge

•Development budgets for AAA games grow ever higher

•The same game will have to appeal to more different people

We need adaptation

Would you like a teacher, or a playmate, that did not care what you did or what you liked?

Why don’t we have this?

The reality

Most gaming devices are always

connected

Most games are multiplayer

World of Warcraft Armory

•Free, open interface to the servers for the world’s largest massively multiplayer game

•Queries can be made about attributes of characters (experience, level, equipment, calendars) and guilds

•12 million subscribers, most of them putting substantial time into the game

A typical AAAconsole game

•~1 million players

•At the end of each level, the game transmits information to the publisher’s servers

•Score received, number of items collected

•Which positions visited, for how long

•Options chosen, not chosen

•Shots fired, jumps jumped etc...

Tomb Raider: Underworld

AAA title Published by Square Enix 2009.

Data from hundreds of thousands of players available; 10.000 selected for analysis.

Tomb Raider: Underworld

•Self-organizing maps used to cluster player behaviour

•Four major behaviour clusters found, interpreted as four playing styles

QuickTime™ and aJVT/AVC Coding decompressorare needed to see this picture.

Tomb Raider: Underworld

•Features from levels 1 and 2 (of 7), including time spent at locations

•We can predict when the player will stop playing with 77% accuracy

•We can also predict total time spent on the game

A. Drachen, A. Canossa, G. N. Yannakakis, Player Modeling using Self-Organization in Tomb Raider: Underworld, in Proc. of IEEE CIG, 2009.T. Mahlmann, A. Drachen, J. Togelius, A. Canossa, G. N. Yannakakis, Predicting Player Behaviour in Tomb Raider: Underworld CIG 2010

Social sciencedata gathering

•Tens or hundreds of participants

•Tens (or fewer) questions per participant

•Non-negligible loss of respondents

•Deception, self-deception, misestimation

•Expensive

In-game automaticdata gathering

•Thousands or millions of participants

•Hundreds or thousands of features tracked per player

•Negligible loss of respondents

•Low noise

•Cheap

This is relevant data

•Results of conscious human actions (unlike much of e.g. sensor network data)

•Playing a game requires diverse cognitive skills

•It is highly likely that in-game behaviour correlates with real-world behaviour

Assassin’s Creed II

•Gameplay sessions recorded and annotated

•Personality tests administered, cultural background probed

Work in progress...

•Correlations found between aspects of gameplay and personality traits

•similar results found in Neverwinter Nights

•Some clear differences between players from different countries

What could we learn about a

player from his/her playing style?

•Personality? Personality disorders?

•Cultural background?

•Political views? Sexuality?

•Propensity for criminal behaviour?

Do we have the methods?

•The data format is typically not conducive to answer straightforward questions

•Linear correlations of major features will only scratch the surface

Data types and methods•Sets of actions taken, items collected

etc: frequent itemset mining

•Temporal sequences of events:sequence mining, recurrent neural nets

•Spatial data, in particular movement:clustering, edge clustering

•Player preferences and affect:neuroevolutionary preference learning

...to be combined with classification methods such as decision trees

Social groups in WoW

Christian Thurau and Christian Bauckhage. Analyzingthe Evolution of Social Groups in World of Warcraft. CIG 2010.

Analyzed with non-negative matrix factorization

Heroes of Newerth

•Multi-player real-time strategy / adventure hybrid

•Replays available online

Heroes of Newerth

•Question: can we find out how – where and in what context – actions of a particular type were frequently performed?

•SPADE to find frequent 3-sequences

•Clustering starting/ending positions

•Edge clustering to visualize

Video by Emil Kastbjerg

Adaptation

Generic Generic player player modelmodel

Single Single player player modelmodel

PlayPlayerer

PlayPlayerer PlayPlay

erer

PlayPlayerer PlayPlay

erer

GameGame

Adaptation Adaptation mechanismmechanism

OfflineOnline

Adaptation through content generation

Capturing player experience

What can we adapt?

•Simple parameters (game speed, number of enemies, money)

•Levels, maps, tracks

•Quests, NPC characters

•Rules

•Reward schedules?

Addiction science

Adaptive levels forSuper Mario Bros

•Player experience model 73-92%

•Level generation competition organized

Player Experience(fun, frustration, anxiety, …)

Level features and rules, playing behavior

C. Pedersen, J. Togelius, G. N. Yannakakis., Modeling Player Experience for Content Creation IEEE TCIAG, 2010

Evolving racing tracks

•Simulation-based fitness: player performance

•Offline content generation

•Content representation: b-spline parameters

J. Togelius, R. De Nardi, and S. M. Lucas, Towards automaticpersonalised content creation in racing games, IEEE CIG, 2007.

Procedural map generation for RTS

J. Togelius, M. Press, N. Beume, S. Wessing, J.Hagelback, and G. N. Yannakakis., Multiobjective Exploration of the StarCraft Map Space, IEEE CIG, 2010

Emotionally adaptive camera

•Player experience model (accuracy 76-88%)

Player Experience(fun, frustration, anxiety, …)

Camera controls, physiological signals,Player behavior

G. N. Yannakakis, H. P. Martinez, and A. Jhala Towards Affective Camera Control in Games , UMUAI, 2010 (to appear).

Physiology and physical play with

Wii

Player Experiencefun, frustration, anxiety,

challenge, predictability, boredom

Movement (acceleration)Physiology (HR, BVP, SC)

Game (e.g. speed)

D. Dimovska, P. Jarnfelt and S. Selvig, Towards Tailoring Player Experience in Physical Wii Games using Physiological and Gesture Data, in Proc. of ACE, 2009.

PCG and Wiihabilitation

• Dynamic difficulty adjustment

• Content = gates

• Content evaluation - player performance

D. Dimovska, P. Jarnfelt and S. Selvig, G. N. Yannakakis, TowardsProcedural Level Generation for Rehabilitation, in Proc. of FDG, 2010

Physical interactive games•Playware

platform: augmented reality playground game for kids

•Bugsmasher, space invadersG. N. Yannakakis, J. Hallam, Real-Time Game Adaptation for

Optimizing Player Satisfaction, IEEE TCIAIG, 2009.

Games for Health

•Treat early-stage PTSD via game-based CBT

•Adaptation to individual triggers and symptoms

Siren: Serious games for conflict

resolution•Design games for teaching social skills in schools

•User (cognitive, affective, cultural) modeling

•User preferences, Natural interaction (face, speech, head-pose)

•PEM-driven adaptive quest (conflict scenario) generation

Evolving game mechanics•Game rules for 2D grid-world

games represented as an integer vector

•Can express games such as Pac-Man

•Simulation-based fitness function: evaluate learnability by learning to play the game with reinforcement learning

•Extension to strategy game rules

J. Togelius and J. Schmidhuber. An Experiment in Automatic Game Design. IEEE CIG 2008

Further reading

G. N. Yannakakis and J. TogeliusExperience-driven Procedural Content GenerationIEEE Transactions on Affective Computing, 2011 (in press but available online)