AI forExperience Management

Context

•We talked last week about tailoring games to

specific players through content generation.

• How else can we drive engagement with the player?

• Talking today about ways the player engages with

games.

‣ How we can use AI to aid that process.

2

Monopoly

• Imagine you’re winning a game of Monopoly with

some friends.

• You are playing the role of a successful business

tycoon with a giant stack of cash.

• Couldn’t you put a bounty on your opponents?

• Hire arsonists to drive property value down?

3

Limited Interactions

• Games typically use a fixed set of rules.

• Limits how we can interact with the game world.

• Constrained by how the designer envisaged the

game be played.

• Inability to step beyond the game rules.

• In many cases the way to win is to think like the

designer.

4

Designer-Based Reasoning

5

The Greatest Game Ever(...Maybe)

6

What isDungeons and Dragons?

• Tabletop role playing game.

• Pen and paper (and imagination) based.

• Groups take the role of adventurers in a Lord of

the Rings style setting.

• Games may take hours to play, usually based around

some sort of quest structure.

‣ E.g. Retrieve the sword of whoever from the tomb of

whoever.7

The Dungeon Master

•One player takes the role of the DM

• In charge of running the game

‣Narrator

‣ Arbitrator

‣ “AI” system for enemies

• Responsible for ensuring that

8

Interactions in D&D

• Dungeons and Dragons doesn’t have rules per se

• Defines a framework for interaction.

• DM can interpret the framework to suit unseen /

unplanned interactions.

‣ A player wants to climb the dungeon wall looking for

secret passages.- This will require an athletic skill check and a dungeoneering

check

- Key is to let the players improvise and say “Yes and...”9

Zork

•Old (oldoldold) school game from the 70s

• Textbased adventure game with text input

• Incorporates some language parsing systems.

‣ “Hit troll” vs “Hit the troll with the Elvish sword”

‣ Parses a variety of verbs, nouns and conjunctions

• Good HCI

• Still a very constrained human/game interaction

10

Starship Titanic

• Game from 1998 designed by Douglas Adams

• Starship Titanic crashes into player’s house, player

boards to try to figure out cause of disaster

• Mystery game in the style of Monkey Island

• Complete freedom to interact with characters using

regular English. Robust parsing and response system

• Tight constraints on lateral thinking

11

Scribblenauts

• 2009 for Nintendo DS (Scribblenauts Remix - 2011)

• Casual puzzle game based around collecting stars

• In order to collect the stars, solve puzzles.

• Solution generally involves “summoning” an object.

•Objects have distinct characteristics.

12

Scribblenauts

13

Scribblenauts

14

Scribblenauts

15

Scribblenauts

16

Scribblenauts

17

Scribblenauts

18

Scribblenauts

• 22,802* objects in the Scribblenauts database

•Object characteristics

‣ Art

‣ Health

‣ AI Behaviour

‣ Animations

‣ Interactions with other objects

• Adjectives give scope for lots of customisation

19

Scribblenauts

• Using the object database frees the player to tackle

problems however they want.

• In effect, the database becomes the framework for

interactions.

• Sufficient variety that the player doesn’t perceive

constraints.

• Encourages lateral thinking

20

Scribblenauts

21

Scribblenauts

22

Scribblenauts

23

Scribblenauts

24

Scribblenauts

25

Scribblenauts

26

Scribblenauts

27

Freeform Interaction

• Allows the player to solve problems in their own

way.

• Massive sense of satisfaction.

• Moves away from “thinking like the designer”

•Works in Scribblenauts primarily because of

simplistic environment

28

Immersive Worlds

•We can engage with Scribblenauts well because,

although simple, the depth of the world makes it

immersive.

• Limits interactions in other ways

‣No unfolding story

‣No characters to interact with

•What if we looked at breadth of engagement rather

than depth?29

Skyrim

• Mentioned Skyrim last week.

• Amazing breadth to the world.

‣ Talked about the option to join Thieves Guild group.

- Also Companions, Dark Brotherhood, Bard’s guild, College of

Magic.

‣ 18 Skill disciplines, each with ~10 individual “perks”

‣ About 300 locations within the world to explore

• Massive “living” world to play in

30

Skyrim - Economy

• Cities in Skyrim maintain a (simplistic) economic

model.

• Affects availability of merchandise, amount of money

NPCs have and prices of goods.

• Player can affect the economy by building up (or

tearing down) local business.

31

Detour

• “Raid on Bungeling Bay” (1984) involved a

helicopter attacking factories in a city.

• City infrastructure was intricately simulated.

• Disrupting supply lines could cripple infrastructure

and in turn, limit military forces counter-attacking

• Level editor for this game would later be released

as “Sim City”.

32

Skyrim - Story

• Skyrim has multiple plots that run in parallel.

•Option to progress whichever you feel like.

• Each group has a storyline as well as Radiant-

generated side quests.

• Different plots can influence each other‣ One character appears in two plots, one as a core character,

another as a guest at a party

‣ Extra interaction options at the party if you already know her

33

Skyrim - Consequences

•Quests and conversations have impact.

• Positive and negative responses and attitudes.

‣ Characters build up a model of the player

• Callbacks - the outcome of a quest will be

referenced at a later date.

34

Skyrim - Conclusions

• The HGI here is not sophisticated.

• Relies heavily on designer-led thinking.

‣ Can only interact with what has been anticipated

• Few solutions to problems based on lateral thinking.

‣ Example - Can’t make a pile of brooms into a ladder

• Implicitly a shallow experience.

• Compensates by providing a very broad diverse

world with consequences - playground of shallow.35

Immersive Worlds

• Immersive worlds engage the player in a way games

like Scribblenauts can’t.

•We can virtually “inhabit” worlds.

• Emotional attachment to characters, locations and

events.

• The plot’s narrative becomes secondary to the

player-centric narrative of the world.

36

Interactive Fiction

• Iteractive Storytelling / Dynamic Narrative

• The story adapts to the responses from the player

• Player’s actions can shape the direction of the story

•Where AI meets Storytelling

37

Choose Your Own Adventure

• Basic version of this is the Choose Your Own

Adventure books

• Page or so of text to read, then a few choices of

next action.

• Associated page number, reader skips to that page

• Rinse and repeat through a plot til an end is

reached.

38

Story Beats

• A “Story Beat” is a literary device.

• A small piece within a scene with an exchange

between the characters and an action / revelation

that advances the story.

• Story > Act > Scene > Beat

• Smallest unit of story

39

AI for Interactive Fiction

•What we’re talking about with IF is :

• Automating the Choose Your Own Adventure

process.

• Making it more subtle than “Turn to Page 14 to fall

down the giant pit”.

• Creating beats on the fly.

40

Branching Narrative

• Branching narrative is most similar to choose your

own adventure.

• Points in the story require the player’s input

• Determines the direction the story will take.

• Effectively a sequence of if/else

41

Branching Narrative

• Combinatorial explosion is a big factor here.

• If every choice has just 2 options, at the kth decision

‣ 2k possible states

•We’ve seen this before in game trees, state spaces

and planning.

42

Branching Narrative

•We can use procedural content systems to lessen

the amount of work to be done.

• Do we need 3 different versions of one beat which

alter just one line dependent on minor actions?

• Can we dynamically tweak a beat at runtime?

• Effectively, this is procedural content customised

based on the player’s history ingame.

43

Multi-Linear Narrative

• Typically media has a single linear narrative.

• There may be multiple arcs at play, but a single story

is being told.

• Games allow us to interact.

• Multiple plot lines can be set up

‣We can explore at will

• Progress (or lack of) in one can affect another.

• Each one may itself be branching44

Interactive Fiction as Planning

• Maria introduced Planning in the first half of the

course.

• Recap

‣ Planning is the process of finding a sequence of actions

that will transform a world from a given initial state into a

state in which certain goals are true.

•What if we don’t talk about actions?

45

Planning with Beats

• Let’s assume we have a pre-made set of Story Beats

•We can see a narrative as a sequence of beats

stitched together.

• A traditional linear narrative is a standard planning

problem.

‣ Characters X and Y start in this situation

‣We want to find a sequence of beats that puts them into

this final situation.46

Planning with Beats

•We want to get the player to a specific point in the

story.

•We want the player to be free to choose their own

path.

•We always want to be guiding the player back

toward the point we’re trying to get to.

• This is not so straightforward.

47

Plan Repair

• Many assumptions related to Automated Planning

‣When we execute a plan, it probably won’t work

•We have a whole set of research ready to go on

“what to do when things don’t work out”

• Effectively this is the same situation as when the

player doesn’t follow the path we are looking for.

48

Drop-In Drop-Out Narrative

• A big goal for narrative is the ability to drop-in and

drop-out and still have the story make sense.

• Consider games that offer co-operative story lines.

‣ Halo 3, Gears of War

• Typically require a constant companion that is

controlled by the computer in single player.

• Another person can assume control.

49

Drop-In Drop-Out Narrative

• Consider an online game such as WoW.

• Large raids involve up to 40.

•What happens when a sub-group is late?

•What happens when a group has to leave early?

50

Drop-In Drop-Out Narrative

• Technique that allows for customisation of the plot.

• In many ways similar to Planning.

• Finding explanations for why extra players arrive.

‣ Alternate routes into a level.

• Explaining why players leave within the context of

the narrative.

‣ E.g. Banished, teleported etc.

51

Interactive Fiction

• IF allows us to generate worlds that feel more

engaging to players.

• The world can react to the player’s actions.

• The AI can direct the player’s attention to ensure

target “gates” are passed through in the narrative.

• Makes the world “come alive”

52

AI for Scenario Control

•We’ve looked at ways we can adapt the plot of the

world to match what the player is doing.

• Can we adapt the pace of the world too?

‣ Alter the flow of enemies.

53

Left 4 Dead

• Survival shooter game from Valve (2008)

• Everyone turned into zombies except four survivors

• Cooperative game for 1-4 players

‣ Take the role of the survivors

‣ Escape the city and find safety

• If playing short-handed, AI players will fill the team

54

Horror Movie Pacing

• The core thing Valve were aiming for was replicate

the dramatic pacing of a horror movie.

• This is a standard device for building suspense.

• “Emotional Intensity”

‣ Starts low, gradually builds, peaks at a high level.

‣ Tapers off to a relaxation level.

‣ Cycle repeats.

55

Measuring Intensity

• In order to model intensity need a range of metrics.

• Problem of feature selection we discussed earlier in

section on Dimensionality Reduction.

• Selected features for this include :

‣Damage being dealt

‣Damage being taken

‣Distance to nearest zombie

• Decays over time.56

Delta(Intensity)

• The desired emotional intensity matches the

current progression through the dramatic cycle.

• The current actual intensity will be different.

• This gives rise to a difference between them.

• The AI Director modulates the game world to

compensate for this difference.

57

The AI Director in Action

58

Altering the Game

• In Left 4 Dead, the principle method of altering the

game is changing the number of enemies.

‣ For peak intensity, hordes of enemies come from all sides

- Spawned by the Director

‣ At low intensity, during relaxation periods, enemies are

removed.

• Pretty basic approach - first pass at the technology

59

Enhanced AI Directors

• More subtle methods can be used to manage the

player’s experience.

‣Weather - varying intensity of storms and lightning to

build intensity.

‣ Items - during intense times, reduce the amount of ammo

available.

‣ Enemy types - keep the player on their toes

60

Hierarchical Concurrent State Machines

• The HCSM is the driving force behind the AI

Director.

• 1998 technique developed for scenario management

in a driving simulator.

61

Hierarchical Concurrent State Machines

• Building block of an HCSM is itself an HCSM

‣ Recursive by nature

• Analogue input wire

• Configuration parameters

•Output wire

62

Hierarchical Concurrent State Machines

63

Hierarchical Concurrent State Machines

• Input - Delta(Intensity)

• Configuration - current parameters of level

•Objective - Minimise Delta(Intensity)

• Child HCSMs - Spawn new enemies, alter level etc.

64

Further Reading

• HCSMs were first introduced in

‣ “HCSM: A Framework for Behaviour and Scenario

Control in Virtual Environments” Cremer, Kearney and

Papelis

• Discussed in some detail on AIGameDev.com

‣ http://aigamedev.com/open/reviews/hcsm-concurrent-

state-machine/ (me)

65

Summary

• Covered “Experience Management”

• Interaction Design for Immersive Worlds

• Interactive Fiction

• Scenario Control

‣ HCSMs

66

Next (Final) Lecture

•What is Game AI?

• Traditional AI vs Game AI

• Emergence

• Debugging

• Software Engineering in AI

‣ Scrum

• Your AI Toolbox

67