1

Generalizability of Goal Recognition Models in

Narrative-Centered Learning Environments

Alok Baikadi Jonathan Rowe, Bradford Mott James Lester

North Carolina State University

2

Goal Recognition in Narrative-Centered Learning Environments

Task: Identify the specific objective that the player is attempting to achieve

Goal recognition models enable the following:• Preemptively augmenting

narrative experiences• Assessing problem solving in

narrative-centered learning environments

• Iteratively refining learning environment

3

Generalization of Goal Recognition

Goal Recognition is typically very domain dependent• Plan libraries• Many domain-independent techniques are only

evaluated on one domain Research Question: Can a domain-specific goal

recognition model be applied in a principled way to a new domain and achieve similar results?

4

Related Work

Goal recognition is a restricted form of plan recognition (Carberry 2001; Kautz & Allen, 1986; Singla & Mooney, 2011)

Investigated widely in numerous domains (Blaylock & Allen, 2003; Charniak & Goldman, 1993; Lesh, Rich & Sidner, 1999)

IO-HMM approach for recognizing high-level goals in simple action-adventure game (Gold, 2010)

PHATT-based approach for behavior recognition in real-time strategy game (Kabanza, Bellefeuille & Bisson, 2010)

N-gram and Bayesian network approaches for goal recognition to support dynamic narrative planning (Mott, Lee & Lester, 2006)

MLN-based approaches (Singla and Mooney, 2011 ; Ha et al., 2011 ; Sadilek and Kautz, 2012)

5

Outline

Goal Recognition Approach Goal Recognition Corpora Evaluation & Discussion Conclusions and Future Work

6

Markov Logic Networks (MLNs)

Statistical relational learning• Combines first-order relational reasoning with statistical learning• Input: A set of first-order predicate calculus formulae, along with

weights• Formulae can be expanded into a Markov Random Field for learning and

inference The joint probability distribution is defined as:

Toolkit: Markov TheBeast (Riedel, 2008)

7

Representation

Predicate Interpretationaction(t, a) Action a happens at time targument(t, a) Argument a observed at time tlocation(t, l) Player is at location l at time tstate(t, s) The narrative is in state s at time tgoal(t, g) Player is pursuing goal g at time t

8

Context in Goal Recognition

Actions are not always independent Traditional goal recognition formulation allows

for all previous observations• Can lead to sparsity issues

Solution: Look for key events in the history that provide insight to the player’s context

Use the structure of the narrative to provide the context

9

Discovery Events

Task progress is represented by a sequence of discovery events

Partial Answers to Central Questions are clues towards the solution

Provides a context for goal recognition: What has the user discovered?

10

Discovery Event Formulae

Milestone formulae recognize which discovery events have already occurred

Uses a cardinality constraint to capture existence

11

Outline

Goal Recognition Approach Goal Recognition Corpora Evaluation & Discussion Conclusions and Future Work

12

CRYSTAL ISLAND: OUTBREAK

8th grade microbiology

Valve Software’s Source engine

Science mystery Goal: Identify

source and treatment of outbreak

13

CRYSTAL ISLAND: Introduction

1. Student plays the role of a new visitor to the island.

2. Student discovers that several team members have fallen sick.

14

CRYSTAL ISLAND: Gathering Information

3. Student gathers clues from sick team members.

4. Student asks the camp’s pathogen experts about microbiology concepts.

15

CRYSTAL ISLAND: Gathering Information

5. Student views microbiology-themed posters.

6. Student reads books about different types of pathogens.

16

CRYSTAL ISLAND: Hypothesis Testing

7. Student conducts tests using laboratory equipment.

8. Student interacts with the lab technician to view microscopic images of pathogens.

17

CRYSTAL ISLAND: Reporting Findings

9. Student presents findings and recommended treatment to camp nurse.

18

Corpus Collection

Eighth grade class from public middle school

153 participants

No prior experience with CRYSTAL ISLAND

Played game for 1 hour, or until they were finished

7 goals available to students (Ha et al., 2011)

19

CRYSTAL ISLAND: UNCHARTED DISCOVERY Upper Elementary Science

Subject 5th grade science Standards aligned

Content Landforms Maps, models &

navigation

Story Adventurous adolescent Shipwrecked crew Complete quests to

explore island

20

CRYSTAL ISLAND: UNCHARTED DISCOVERY Video

Video

21

Corpus Collection

Onsite at 8 schools 831 fifth grade students 62% Caucasian, 14% African

American, 8% Asian, 16% Other

Teacher-driven implementation in classrooms

6 one hour sessions over 4 weeks

12 goals available during the first 2 weeks

22

Goal Extraction Procedure

Goal-achieving actions were identified Actions between previous goal and

current goal were labeled with current goal

Goal-achieving actions were removed

23

Outline

Goal Recognition Approach Goal Recognition Corpora Evaluation & Discussion Conclusions and Future Work

24

Empirical Evaluation

State of the Art Baseline:• Factored model (Ha et al., 2011)• Uses MLNs to relate the current time step to the previous time step

Each model was evaluated using 10-fold student-level cross-validation

Each model was evaluated according to three metrics:• Accuracy: Measured as F1 score• Convergence rate: Percent of sequences which eventually predicted the

correct goal• Convergence point: In sequences that converged, the percent of actions

that had to be observed before a consistent prediction was made

25

Baseline Model

26

CRYSTAL ISLAND: OUTBREAKDiscovery Events Model

27

CRYSTAL ISLAND: UNCHARTED DISCOVERYDiscovery Events

28

EXPERIMENTAL RESULTS

Model F1 Convergence Rate

Convergence Point

Baseline 0.488 30.906 50.865Discovery Events 0.546 50.056 35.862

Model F1 Convergence Rate

Convergence Point

Baseline 0.226 11.915 87.786

Discovery Events 0.244 29.973 79.350

Crystal Island: Outbreak

Crystal Island: Uncharted Discovery

29

Outline

Goal Recognition Approach Goal Recognition Corpora Evaluation & Discussion Conclusions and Future Work

30

Conclusions

Goal recognition models show considerable promise for enhancing the effectiveness of narrative-centered learning environments

Encoding narrative discovery events in Markov Logic is a natural approach for representing context for student actions in goal recognition

Experimental findings from two narrative-centered learning environments suggest that narrative discovery events enhance the accuracy and convergence of state-of-the-art MLN-based goal recognition models.

31

Future Work

Investigate combinations of discovery events• Some of the milestones may have provided more information

than others• Use automated feature selection

Integrate goal recognition into a runtime environment• Can establish intuition for how accurate a model is necessary

Elicit feedback from player• Assumes goals achieved are intended• May cause some bias

32

Collaborators

Research StaffEleni LobeneRob Taylor

PostdocEunyoung Ha

Digital Art StaffKirby CulbertsonSarah HeglerKaroon McDowell

Graduate StudentsJulius Goth Wookhee Min

Joe Grafsgaard Chris Mitchell

Eunyoung Ha Jennifer SabourinSeung Lee Andy Smith

Sam Leeman-Munk

Undergraduate StudentStephen Cossa

Affiliated FacultyCarol Brown (East Carolina University)Roger Conner (East Carolina University)Patrick FitzGerald (Art + Design)Elizabeth Hodge (East Carolina University)James Minogue (Elementary Education)John Nietfeld (Educational Psychology)Marc Russo (Art + Design)Hiller Spires (Curriculum & Instruction)Eric Wiebe (STEM Education)

Affiliated Post-Docs and Graduate Students (Art, Education, Psychology)Megan Hardy (Human Factors)Kristin Hoffman (Educational Psychology)Angela Meluso (Curriculum & Instruction)Lucy Shores (Educational Psychology)Sinky Zheng (Curriculum & Instruction)

33

Acknowledgments

Support provided by the National Science Foundation under grant DRL-0822200. Additional support was provided by the Bill and Melinda Gates Foundation, the William and Flora Hewlett Foundation, EDUCAUSE, and the Social Sciences and Humanities Research Council of Canada.

34

35

Goal RecognitionExample

36

Goal Recognition Example

37

Goal Recognition Example

38

Goal Recognition Example

39

Goal Recognition Example

40

Goal Recognition Example

41

Goal Recognition Example

?

42

Goal Recognition Example

43

Goal Recognition Example

44

Goal Recognition Example

45

Goal Recognition Example

?

46

Goal Recognition Example

47

Goal Recognition Example

48

Goal Recognition Example

49

Goal Recognition Example

What is the player’s current goal?

50

Goal Recognition Example