Cristina ConatiDepartment of Computer ScienceUniversity of British Columbia

Plan Recognition for User-Adaptive Interaction

Research Context

User-Adaptive Interaction (UAI): interaction that can better support individual users by adapting to their specific needs

User Modeling: how to infer, represent and reason about user features relevant for adaptivity.

User Model

AdaptationKnowledge/Skills

Beliefs/Preferences

Goals/PlansActivities

Emotions

Meta-cognitive skills………

Overview

Brief examples of our plan/goal/activity recognition work in the context of UAI

Two research directions– Using eye-tracking information to facilitate plan

recognition– Explaining to the user the reasoning underlying the

adaptive intervensions

Adaptive Support To Problem Solving

• A tutoring agent monitors the student’s solution and intervenes when the student needs help.

• Example: Andes, tutoring system for Newtonian physics (Conati et al UMUAI 2002)

Fw = mc*g

Think about the direction of N…

N

Several sources of uncertainty Same action can belong to different solutions, or

different parts of the same solution Solutions steps can be skipped - reasoning behind the

student’s actions can be hidden hidden from the tutor Correct answers can be achieved through guessing.

Errors can be due to slips There can be flexible solution step order

Probabilistic Student Model

Bayesian network (automatically generated)– represents how solution steps derive from physic rules and

previous steps

Captures student interface actions to perform– on-line knowledge assessment, – plan recognition – prediction of students’ actions

Performs plan recognition by integrating information about the available solutions and the student’s knowledge

Example 2

Solution• Find the velocity by applying the

kinematics equation Vtx

2 = V0x2 + 2dx*ax

• Find the acceleration of the car by applying

Newton's 2nd law Fx = Wx + Nx = m*ax

If the student draws the axes and then gets stuck, is she trying to write the kinematics equations to find V? trying to find the car acceleration by applying Newton’s laws

RuleR

Fact/GoalF/G

RA Rule Application

R -try-Newton-2law

R-find-all-forces-on-body

R- choose-axis-for-Newton

F-N-is-normal-force-on-carG-find-axis-for-kinematics

G-try-Newton-2law

G_goal_car-acceleration?R -try-kinematics-for-velocity

R-find-kinematics quantities

R- choose-axis-for-kinematics

F-D-is-car-displacement

G-try-kinematics

G_goal_car-velocity?

F-A-is-car-acceleration

G-find-axis-for-newton

0.5 0.9

0.5

0.8

0.7

0.5

1.0

F-axis-is 20

0.95

0.83

0.9

0.5

0.68

0.4

0.2 0.9 0.72

/ 0.9

/ 0.6

CPTs

Evaluation

Several studies showed effectiveness of Andes tutoring

Could not evaluate the plan recognition component directly, because of lack of ground truth values

(Conati et al UMUAI 2002)

Adaptive Support To Learning From Educational Games

Tricky problem- Help students learn- While maintaining fun

And engagement

Model of UserKnowledge

Model of UserAffect

Goal recognitoon for Modeling User Affect (via Cognitive Appraisal) (Conati Maclaren 2009)

GoalsSatisfied

Goals

Personality

InteractionPatterns

Emotion towardGame state

ti

UserAction

Outcome

EmotionsTowards Self

Agent ActionOutcome

GoalsSatisfied

Goals

ti+1

Emotion towardGame state

EmotionsTowards Agent

Personality

InteractionPatterns

Subnetwork for Goal Assessment

Goals

Extraversion

NeuroticismAgreeableness

Conscientiousness

HaveFun

AvoidFalling

BeatPartner

LearnMath

Succeed byMyself

Follow Advice FallOften

Ask AdviceOften

Move Quickly

Use Mag. GlassOften

Personality [Costa and McCrae, 1991]

Interaction Patterns

Links and probabilities derived from data (Zhou and Conati 2003)

Evaluation

DDN with goal assessment performs better than variation with goals initialized with population priors

Pretty good results on emotions recognition (~70%), but could be improved if we modeled goals as dynamic (changing priorities)

(Conati et al UMUAI 2002, Conati 2010)

Current work

See if we can improve goal recognition by including information on user attention

In previous research, we showed that including gaze information can improve a system’s prediction of user reflection/learning (Conati and Merten, Intelligent User Interfaces 2007)

We are now looking at whether eye-tracking can help recognize user goals and intentions (hints, no hints)

Adaptive Support To Interface Customization

MICA: Mixed-initiative support in creating a “personal interface”with tailored toolbar and menu entries (Bunt Conati Macgrenere IUI 2007)

Example: Adding Features

Example: Adding Features

Suggestions Generation

User Performancewith a given

Personal interface

Expertise

Expected Usages

Interface Layout

Personal interface with best expected performance

Overview

Brief examples of our plan/goal/activity recognition work in the context of UAI

Two research directions– Using eye-tracking information to facilitate plan

recognition– Explaining to the user the reasoning underlying the

adaptive interventions

How to provide effective adaptivity without violating the basic principles of HCI– Predictability, Controllability, Unobtrusiveness,

Transparency

One possible approach: – Enable the system to explain to the user the rationale

underlying its suggested adaptive interventions

One Challenge of UAI

Example: Adding Features

Rationale: How

Rationale: How

Formal Evaluation of Mica’s rationale

Compared versions of MICA with and without rationale [Bunt , Mcgrenere and Conati UM 2007]

Within subject laboratory study. – Participants performed guided tasks with MSWord, designed

to motivate customization User Model initialized with accurate information

– Expected usages frequencies obtained from guided tasks– Expertise obtained via detailed questionnaire

Study 2 (Rationale vs. No Rationale): Main Findings

No performance differences

94.2% (Rationale) vs 93.3% (No Rationale) recommendations followed

Preference Results

0

1

2

3

4

5

6

7

8

9

10

Overall Agreement Trust SpecificUnderstanding

GeneralUnderstanding

Predictability

# of

par

ticip

ants

Rationale

No-Rationale

Equal

Majority of users prefer to have the rational present, but non-significant number don’t need or want it.

Identified aspects of this context that may make rationale unnecessary for some– Found the rational to be common sense– Unnecessary in a mixed-initiative interaction or

productivity application– Inherent trust

Design implications: rationale should be available but not intrusive

rationale

no rationale

Open Questions

When is it important to provide the rationale? How much information should be given? How to handle user feedback?

Conclusions

Plan/Goal/Activity recognition crucial in user-adaptive interaction

Important to explore new sources of information for accurate user modeling– E.g. eye tracking

Important to increase UAI acceptance via mixed-initiative approaches, that possibly include explanations of system behavior

Understanding user goals and limitations in interactive with information visualizations

How can gaze information help?

video