Learning, Recognizing, and Assisting with Activities

Tom Dietterich

Oregon State University

Assumptions

• Goal: Integrated, autonomous, and useful AI systems– Must collaborate well with people

• Must recognize and understand human goals, attentional state, costs of coordination, etc.

Use Case 1: Edit and return document

Document1

Person1

Attach

Person2

Save Attachment

Doc1.doc

Message1

Send

SaveAs

Doc2.doc

ReplyTo

Attach

Send

AGENT could:• automatically create TODO item when email arrives• remind user when deadline is near• detect when user has finished editing Doc2.doc and offer to

send it back to Person1• automatically remove TODO item when completed

Use Case 2: NSF Proposal Review

print

/www.fastlane.nsf.

gov/jsp/homepage/

prop_review.jsp

Request to review, includes proposal ID and password

web pagepaste ID, password

web page

navigate

download

service/nsf/proposal.pdf

/www.fastlane.nsf.g

ov/jsp/homepage/

prop_review.jsp

web page

web page

navigate

web page

navigate

paste ID, password

web page

navigate

web page

submit

fill out review form

fastlane.nsf.govfastlane.nsf.gov

web page

logout

open URL

AGENT could:• automatically create TODO item when email arrives• remind user when deadline is near• automatically login and download & print proposal• automatically login and navigate to “review form” page• automatically remove TODO item when completed

confirmation page

Use Case 3: Prepare Quarterly Report

Attach

From: beers@sri.com

g/darpa/calo/management/arpa/

Q3-report.doc

SaveAs

Send

Save Attachment: Q3-report-chin.doc

Save Attachment: Q3-report-williams.doc

Save Attachment: Q3-report-sanchez.doc

Paste

Chin

Williams

Sanchez

Save Attachment

Attach

ReplyTo

g/darpa/calo/management/arpa/

Q3-report-template.doc

Edit using WORD

Send

reminder

AGENT could offer to• automatically create TODO item when email arrives• automatically save attachment and open it in Word• automatically create outgoing email msg, address it to the correct

recipients, and attach the template• automatically track the email replies and save the attachments (with

the right names) in the right folder• automatically offer to send reminders to the missing subcontractors• automatically open up the template and all replies in Word• automatically attach the final file to a reply email to Melissa• automatically delete TODO item when complete

Research Challenges

• Representing Workflows

• Learning Workflows

• Recognizing Workflows

• Deciding (Learning) When and How to Help

Representing Workflows• For what purpose:

– execution: • sequence of actions (possibly with conditionals and iteration)

– recognition:• partially-ordered sequence of actions (with conditionals and iteration)• capture additional features to aid recognition (e.g., email speech acts)

– learning:• need action models to detect unobserved steps and understand goals

– assistance• need action models to understand goals

• Workflow steps:– informational inputs (file name, file, URL)– action (click Download)– action models (creates file on disk with file name; contents = contents of

URL file)

Representing WorkflowscommentOnDocument :- mailArrived(EmailRID, Requester, SpeechAct,

Deadline, [Attachment1]), outlookOpen(EmailRID), attachmentSave(EmailRID, Attachment1,

FileRID), wordEditDocument(FileRID, EditedFileRID), outlookOpen(EmailRID), outlookComposeReply(NewEmailRID,

EmailRID), outlookSendReply(NewEmailRID, Requester,

SpeechAct2, [Attachment2]), outlookAttachmentInfo(NewEmailRID,

EditedFileRID, Attachment2).

wordEditDocument(FileRID, FinalRID) :- wordOpen(FileRID), finishEdit(FileRID, FinalRID).

// simply close the file and return itfinishEdit(FileRID, FileRID) :- wordClose(FileRID).

// close the file, then later re-open it and continue

finishEdit(FileRID, FinalRID) :- wordClose(FileRID), wordOpen(FileRID), finishEdit(FileRID, FinalRID).

// perform a SaveAs and then continuefinishEdit(FileRID, FinalRID) :- wordSaveAs(FileRID, NewFileRID), finishEdit(NewFileRID, FinalRID).

Learning Workflows

• Learning by Demonstration– LAPDOG: PBD system at SRI– Lau, et al. at IBM and before that UW– PLOW: Allen et al. Rochester

• Learning by Observation (unsupervised)– Weld et al.

Recognizing Workflows

• Challenges on the desktop– Multiple workflows interleaved– Multiple instances of the same workflow

interleaved• reviewing multiple NSF proposals

– Sharing across workflows• log in and navigate only once, then download

multiple files

– Unmodeled background events• IM, nytimes.com, weather.com, etc.

Recognition Task

• Given:– a set of workflow schemas– an observation sequence

• Find:– all instances of those workflow schemas in the

observation sequence– detect each instance as early as possible– report the current state of all active workflow schemas

at each point in time

• Metrics:– false positives, false negatives, timeliness

Assistance

• What steps can the AGENT do?• What steps should the AGENT do?• How and when should the AGENT coordinate

with the user?

• Decision-theoretic collaboration– model the user’s intentions and attentional state– estimate the expected benefit of AGENT’s assistive

plan (including coordination cost)– choose action that maximizes expected benefit

Rich Intention Structures

• Goal stack– traditional programming languages– hierarchical reinforcement learning formalisms– cognitive architectures: SOAR, ACT-R

• Goal graph– ABL (Mateas)

• The user’s TODO list is an intention structure– so is the Inbox for many people

• Revised statement of our goal:– representation, learning, recognition, and assistance

with rich intention structures

Related Topics

• Argumentation and Persuasion– How do two agents exchange information in order to reach

agreement?

• Explanation-based Teaching and Learning– AGENT makes a mistake– user says “Why did you do that?”– AGENT explains– user corrects parts of the explanation– etc.

• Transfer Learning– How do I transfer to you something I’ve learned when

• you have a different ontology• I can’t give you all of my training data (privacy, bandwidth)?

Summary

• Goal: AI AGENT that can help humans

• Prerequisite: AGENT must understand what its user is doing