Agents that reduce work and information overload

Pattie Maes, 1994

Presented by Huy Nguyen

Pattie MaesProfessor at MIT Media Lab

Fluid Interfaces research group, “radically reinvent the human-machine experience”

Interest: cognitive enhancement - how immersive and wearable systems can actively assist people

TED Talk - Human Machine Symbiosis

Motivation

Current dominant interaction metaphor is direct manipulation.

In AI, Autonomous agent can implement indirect management.

Two major problemsCompetence - acquire the knowledge to decide when to help the user?

Trust - user is comfortable delegating tasks to an agent

Previous approaches[1] Making the end-user program the interface agent (Lai, Malone & Yu, 1988)

● Semi-autonomous agent: collection of user-programmed rules

● Can buy “agents” programmed by other users

Previous approaches● Competence - requires too much

insight, understanding and effort form the end user

● Trust - “the program we write typically behave differently than expected”

Previous approaches[2] Knowledge-based approach: providing agent with extensive domain model and user model

● Competence - huge amount of work for knowledge engineer, also knowledge is fixed once and for all

● Trust - programmed by somebody else

An alternative approachHypothesis: under certain conditions, an interface agent can “program itself” (i.e., it can acquire the knowledge it needs to assist its user)

Given minimum background knowledge, learn from users and other agents

Conditions:

1. the use of application has to involve a substantial account of repetitive behavior.

2. this repetitive behavior is potentially different for different users

Four ways to acquire knowledge

Four ways to acquire knowledge

Example: electronic email agentMaxims: prioritize, delete, forward, sort and archive mail messages on behalf of users (Lashkari, Metral & Maes, 1994).

Memorizes all action situation-pairs generated, predict new action based on example stored in memory.

Example: electronic email agentCompare new situation with memorized situations, try to find nearest neighbor (distance = weighted sum of features).

Occasionally, analyzes correlations between feature and action (reflection)

Example: electronic email agentIn addition to predicting, also measure confidence

2 thresholds set by the user: “tell me” and “do it”

Communicate internal states to user via facial expression (cartoon faces)

Example: electronic email agentMaxims gradually gains competence by observing the user and acquiring more examples.

Solution to cold start problem:

● User can instruct agent explicitly

● Multiagent collaboration

Other examplesMeeting scheduling agent

News filtering agent

Entertainment selection agent

Instructable Intelligent Personal Agent

Azaria, A., Krishnamurthy, J., & Mitchell, T. M. (2016, March). Instructable intelligent personal agent. In Thirtieth AAAI Conference on Artificial Intelligence.

ConclusionShow how machine learning-powered interface agents can solve the Competence and Trust problem.

Many open questions:

● How can we guarantee the user’s privacy● How can different agents collaborate?● How to motivate users to share their experienced agent’s knowledge?● Should a user be held responsible for his or her agent’s actions?

Discussion #1 - The concept of intelligent agent“Siri and Alexa, on the other hand, are like interfaces into a wide variety of functionality and have a sense of being their own entity with a personality. Their conversational nature fulfills their purpose as a voice interface. I agree that these things are not really agents in the sense of Maes because they don't take action on your behalf.” - Sujeath

“There’s a fundamental difference between actually doing something on someone else’s behalf and merely providing support. And even though chatbots, voice-based apps, and other similarly popular systems today are often referred to as “conversational agents,” they often aren’t worthy of that title, or at least not in the same form that the true agents in Maes’s paper are” - Julia

“Remembrance Agent” - Jason

Discussion #2 - Information management“I'm not against them adding features to Alexa because they keep hearing some non-registered request, but of course if they're using my data for other non-improvement stuff and not letting me opt in/out, it feels kind of dirty. Then again, I figure all the companies/government/etc. already knows everything about me from ole' Google, so I'm kind of meh toward most privacy stuff like that” - Steven

Discussion #2 - Information management“If I have an agent that manages my inbox or scheduling, to what extent do my human peers / colleagues / friends / family know that they’re speaking with a virtual agent rather than with me, the human Julia? [...] In short, how / who / when should we disclose our digital agent helpers as they become more convincingly human?” - Julia

Google Duplex - “Google Assistant making the call seamlessly in the background for you”

Discussion #3 - Open questions“I liked the question of whether humans should be held responsible for the actions of their agents. I think when something bad happens, someone has to be responsible. I think people are smart enough not to blindly trust a computer's diagnosis of a patient and we are pretty good at being conservative about this. When the stakes are high, programming errors and the randomness of machine learning cannot play a part. A human must ultimately be responsible in the chain of reasoning.” - Sujeath

“I am working on a system which is similar to this in terms of user interaction. The idea is that it is an authoring tool for intelligent tutoring systems via model tracing by teaching an artificially intelligent agent on how to correctly solve problems in the interface. The agents I am using are a bit different from this because they create generalizable rules instead of using nearest neighbor examples.” - Danny

Thanks for listening!