Theory of Mindand an application to ECA interaction

Christopher PetersLINC

University of Paris 8

WP6 workshop, 10-11th March 2005, Paris

Overview• What is Theory of Mind?• Some approaches

– Cognitive– Philisophical– Evolutionary Psychological

• Adopted approach– Baron-Cohen (1994)

• Application to interaction and agents– Interaction initiation

Theory of Mind

• Specific cognitive ability to understand others as intentional agents– Interpret their behaviour– Attribute mental states– Form theories of their intentions,

desires and beliefs– Allows prediction

Reproduced from REF. 72 © 2000 Elsevier Science

Some Approaches• Cognitive approach (Leslie 1987)

– Pretend play in 2 year olds– Meta-representations

• Philisophical approach (Dennett 1987)– Design stance and physical stance– Intentional stance

• Evolutionary psychological approach (Baron-Cohen 1994)

One approach• Evolutionary approach to theory of mind

– Selective advantages for organisms with an ability to interpret and predict behaviour on the basis of mental states

– Provides the ‘why?’ and facilitates prediction

Perrett and Emery (1994)• Direction of Attention

Detector• Mutual Attention

Mechanism

Baron-Cohen (1994)• Eye Direction and

Intentionality Detectors• Theory of Mind Module

Intentionality Detector(ID)

• Represents behaviour in terms of volitional states (goal and desire)

• Based on visual, auditory and tactile cues• Attributes intentionality characteristic to objects

based on the presence of certain cues• Process dyadic representations • Goal-desire relationships with objects

– A bee wants to go to a colourful flower– A brick wants to fall onto the ground!?

• Detect the presence of eye-like stimuli– Present in a large number of species

• Visual modality• Detect and represent eye direction as an “Agent”

seeing the Self or something else

• Process dyadic representations– Agent-relation-Self, Agent-relation-Object– Agent1-relation-Agent2, Self-relation-Object

Eye Direction Detector(EDD)

Shared Attention Mechanism(SAM)

• Represent if the Self and another agent are attending to the same object or event

• Triadic relationships – Agent and self are both attending to same object– Self-relation-(Agent-relation-Object)– Self-relation-(Agent1-relation-Agent2)

• Links ID to EDD– Allows eye direction to be

read in terms of volitional states

Theory of Mind Mechanism(ToMM)

• Represent the full range of mental states including the episdemic ones

• Integrate mental state knowledge into a coherent theory of interpreting action

• M-Representations– Agent-Attitude-“Proposition”

• Allow one to:– Make sense of an agents current behaviour– Predict an agents future action

• Interpretation of behaviour in mentalistic ways

Details

• Modules are special purpose computational systems in the social brain

• All four modules (ID, EDD, SAM, ToMM) are functioning by 4 years of age in humans– EDD most basic and found in many vertebrates as a

sensitivity to predators looking at the animal– ID appears to be present in many primates– SAM only partially in great apes– Much less evidence for ToMM outside of humans

Useful• Information processing approach• Modular• Emphasis on the eyes and direction of attention

of the agent– Special relationship between EDD and SAM– Triadic representations built more easily in the visual

modality– When the goal of an action is uncertain, first place

young children and adults look for information to disambiguate the goal is the eyes

Importance of the Eyes• Eyes play an important role in

social interaction• Looking is noticeable

– Social hierarchies and attention structure for cohesion

Copyright © Steve McCurry

http://www.unep-wcmc.org Copyright © 2003 Wild Horizons Publishing

Eye Contact• Psychological Arousal

– Aggressive cue• Threatening displays in

many non-human primates• Dominance

– Affiliative cue• Youngsters who can

secure eye contact …• Gain attention of their

parent • Being fed or having other

needs satisfied

www.adventures.comwww.adventures.com

Conversation initiation • Problems with interaction initiation:

– Constrained by distance• Close proximity required

– E.g. Touch screen, enter a certain proximity, etc

– Explicit cues must be exchanged• User speaks to ECA, or vice versa to initialise• ‘All-or-nothing’ approach

• ECA interactions end up with interactants in close proximity… – …but should not have to begin that way– More subtle and flexible

Application• Conversation initiation for agents

– How do humans start interactions?– How can we let somebody know that we:

• Want to interact with them• Do not want to interact with them

– Speech• Explicit statement of ones intention, but risky• Avoid social embarrassment of engaging in

discourse with an unwilling participant (Goffman)• Nice to predict ones intention to interact before

committing

Application– Gesture and facial expression

• Less explicit/risky• But how to know who the expression is directed

at?

– Direction of attention• Eye behaviour, gaze, body orientation,

locomotion• Allows earlier subtle ‘negotiation’• Probe for intention to interact before committing

Application• Idea

– Use ToM to formulate theory on the intention of the other to interact

– Base theory on direction of attention behaviours

– Inherently uncertain process• Perception and Interpretation

– Full model should also account for utterance, gesture and facial expression

• Baron-Cohen doesn’t include these

Direction of Attention• Extension by Perrett and Emery (1994)

• More general Direction of Attention Detector (DAD)– Eye, head, body and locomotion direction

Mutual Attention

• Extension by Perrett and Emery (1994)• SAM relationships triadic

– But attention can also be shared between two agents

• Mutual Attention Mechanism (MAM)– Activates only with mutual gaze and eye

contact– Dyadic relationship

• X observes Y and Y observes X simultaneously

Integrated Model• Schematic of final model including

extensions

• Modules are building blocks– Foundation of an agent perception and

interpretation system

Possible Uses• Dual role

– Allow ECA’s to detect humans who want to interact• Do so in a more sociable manner• Subtle interaction initialisation rather than ‘all-or-nothing’

approach• Interaction starts at a distance• Allows a more human-like interface with the ECA

– Allow ECA’s to generate behaviours that show humans their intention to interact

• Do so in a more human-like manner i.e. not always sudden and explicit

• Orienting of attention direction to match attentive behaviours of users or to elicit attention from viewers

Our Model• Base on simple, high-level theories:

– Have they seen me?• ID, DAD and MAM

modules

– Have they seen me looking?

• ID, DAD and MAM modules

– Are they interested in interacting?

• Level of Interest metric

• Theories are uncertain and represent best guesses based on perception

Our Model• Model involves other important mechanisms

– Visual Attention– Memory and integration of percepts

• Implementation in development – Virtual Environment– Interaction between agents– Covered as part of tomorrows talk on Perception of

Attention

Conclusion• Presented a model of theory of mind based on

perception of direction of attention– Potential application to improving interface with ECAs– Perhaps provide some insight into how users may

change their perception towards ECAs (ID)

• Baron-Cohens model useful for computational modelling

THANK YOU!