Human Social Interactionperspectives from neuroscience

Dr. Roger NewportRoom B47

Student Drop-in: Tuesdays 12-2

www.psychology.nottingham.ac.uk/staff/rwn

roger.newport@nottingham.ac.uk

Understanding Action: social visual cues and biological motion

1

This lecture:

Your marks and stuff

Part 1 -

Social visual cues - the importance of eye gaze

Part 2 -

Social visual cues - detecting human motion

I Went to church last Sunday

She passed me on by

I knew her mind was changing

By the roving of her eye

The face provides information about:identityperceived personalityemotional state

The eyes provide information about:direction of attentionemotionintentionsocial statussocial rulesromantic attractioncorriedoo

Understanding actions: The head and eyes

The face is the most important object that we see every day.

Within the face the eyes are the most important feature.

2

Low-gazing females are rated as less-attractive by males

Objects that are gazed at by attractive people are rated as more desirable

IDIntentionality

Detector

TOMMTheory of Mind Mechanism

SAMShared Attention

Mechanism

EDDEye direction

Detector

5aThe eyes - models of eye gaze processing

Baron-Cohen

ID - primitive mechanism that interprets motion in terms of desires & goalsEDD- detects eye-like stimuli & works out direction of gaze. Attributes ‘seeing’ to the ownerSAM - links above 2 & identifies when you and other are looking at same thingTOMM - infers mental states

5bThe eyes - models of eye gaze processing

Emery

A: L and T looking at each other vs notB: L detects that T is not looking at them and follows B’s gazeC: L&T looking at same thingD: Combination of A and C - both know that they are both looking at the same thingE: Inferring intentions using a combination of D and higher order mechanisms

Farroni et al., 2006. Eye gaze processing: an innate skill?

Babies (1-5 days old) prefer to look at upright straight head direct gaze vs. others

4

A primitive configuration mechanism?

By 4 months infants a) Prefer direct to averted gaze

b) Can follow gaze

c) Will orient more rapidly to a location cued by an adult

The eye as a special stimulus. Response of cells in striate cortex.

An evolutionary advantage for efficient gaze processing 6

Various mammals have developed specialised systems for detecting eye-like stimuli

Rapid detection and reaction to predators

Psychophysical evidence for a specialised eye detection system

Fast

Slower

7

Automatic and obligatory processing of gaze

Human psychophysical evidence for a specialised eye detection system

8

+

+ *

+ @

+

-->

+ @

Covertpull

Overt

+

+ @

Covertpush

9Different types of attention shift

Ristic and Kingstone, 2004

10Gaze cuing - automatic processing of gaze shifts in others

Systems active for arrows different to those for gaze.

Eye gaze processing is special. Hietanen et al., 2006 1

Perrett, D.I. et al. (1985)

Monkey cell gaze preference

Where in the brain? Evidence for STS involvement: animal studies

Neurophysiological responses from cells in the superior temporal sulcus of the macaque to indirect and direct eye contact of conspecifics (adapted from Perrett & Mistlin, 1990).

10

Evidence for STS involvement: Monkey lesion studies 11

Eye gaze processing is impaired following STS lesions

Unable to make gaze-direction judgements, but unimpaired on other face processing tasks.

Various Perrett et al., studies

STS activation in humans12

From Allison, 2000

Wicker PET averted vs. mutual gaze videos

H&H fMRI static images

Puce fMRI alternating static gaze

STS, but not MT

Kingstone et al., 2004

Evidence for STS involvement: Human imaging studies 13

Subtracting neutral from cued removes STS activation

Hietanen et al., 2006 Why no STS activation? 14

Evidence from Autism

Autistic children have difficulty using gaze information to interpret the intentions of others. Any evidence related to STS activity?…

Which sweet does Charlie prefer?

Which one is looking at you?

15

Pelphrey et al., 2005

N A

Eye gaze related (A&B) and incongruence related (C&D) STS activity

16Evidence from Autism

Human neuropsychological evidence: Patient MJ - a rare STG lesion

Look it up yourself et al., 2006

17

18

Gaze is not the only cue to social attention 19

Psychophysical studies of other visual social cues

20

21Animal SCR studies

Rolls et al.

22

Perrett’s DAD

IDIntentionality

Detector

TOMMTheory of Mind Mechanism

SAMShared Attention

Mechanism

EDDEye direction

Detector

B-C’s EDD23

24

Does this fitPerrett’s DAD?

Human psycho-phyiscs

Congruent v incongruent head and eyes

Task, to indicate eye direction regardless of head and vice versa

C CI I

Eyes Summary

There are various facial visual cues that can inform us about the direction of attention of others

Of these they direction of gaze seems to be the most readily detectable

Although gaze may be the most important, gaze alone is not sufficient to tell us about another’s direction of attention

Cues from eye, body and head direction are possibly processed in parallel

All this seems to be processed in the STS

This information should be taken into account when designing models of social behaviour

25

Dr. Roger NewportRoom 331

Office Hours: Tuesdays 10-12

www.psychology.nottingham.ac.uk/staff/rwn

Human Social Interactionperspectives from neuroscience

Understanding Action: biological motion

1

biological and non-biological motion

11

What is special about Biological motion?

Actions often used for social communication

Actions involve movement

Detection of biological movement (i.e. of animals) important for survival

Evolutionary advantage often = specialised neural network

Innate skill? ability from 3 months.

Goes beyond facial expressions and social attention cues:

Biological motion

Non-biological motion

2

From bio motion we can perceive:

Identity (face/body movement)

Gender

Attractiveness

Affect

Intentions

What is special about Biological motion

Goes beyond facial expressions and social attention cues:

Social intentions

3

BMLwalker

BML Walker

Observing point-light displays

Observing real vs robot

http://www.biomotionlab.ca/Demos/BMLwalker.html

4Ways of studying biological motion

http://www.psy.vanderbilt.edu/faculty/blake/BM/BioMot.html

Types of point-light stimuli used in studies

5

Scrambled, inverted and normalMotion

Ways of studying biological motion

Perrett studied 50 ‘hand action’ STS cells as monkeys viewed various human hand movements(1) Most cells responded best to particular movement type(e.g. grasping vs. retrieving action.(2) Responsiveness for that action generalized across objects(3) Responsiveness generalized across various ways of makingthe same action (e.g. fast versus slow, near versus far (4) Responsiveness was greater when the action was goal-directed;

Cells therefore encode particular goal-directed hand actions.

Cells in superior temporal polysensory (STP) cortex in monkeys receive inputs from dorsal (action) and ventral (perception) streams (integrating form and motion).

Such inputs would allow the detection of form from motion. Some cells in STP respond selectively to biological motion.

Evidence from monkey studies (mainly Perrett et al.) 7

Human imaging studies 8

Many left STS/IPS but also right STS

Up to 2000 only

Random dots drifting left or rightRandom dot cube rotating left or rightRandom dot cube rotating left or right with masking elementsUpright point-light walker facing left or rightInverted point-light walker facing left or right Upright point-light walker facing left or right with masking elementsInverted point-light walker facing left or right with masking elements

Human imaging studiesGrezes et al. 2001STS activationRed = walkerGreen = cubeBoth = yellowNote bio always more anterior than nonbioLeft IPS non-rigid bio motion

9

NB masked inverted BM not detected in behavioural task

Activation in this subject is bilateral, but most subjects showed a right hemisphere bias

Grossman and Blake, 2001

Posterior STSBio>InvertedInverted>scrambledBio>imagined

Human imaging studies10

Human imaging studies - difference between motion sensitive and Bio motion sensitive areas

Beauchamp et al. 2002

11

12

(A) Grand averaged ERPs across 12 subjects elicited by biological (blue) and scrambled (red) motion.

(B) Scalp distribution of ERP difference amplitudes (ERP amplitude in bio motion condition minus scrambled motion).

Clear differences (increased activation) observed after 200 ms in right occipitotemporal region (STS probably) 200ms after stimulus onset

13

Grossman et al., 2005 10 min 1Hz rTMS

Pre-baseline

STSPost baseline

MT

Upright Inverted

Posterior STS rTMS disrupts bio motion

14

Patient ALDamage to right OT Junction (plus massive LH damage)Couldn’t see form from motioncouldn’t see Johannson figuresCouldn’t recognise familiar people if they moved

but could describe motion itself (direction of).

Dissociation with LM (Zihl’s famous akinetopic patient) who could not see first order motion

AL and LM’s lesions barely overlap. LM more posterior.

Evidence from lesion studies15

Evidence from Schizophrenia 16

Deficits in motion perception have been suggested as an important feature of schizophrenia (results based on e.g. velocity discrimination)

ST cortex volume is reduced in schizophrenia patients Do schizophrenics have deficits in biological motion perception?

Kim et al., 2003:task to judge whether point-light animations looked human or not

Half the stimuli were from a real person in motion (e.g. dancing, jumping, walking etc). Other half were scrambled versions

control task: perceptual grouping task,

Schizophrenia patients showed deficits in detecting biological motion but not in the control task

Recent studies

Detecting head and eye direction seem to interact

Timing of neural response to head and eye gaze inconclusive so far

Are the neural correlates of gaze processing affected by head orientation and task demands?

Explicit versus implicit gaze processing assessed by ERPs

Viewed images of direct or 3/4 turned head with averted or direct gaze. 1) made gaze judgments (explicit gaze task), 2) made head orientation judgments (implicit gaze task).Measured ERPs + behavioural RT task

Current Research October 2007

Rationale

Methods

Results

Behavioural task: incongruent head and eyes slowed RTs and lowered accuracy. Best performance in straight head AND eyes conditionSuggests gaze processed automatically regardless of task demands

ERP analysis:Greater response at 400-600ms for straight gaze regardless of task

The STS processes biological motion

Does it process body part-specific motion, or movement-specific motion?

If activations overlap then movement-specific, if not then body part-specific

Common and distinct brain activation to viewing dynamic sequences of face and hand movements

fMRI

Current Research September 2007

Rationale

Methods FaceHand andGratingAll radial motion (so same type of motion)

Green = all motion (all motion - fixation condition)Blue = hand vs radialRed = face vs radial (showing different area to hand)

Purple = hand v radial and face v radial overlap in pSTS (i.e. motion-specific, not body-specific area)

Results

Infants < 18 months do not have a pictorial representation of the human body

Infants ~ 6 months are compelled by biological motion

Do infants treat human bio motion as a solid human form?

Infants perceive human point-light displays as solid forms

Point light stimuli that pass through or behind a solid object

Current Research July 2007

Rationale

Methods Results

Infants look for longer (are more interested in) violations

Summary

Detecting eye, head and body movements are essential for normal everyday human social interaction

Eye gaze and biological motion are special

They have dedicated brain regions and may be innate

The STS region of the brain is crucial to these functions

Parts of the STS may be action-specific

Damage to this area may impair normal human social interaction

Next week,

Action understanding, shared representations and (possibly) imitation