Frames of Reference for Perception and Action

in the Human Visual System

MELVYN A. GOODALE* AND ANGELA HAFFENDENDepartment of Psychology, University of Western

Ontario, London, Ontario N6A 5C2, Canada

Thomas E. Shepherdthomas.e.shepherd@boeing.com

3 December 2004

Neuroscience & Biobehavioral Reviews, Vol. 22, No. 2, pp. 161–172, 1998q 1998 by Elsevier Science Ltd

Thesis

• We do not actually experience the visual information that is used to directly control physical actions– reaching for a peripheral object– moving within an environment (?)

• The visuo-motor systems that mediate our physical actions are independent from those that allow us to perceive objects and events

A case of Neurological Dissociation

• A case study of selective visual system damage– DF is a young woman who experienced brain damage from CO

exposure• occipital lobe damaged but primary visual cortex spared

– DF is no longer able to visually ‘recognize’ perceptual information about objects in the world ( including people )

• size, shape, orientation, and identity• she does recognize them from sound or touch

– DF can still correctly articulate her hands to touch or grasp the objects she cannot visually perceive

• Reciprocal case studies exist – people who can still visually perceive the world – but cannot accurately articulate grasping motions

DF’s Failed Visual Perception

A : graph of DF’s hand articulation immediately after observing an objectB : graph of DF’s hand articulation after a 2 second dely

DF could not retain a visual “percept” of the observed object …

Two Separate Visual Systems?

• Observation : – “… the human visual system does not construct a

single representation of the world for both visual perception and the visual control of action.”

• Evidence :– neurological dissociation case studies– two prominent visual “pathways” have been identified

in the cerebral cortex of monkeys• Hypothesis :

– the two “visual streams” have different functions and perform different transformations upon a common visual input

The Hypothesis

• The perceptual representations constructed by the ventral stream are part of a high-level cognitive network that enables an organism to select a particular course of action with respect to objects in the world.

• The visuomotor networks in the dorsal stream are responsible for the programming and on-line control of the particular movements that the selected action entails.

To summarize so far

• similar but not identical (?) visual information about object size, shape, orientation, and location is available to both visual systems

• the transformational algorithms that are applied to these inputs are tailored to the function of each system

Unseen Visual Events

• If two visual stimuli of unequal intensity occur at the same time, the more intense stimulus can often ‘‘mask’’ the less intense stimulus.– ‘‘forward masking’’ is when the masking stimulus precedes the

less intense stimulus– “backward masking’’ is when the masking stimulus follows the

less intense stimulus– backward masking has been reported with intervals as long as

100 ms

• Even though the “perception” of the visual event might be masked, a variety of voluntary motor outputs can be driven by the un-detected event!

Different Spatial Coding for Perception and Action

White bars indicate “single step” targets.Gray bars indicate “two step” targets.

Subjects were asked to move their finger from a central target to a new target that appeared suddenly in their peripheral visual field. On half the trials, the peripheral target stayed in position until the subject had completed the aiming movement. On the remainder of the trials the target was displaced to a new position, 10% further out from where it had originally appeared.

Even though motor output was sensitive to changes in target position, the subjects remained unaware that the target had jumped to a new location.

The Ebbinghaus Illusion

Visual perception of object size is influence by its relationship to neighboring objects.

Size coding for Perception and Action

A physical implementation of the Ebbinghaus Illusion with different sized disks being grasped by test subjects.

The calibration of grip size was largely impervious to the perceptual effects of the size contrast illusion, again indicating separate paths for visual perception and visual motor control.

Perception and Visuomotor Control in the Periphery

Summary and Conclusion

• The paper claims strong evidence for the parallel operation of two types of visual processing systems.– Perception and Visuomotor Control

• Each system has evolved to transform visual inputs for different functional outputs and has very different characteristics.

• Perhaps this dichotomy of Visual Systems is a contributing source to “spatial contraction” in immersive Virtual Reality environments.