Christopher FiorilloBiS 527, Spring 2012

042 350 4326, fiorillo@kaist.ac.kr

Part 1: Inference in Perception, Cognition, and Motor Control

Reading: Students should visit some websites where they can experience and learn about illusions. Addresses are given in later slides.

Neurophysiology and Information:Theory of Brain Function

What is the function of the brain?• We would like to state this in terms

that are as general as possible, while also being quantitative

• If it is possible to develop a general theory of how the brain works, then we must first identify a general computational goal

The function of the brain is to produce motor outputs• Plants do not have a nervous system• Tunicates (sea squirts) have a nervous system early in development.

They swim until they find a good place to live. Then they eat (digest) all of their brain and never move again.‣ This general point has been described eloquently by Daniel Wolpert, a neuroscientist

who studies motor systems.• Animals must select some motor actions over others. This is “decision-

making”

Uncertainty is the Problem in Making Decisions• If I knew everything, then I would have no

uncertainty• I would always know exactly what I should do, and I

would make perfect decisions• The more information I have about the world, the

better my decisions, and the more successful I am (in terms of biology)

• Uncertainty is the only problem in making decisions• The function of the brain is to get as much

information as possible (about aspects of the world that are of biological importance)

• In other words, the function of the brain is to minimize its uncertainty about the world

Inference• Rene Descartes (1596-1650) famously wrote: “Cogito

ergo sum,” I think therefore I am.– He argued (very persuasively) that he could be certain of his

own existence, but that everything else that he knew was uncertain

– This was among the most important arguments ever made in epistemology, the study of knowledge

• “Inference” refers to knowledge of the world, which is necessarily characterized by uncertainty.

• I use “inference” as a synonym of “estimate” or “predict” or “hypothesis” or “guess.”

• I also use “knowledge” and “information” as synonyms• Inference is quantitatively described through

probabilities, which we will discuss in later lectures

Why is everything uncertain?• For any particular aspect of the world that one wants to

know about, there are other aspects that interfere.– The world consists of many “things,” and they interfere with one

another. Any event could have more than one possible cause.• Hermann von Helmholz gave the example of being

punched in the eye and seeing a flash of light– Obviously that does not happen often. But when you see light, you

cannot be certain whether it comes from photons, or from mechanical pressure on your eye.

• von Helmholz was also a founder of what is now known as “thermodynamics,” a branch of physics that deals especially with heat

• Thermodynamics and statistical mechanics gives a more general answer to the question of why everything is uncertain

Information Flows through A Series of Molecular

“Sensors”

Retinalthe light sensor in

rhodopsin

A

B

C

EnergyInformation Inference

Perception

Heat Makes Inference More Difficult

• Retinal is used by the eye because it is sensitive to light. But like all other sensors, it is also sensitive to heat.

• Heat is the movement of molecules. – The rhodopsin molecule, including retinal, has internal movement in the absence of

any external force.– Other molecules collide with rhodopsin– These microscopic mechanical forces can also change the conformation of retinal

• The conformation of a sensor is influenced by light, chemical concentration, etc. But it is also influenced by heat, and sometimes by other events, such as being punched in the eye

• All of the information of the brain (and the universe) relies upon molecular sensors such as this. There is nothing else in the brain. Thus there is always at least a small amount of uncertainty.

Everything the Brain Does is Inference• Every function of the brain can be understood

as inference (or prediction)– “Performing inference” simply means have the right

information at the right time• “Bayesian” methods are being used to try to

quantify inference using probabilities• However, there are many aspects of brain

function that can be understood as inference even without careful quantitative methods

• We will next discuss examples, starting with perception.

Perceptual Bistability• These images exhibit perceptual bistability

– There are two ways in which they are perceived, and they alternate

• There are numerous “objects” that are consistent with these images. The sensory evidence is ambiguous.

• The brain guesses that one of two objects is the most probable given knowledge of objects in the real world. Perception corresponds to the “best guess.”– For example, the lower image is seen as a 3-D cube, rather than

as an unusual combination of 1-D lines

• In the visual system, there are neurons for faces and vases and cubes of various orientations, but not for objects that have not been learned through experience.

• We expect to see faces and vases and cubes, but other possibilities are improbable

Binocular Rivalry• When different images are presented to the two eyes,

one image is seen (usually for a few seconds), and then the other– One does not see a superposition (mixture) of the two, or an

average

• To observe rivalry, view the images from a short distance, but allow your eyes to “relax” as though you are looking at something far away

• Rivalry demonstrates that some of the information that is present at low levels of the visual system is “suppressed” and does not advance to the high levels

• Since the two images are not compatible with one another, the high levels see only one or the other– This is a rational inference

– When the two images differ quantitatively rather than qualitatively, then one observes an average of the two

• The alternation could be explained by adaptation of neurons at higher levels of cortex

• Rivalry is not seen at all in LGN. It is weakly present in neurons of striate cortex, and becomes stronger at higher levels. It is essentially complete in IT cortex.

Illusions• Many illusions are the result of “rational inference”• The sensory evidence coming into the brain is ambiguous

– There are many possible interpretations of what is the true state of the world.

• To guess, the brain combines prior information with new sensory information• Prior information is already in the brain as the result of learning about

patterns in the sensory world• Some patterns are common, and these are the patterns that the brain expects

to see• Illusions occur when sensory patterns are unusual (in a statistical sense). The

rational, best guess is that the pattern is similar to the pattens that have been experienced in the past (based on the prior information). But in some rare circumstances, our best guess is wrong, and we experience an illusion.

• The critical point is that when we experience an illusion, we are doing the best we possibly can with our limited information. Our brains are working perfectly.

The Aperture Problem• The direction of motion of a large object is ambiguous

when viewed in a small region of space (an “aperture”)• http://www.psychologie.tu-dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/barber_pole.html

• Given the current sensory evidence, motion could be in any direction over a range of 180 degrees

• What is the relevant prior information (what are the most common patterns in the world)?–There is more likely to be one object moving with one

velocity rather than multiple objects moving with multiple velocities.

–Objects tend to move slower rather than faster.– A Detailed Study

• Weiss, Simoncelli, and Adelson. Motion illusions as optimal percepts. Nature Neuroscience 2002

An Audiovisual Illusion• The McGurk Effect

–http://www.youtube.com/watch?v=T4fUi0eG1X4&feature=related

• When we watch someone speak, our perception of their words depends on both auditory and visual information

• Conscious perception corresponds to high-level multisensory neurons, not low-level monosensory neurons

Inference in Language• Language comprehension relies on knowledge

of the statistical structure of language• Grammar specifies rules, and knowledge of

these rules is important for comprehension.• The probability of a particular word depends

strongly on the preceding words, and on the context (subject matter)

• Seidenberg, “Language acquisition and use: learning and applying probabilistic constraints” Science, 1997.

“Bayesian Integration in Sensorimotor Learning”• Kording and Wolpert, Nature 2004• As in studies of visual illusions, this study

provided evidence of Bayesian integration• In the case of visual illusions, the prior information

comes from a lifetime of experience with the visual world.– Prior information is not controlled by the

experimenters• In this experiment, they manipulated a frequency

distribution, and they showed that subjects were able to learn about that distribution and use that knowledge as their prior in Bayesian integration.

• 2 types of information were used: information about past target locations, and visual information about current finger (and cursor) location. Finger movement depended on both of these (in an “optimal” manner)

Prediction in Motor Control• Prediction of external forces is necessary to maintain

posture and for many aspects of motor control• Multiple types of information are important.

– Proprioceptive information about relative position of body parts– Visual information– Information of about self-generated motor outputs

• These can be illustrated by a simple experiment that we will attempt to perform in class– When I remove a weight from my own hand, I can predict the precise

time of the change in force and I can therefore adjust the contraction of my muscles (unconsciously) so that my hand does not move.

– When someone else removes a weight from my hand, I cannot predict the precise time of the change in force, and therefore I cannot keep my hand still, even if I try.

Bayes’s Theorem

• Posterior distribution = prior X likelihood• “Best guess” = combination of prior knowledge with

current sensory evidence– Information can be separated in various ways

• prior knowledge, current visual evidence, current auditory evidence, etc.

• Examples:–The aperture problem–McGurk Effect–Sensorimotor task

P(B|AC)P(A|BC) = P(A|C) xP(B|C)