Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti:...
-
Upload
melvin-moody -
Category
Documents
-
view
215 -
download
0
Transcript of Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector 1 Laurent Itti:...
1Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence
Lecture 7. Didday Model of Winner-Take-All
Reading Assignments:
TMB2:4.3, pp. 194-197. Prey Selection - or Winner Takes All4.4. A Mathematical Analysis of Neural Competition
2Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
The Prey-Selector Model of Didday
Consider how the frog's brain might select one of several visually presented prey objects.
The task: to design a distributed network (not a serial scan strategy) that could take a position-tagged "foodness array" and ensure that usually the strongest region of activity would influence the motor control system.
The prey-selector: an early, biological example of what connectionists call a winner-take-all network.
Didday’s model adds a relative foodness layer of cells in topographic correspondence to the retinotopic foodness layer.
The new layer yields the input to the motor circuitry.
3Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Winner-take-all Networks
Goal: given an array of inputs, enhance the strongest (or strongest few) and suppress the others
No clear strong input yieldsglobal suppression
Strongest input is enhancedand suppresses other inputs
4Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Winner-Take-All Networks
Basic architecture:- topographically organized
layer of neurons- each neuron receives
excitatoryinput (e.g., sensory)
- each neuron also sends inhibition
to neighboring neurons,
possibly via an inhibitory
inter-neuron
a
b
Example of 2-neuron WTAwith spiking neurons
strong stimulus
weak stimulus
5Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Didday’s Model
= copy of input
= inhibitory inter-neurons
= receives excitationfrom foodness layerand inhibition fromS-cells
retinotopicinput
6Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Didday's transformation scheme from foodness to relative-foodness
It uses a population of S “sameness“ cells in topographic correspondence with the other layers.
Each S-cell inhibits the activity that cells in its region of the relative-foodness layer receive from the corresponding cells in the foodness layer by an amount that augments with increasing activity outside its particular region.
This ensures that high activity in a region of the foodness layer penetrates only if the surrounding areas do not contain sufficiently high activity to block it.
7Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Hysteresis
How does the network respond to a change in the maintained stimulus pattern?
Once in equilibrium, one may increase a non-maximal stimulus s2 so that it becomes larger than the previously largest stimulus s1, yet not switch activity to the corresponding element.
In neural networks with loops - an internal state resists dependence on input: buildup of excitation and inhibition precludes the system's quick response to new stimuli.
For example, if one of two very active regions were to suddenly become more active, then the deadlock should be broken quickly.
In the network so far described, however, the new activity cannot easily break through the excitation of its competitor and the global inhibition: there is hysteresis.
8Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Hysteresis in a Ferromagnet
Not a single functionmagnetic field directionbut a history-dependent relation.
Direction of External Magnetic
Field
In a ferromagnetic material, the atoms act like magnets which tend to align themselves with their neighbors -- this cooperation is opposed by thermal noise.
A ferromagnet becomes magnetized when the atomic magnets have high probability of pointing in the same direction.
9Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Didday's Transformation Scheme From Foodness to Relative-Foodness
Didday uses a local mechanism, to combat hysteresis, introducing a Newness cell - "N-cell" for each S-cell to monitor temporal changes in the activity of its region.
For a dramatic increase in the region's activity, it overrides the inhibition on the S-cell and permits this new level of activity to enter the relative foodness layer.
10Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Formalizing: An "N-cell" for each S-cell
Introducing an "N-cell" for each S-cell
Formally: neurons Ni have activity ki proportional to dSi/dt.
We then postulate that theinput to the ith element toour array is no longer si
but si + ki.
If ki is large enough for
long enough, the relativefoodness Ei can gain
ascendancy.
Si
dSi/dt
11Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Didday Prey-Selector
Formalizing Didday's Model
The rest of this lecture provides a mathematical analysisfrom TMB2, Section 4.4, and is based on the paper:
Arbib, M.A., and Amari, S.I., (1977) Competition and Cooperation in Neural Nets, in Systems Neuroscience (J. Metzler, Ed.), New York: Academic Press, pp. 119-165.
To see the mathematics, switch to the Word file:
7W. Amari-Arbib Mathematics