Information in “Associative” Learning
description
Transcript of Information in “Associative” Learning
Information in “Associative” Learning
C. R. GallistelRutgers Center for Cognitive Science
Sloan-Swartz 7/22/08 2
Temporal Pairing
• Thought to be essential for the formation of associations
• Assumed to be the critical variable in work on neurobiology of learning (LTP)
• Basis of unsupervised learning in neural net models
Sloan-Swartz 7/22/08 3
But
• It’s never been objectively defined for any paradigm: What is the critical interval?
• Neither necessary nor sufficient for development of a conditioned response to the CS (the warning signal)
Sloan-Swartz 7/22/08 4
Not Necessary
• Subjects develop a conditioned response to a CS that is never paired with the US (the predicted event)--conditioned inhibition
• Pavlov and Hull struggled with this problem
• It has not been solved
Sloan-Swartz 7/22/08 5
Not Sufficient
• The truly random control (Rescorla, 1968)– It is the mutual information between CS & US that is
critical
– Not their temporal pairing
Sloan-Swartz 7/22/08 6
It’s Information!
• People believe in “temporal pairing” because they are intuitively sensitive to the fact that a relatively more proximal warning gives more information
• It’s the information that matters, not the temporal pairing
Sloan-Swartz 7/22/08 7
Information Derives From Temporal Representation
• Information-theoretic analysis explains BOTH cue competition AND the data on the temporal pairing
• Founded on the assumption that animals learn the intervals
• AND, they represent the uncertainty with which they can remember them (about +/- 15%)
Sloan-Swartz 7/22/08 8
Principles I
• Subjects respond only to stimuli (CSs) that provide information about the timing of future events (USs)
• CSs inform to the extent they change the subject’s uncertainty about the time to the next US
Sloan-Swartz 7/22/08 9
Principles II
• Bandwidth maximization by minimizing number of information-carrying CSs attended to
• Information carried by intervals and numbers
• They are what is learned
• Weber’s law: uncertainty scales with delay: =wT
Sloan-Swartz 7/22/08 10
Rate-Change Protocols
€
˙ H = λ log2
e
λΔτ
⎛
⎝ ⎜
⎞
⎠ ⎟
€
H =1
λλ log 2
e
λΔτ
⎛
⎝ ⎜
⎞
⎠ ⎟= k − log 2 λ
Hb −Hcs = k−log2 λb( )− k−log2 λcs( ) =log2 λcs −log2 λb
Information communicated by CS log2
λcs
λb
⎛
⎝⎜⎞
⎠⎟=log2
Ius-usIus-us|cs
⎛
⎝⎜⎞
⎠⎟
Sloan-Swartz 7/22/08 11
Delay Protocols
• They are additive
• Only one depends on protocol parameters
H =log2
λcs
λb
⎛
⎝⎜⎞
⎠⎟+ k λcs =1 T
€
k =1
2log2
e
2π
⎛
⎝ ⎜
⎞
⎠ ⎟− log2 w
• Two sources of information:
1) The rate change 2) The fixed delay
Sloan-Swartz 7/22/08 12
Gibbon & Balsam
• Reinforcements to acquisition, as a function of the
Ius-us/Ics-us ratio
• Slope (log-log) ~ -1
Sloan-Swartz 7/22/08 13
Trials Don’t Matter
• These two protocols are equi-effective!• The number of trials is not in and of itself a
learning-relevant parameter of a training protocol• Gottlieb (2008)
Sloan-Swartz 7/22/08 14
Associability
• where Ncs-us = the number of CS reinforcements required to produce an anticipatory response.
(The onset of conditioned responding is abrupt)
• Definition parallels definition of sensitivity (1/Intensity) in sensory psychophysics
• Purely operational: no implication that associations exist
A =1 / Ncs-us
Sloan-Swartz 7/22/08 15
Informativeness
• We define the ratio of the background rate to the rate in presence of CS to be the informativeness of the CS-US relation in an associative learning protocol
• Thus, the information conveyed is the log of the informativeness
Sloan-Swartz 7/22/08 16
A Simple Quantitative Law
€
Associabilty ∝ Informativeness
A ∝λcs
λb=
IUS-USIUS-US|CS
Sloan-Swartz 7/22/08 17
Why trials don’t matter
• When there are 8 times fewer trials,• the trials are 8 times more informative• Provided one maintains total protocol duration• The only way to speed up learning is to increase
informativeness of the CS-US relation.• Adding trials won’t do it!
Sloan-Swartz 7/22/08 18
Conclusion 1
• Temporal pairing is–Undefinable
–Insufficient
–Unnecessary
• “Trials” are a pernicious fiction. Banish them from your models
Sloan-Swartz 7/22/08 19
Conclusion 2
• What matters is the mutual information (between CS and US), a component of which is the change in US rate when the CS comes on
• The informativeness of the CS-US relation is the factor by which CS onset changes the expected time to the next US
• Associability is proportional to informativeness
• That’s why people believe in in temporal pairing
Sloan-Swartz 7/22/08 20
Conclusions 3
• Focus on mutual information gives an empirically supported quantitative account of the notion of temporal pairing
• And an account of “cue competition:” how the system solves the multivariate prediction problem (aka the assignment-of-credit problem; what is predicting what), the other problem posed by Rescorla’s experiment
Sloan-Swartz 7/22/08 21
Thank You
• Collaborators– The late John Gibbon– Peter Balsam– Stephen Fairhurst– Daniel Gottlieb
• Support– RO1 MH68073 Time and Associative Learning