Three Kinds of Learning Chrisantha Fernando Marie Curie Fellow, Collegium Budapest Chrisantha...
-
Upload
anna-greer -
Category
Documents
-
view
214 -
download
1
Transcript of Three Kinds of Learning Chrisantha Fernando Marie Curie Fellow, Collegium Budapest Chrisantha...
Three Kinds of LearningThree Kinds of Learning
Chrisantha Fernando
Marie Curie Fellow, Collegium Budapest
Chrisantha Fernando
Marie Curie Fellow, Collegium Budapest
AimAim
I present some examples of learning Associative learning Causal inference Insight based problem solving
My aim is to understand the mechanisms underlying these learning behaviours.
I present some examples of learning Associative learning Causal inference Insight based problem solving
My aim is to understand the mechanisms underlying these learning behaviours.
Classical ConditioningClassical Conditioning
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture. Salivation(Response)
Smell of Food (US)
Sound of Metronome (CS)
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Ivan P. Pavlov (1927)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Electric Shock (US)
Light touch (CS)
Withdrawal (Response)
QuickTime™ and a decompressor
are needed to see this picture.
Hawkins et al, 1989
Pre-Synaptic (Eccles) Post-Synaptic (Hebb)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Coincidence detectors
ParameciaParamecia
What could they learn?What could they learn?
Temperature change precedes O2 change in marine ecosystems by 20 minutes.
Photon flux may precede temperature changes.
Aerobic to anaerobic respiration from mouth to gut (signaled by increasing temperature).
Temperature change precedes O2 change in marine ecosystems by 20 minutes.
Photon flux may precede temperature changes.
Aerobic to anaerobic respiration from mouth to gut (signaled by increasing temperature).
QuickTime™ and a decompressor
are needed to see this picture.
David S Goodsell, 1998 “The Machinery of Life”
Previous WorkPrevious Work
QuickTime™ and a decompressor
are needed to see this picture.
na = [A] + [AB]
nb = [B] + [AB]Gandhi et al, 2007
A Simple Learning CircuitA Simple Learning Circuit
In collaboration with molecular biologists, I have designed Hebbian learning circuits for plasmids carried by E. coli.
In collaboration with molecular biologists, I have designed Hebbian learning circuits for plasmids carried by E. coli.
v = w.u
dwi/dt = uiv
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
A Gene Regulatory Network
No pairing Pairing
Output P
Artificial Evolution in Silico
SBMLEvolver Synthetic Biology Toolbox
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Evolution in vivo
What will evolve?
Fitness functionFitness function
€
fitness =cconst + T7(P)
min. P if nosignal is active6 7 8
(T8(P)− T7(P))maximise learning intest caseSet to 0, if < 0
1 2 4 4 3 4 4 (T8(P)−C8(P))
minimise learning incontrol caseSet to 0, if < 0
1 2 4 4 3 4 4 (T2(P)− T4(P))min. output as responseto U2 compared to U1 Set to 0, if < 0
1 2 4 4 3 4 4 + T2(P)− T8(P)
equal output as responseto U1 before and U2after learning
1 2 4 4 3 4 4 + a+ b+ c+ d
punishments1 2 4 3 4
€
dwijdt
= η o(vi − vθ )uj
€
dwijdt
= η o(wmax − wij )viuj
€
dwijdt
= η oviuj − αvi2wij
€
dwijdt
= η oviuj − αwij(1− wij )(wθ − wij )
€
dwijdt
= η oφ(vi − vi )uj
€
d vidt
= nv(vi2 − vi )
Presynaptic Subtractive Norm Oja (L2 Norm)
Weight consolidationBCM
MAPK ImplementationMAPK Implementation
Why?
Later….
C
A
B
R
C
A
B
R
Learns patient specific (contingent) associations
Susan
Jeffery
Similar Recent Work…Similar Recent Work…
Prediction, but no associative learning. Prediction, but no associative learning.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Tagkopoulos et al, 2008 (in press)
Limitations of Associative Learning
Limitations of Associative Learning
Can only learn associations between pre-specified stimuli
Learns only associations, not cause and effect relationships.
Can only learn associations between pre-specified stimuli
Learns only associations, not cause and effect relationships.
“A Concept of Cause and Effect”
“A Concept of Cause and Effect”
“My argument is that causal understanding gave rise to tool-making; that was the evolutionary advantage. It's tool-making that's really driven human evolution. This is not widely accepted, I'm afraid, but there's no question about it. It's tools that really made us human. They may even have given rise to language.”
“My argument is that causal understanding gave rise to tool-making; that was the evolutionary advantage. It's tool-making that's really driven human evolution. This is not widely accepted, I'm afraid, but there's no question about it. It's tools that really made us human. They may even have given rise to language.”
Lewis Wolpert, 2007
What is causal Inference?What is causal Inference?
Does dropping a coin into a tin of coins cause the number of coins in the tin go up? Can moving a piece on a chess-board cause the opponent's queen to be pinned? Can ignorance cause poverty? Can poverty cause crime? Can ignorance cause a TV set to be moved through a broken window? Can inserting a certain sort of twig in a certain way into a particular partly built nest cause the
nest to become more rigid?
Analysing the concept of causation is probably the hardest unsolved philosophical problem. It's at the root of problems about relations between mind and body (or relations between virtual and physical machines).
Does dropping a coin into a tin of coins cause the number of coins in the tin go up? Can moving a piece on a chess-board cause the opponent's queen to be pinned? Can ignorance cause poverty? Can poverty cause crime? Can ignorance cause a TV set to be moved through a broken window? Can inserting a certain sort of twig in a certain way into a particular partly built nest cause the
nest to become more rigid?
Analysing the concept of causation is probably the hardest unsolved philosophical problem. It's at the root of problems about relations between mind and body (or relations between virtual and physical machines).
Sloman, 2008 (pc)
Causal InferenceCausal Inference
What is the difference between causal inference and associative learning?
Weak: To utilize more than pair-wise correlations (perhaps unconsciously).
Strong: Combining observation of conditional probability P(X|Y) with novel appropriate interventions i.e. why don’t Pavlov’s dogs spontaneously ring the bell
when they are hungry? (without reinforcement). Humans do generate hypotheses based on CP and
produce interventions to test causal models. Parties -> Wine -> Insomnia Wine <- Parties -> Insomnia
What is the difference between causal inference and associative learning?
Weak: To utilize more than pair-wise correlations (perhaps unconsciously).
Strong: Combining observation of conditional probability P(X|Y) with novel appropriate interventions i.e. why don’t Pavlov’s dogs spontaneously ring the bell
when they are hungry? (without reinforcement). Humans do generate hypotheses based on CP and
produce interventions to test causal models. Parties -> Wine -> Insomnia Wine <- Parties -> Insomnia
Structuring InterventionsStructuring Interventions
A --> B --> C --> D Intervene at C: A-> B C-->D
A <-- B <-- C <-- D Intervene at C: A <-- B <-- C D
A --> B --> C --> D Intervene at C: A-> B C-->D
A <-- B <-- C <-- D Intervene at C: A <-- B <-- C D
Algorithms exist to discover causal networks.
Algorithms exist to discover causal networks.
Bayesian learning Know prior probability of causal graphs Know probability of observations given each graph Use Bayes theorum to calculate probability of graph given observations and priors Fined the best graph
Constraint-based learning For each pair of variables a and b in V, search for a set Sab such that (a ||_ b |Sab) holds in
P, i.e. a and b should be independent in P, conditioned on Sab. Construct an undirected graph G such that vertices a and b are connected with an edge iff no set Sab can be found. Connect dependent nodes
For each pair of non-adjacent variables a and b with a common neighbor c check if c is an element of Sab. If it is continue, if it is not then add arrow heads pointing at c i.e. a--> c <-- b.
In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions:
i. the orientation should not create a new v-structure, ii.the orientation should not create a directed cycle.
Bayesian learning Know prior probability of causal graphs Know probability of observations given each graph Use Bayes theorum to calculate probability of graph given observations and priors Fined the best graph
Constraint-based learning For each pair of variables a and b in V, search for a set Sab such that (a ||_ b |Sab) holds in
P, i.e. a and b should be independent in P, conditioned on Sab. Construct an undirected graph G such that vertices a and b are connected with an edge iff no set Sab can be found. Connect dependent nodes
For each pair of non-adjacent variables a and b with a common neighbor c check if c is an element of Sab. If it is continue, if it is not then add arrow heads pointing at c i.e. a--> c <-- b.
In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions:
i. the orientation should not create a new v-structure, ii.the orientation should not create a directed cycle.
QuickTime™ and a decompressor
are needed to see this picture.
CrowsCrows
What is the evidence for causal understanding in crows?
What is the evidence for causal understanding in crows?
QuickTime™ and aH.263 decompressor
are needed to see this picture.
I’m not convinced…I’m not convinced…
Tool use is innate in New Calodonian crows; no social learning is required.
Crows can make the right length and thickness of tool for the right hole, on the first trial. This could still be associative learning.
Must exclude simple strategies, e.g. random search, win-stay loose shift, reinforcement learning, operant conditioning, etc… “Rather than giving subjects a defined set of choices, they are placed in a situation
where they have a low probability of solving a task by chance alone (for example, in a hook making task an animal may be given a piece of pliable material that can be changed into an infinite number of shapes, but only a small subset of these shapes would be functional)”
How does one define the null-hypothesis, e.g. what is the probability of manufacturing a hook-shaped object by chance alone?
Tool use is innate in New Calodonian crows; no social learning is required.
Crows can make the right length and thickness of tool for the right hole, on the first trial. This could still be associative learning.
Must exclude simple strategies, e.g. random search, win-stay loose shift, reinforcement learning, operant conditioning, etc… “Rather than giving subjects a defined set of choices, they are placed in a situation
where they have a low probability of solving a task by chance alone (for example, in a hook making task an animal may be given a piece of pliable material that can be changed into an infinite number of shapes, but only a small subset of these shapes would be functional)”
How does one define the null-hypothesis, e.g. what is the probability of manufacturing a hook-shaped object by chance alone?
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Both Betty and Bob use trial and error search.
Causal Inference in RatsCausal Inference in Rats
What is the evidence for causal inference in rats?
What is the evidence for causal inference in rats?
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Causal Inference in ChildrenCausal Inference in Children
What is the evidence for causal inference in humans?
Understanding interventions (monkey sneezing + blickets, etc….) A+ --> B- --> AB+ --> AB+ (Children choose A) A+ --> A+ --> A+ --> B- --> B+ --> B+ (Choose randomly)
Retrospective disambiguation (by children) e.g. AB+ --> A-, AB+ --> A+
What is the evidence for causal inference in humans?
Understanding interventions (monkey sneezing + blickets, etc….) A+ --> B- --> AB+ --> AB+ (Children choose A) A+ --> A+ --> A+ --> B- --> B+ --> B+ (Choose randomly)
Retrospective disambiguation (by children) e.g. AB+ --> A-, AB+ --> A+
QuickTime™ and a decompressor
are needed to see this picture.
Gopnik & Schultz, 2004
QuickTime™ and a decompressor
are needed to see this picture.
Gopnik & Schultz, 2004
QuickTime™ and a decompressor
are needed to see this picture.
Gopnik & Schultz, 2004
Our approachOur approach
To study intra-brain causal inference. To study intra-brain causal inference.
Insight in HumansInsight in Humans
QuickTime™ and a decompressor
are needed to see this picture.
"Draw four continuous straight lines, connecting all the dots without lifting your
pencil from the paper."
How to Solve it?How to Solve it?
What is an ‘insight problem’? A problem that requires restructuring of the initial problem representation, e.g. goal states, operators, constraints.
What kinds of algorithm are used to solve these and related problems? What determines the set of goal, operators, constraints?
What is an ‘insight problem’? A problem that requires restructuring of the initial problem representation, e.g. goal states, operators, constraints.
What kinds of algorithm are used to solve these and related problems? What determines the set of goal, operators, constraints?
Missionaries and CannibalsMissionaries and Cannibals
3 missionaries and 3 cannibals must cross a river using a boat which can carry at most two people.
For both banks, if there are missionaries present on the bank, they cannot be outnumbered by cannibals.
The boat cannot cross the river by itself with no people on board.
3 missionaries and 3 cannibals must cross a river using a boat which can carry at most two people.
For both banks, if there are missionaries present on the bank, they cannot be outnumbered by cannibals.
The boat cannot cross the river by itself with no people on board.
Intermediate goals may be usedIntermediate goals may be used
Early moves balance number of M & C on each side of river.
Intermediate moves maximize progress from one side to other.
Later moves avoid revisiting previous states.
Early moves balance number of M & C on each side of river.
Intermediate moves maximize progress from one side to other.
Later moves avoid revisiting previous states.
There is some evidence…There is some evidence…
Non-maximal moves that allow a subsequent move to make more progress are retained as promising states for future trials.
Goal criteria are relaxed and changed based on the quality (immediate benefit) of generated solutions.
Sometimes hill-climbing to a ‘wrong’ goal criteria can get stuck in local minima.
Non-maximal moves that allow a subsequent move to make more progress are retained as promising states for future trials.
Goal criteria are relaxed and changed based on the quality (immediate benefit) of generated solutions.
Sometimes hill-climbing to a ‘wrong’ goal criteria can get stuck in local minima.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
3 moves
CHRONICLE, MACGREGOR, AND ORMEROD,2004
-7426 legal 3-move sequences-2 reach ring solution-176 reach 2 group solution-7426 sequences are not eqiprobable under random selection assumption. -< 1/3 participants solve problem within 10 minutes.
-Choice of the correct first move based on the improved goal scores available from the second move was crucial.
-Few subjects even conceived of a two group solution when asked to “produce a shape where each coin only touches two others”.
Brain damage helps some problem solving!
Brain damage helps some problem solving!
II = III + I Type A
IV = III - I Type B
VI = VI + VI Type C
QuickTime™ and a decompressor
are needed to see this picture.
Solutions here
QuickTime™ and a decompressor
are needed to see this picture.
Reverberi et al 2005
Our approachOur approach
To study mechanisms for restructuring of problem representations.
To study mechanisms for restructuring of problem representations.
QuickTime™ and a decompressor
are needed to see this picture.
Poelwijk et al 2007
ConclusionsConclusions
What neural mechanisms underlie causal inference, and solving insight problems?
What changes allow humans to have these capacities but precludes other apes from having them?
What algorithms can best predict human performance in such problems?
What neural mechanisms underlie causal inference, and solving insight problems?
What changes allow humans to have these capacities but precludes other apes from having them?
What algorithms can best predict human performance in such problems?
Thanks toThanks to
Eors Szathmary Lewis Bingle Anthony Liekens Aaron Sloman Jon Rowe Dov Stekel Christian Beck & Thorsten Lenser
Eors Szathmary Lewis Bingle Anthony Liekens Aaron Sloman Jon Rowe Dov Stekel Christian Beck & Thorsten Lenser
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressorare needed to see this picture.
QuickTime™ and a decompressorare needed to see this picture.