Abstract: This overview of the free energy principle offers an account of embodied exchange with the...
-
Upload
howard-miles-sutton -
Category
Documents
-
view
213 -
download
0
Transcript of Abstract: This overview of the free energy principle offers an account of embodied exchange with the...
Abstract: This overview of the free energy principle offers an account of embodied exchange with the world that associates conscious operations with actively inferring the causes of our sensations. Its agenda is to link formal (mathematical) descriptions of dynamical systems to a description of perception in terms of beliefs and goals. The argument has two parts: the first calls on the lawful dynamics of any (weakly mixing) ergodic system from a single cell organism to a human brain. These lawful dynamics suggest that (internal) states can be interpreted as modelling or predicting the (external) causes of sensory fluctuations. In other words, if a system exists, its internal states must encode probabilistic beliefs about external states. Heuristically, this means that if I exist (am) then I must have beliefs (think). The second part of the argument is that the only tenable beliefs I can entertain about myself are that I exist. This may seem rather obvious; however, if we associate existing with ergodicity, then (ergodic) systems that exist by predicting external states can only possess prior beliefs that their environment is predictable. It transpires that this is equivalent to believing that the world and the way it is sampled will resolve uncertainty about the causes of sensations. We will conclude by looking at the epistemic behaviour that emerges under these beliefs, using simulations of active inference.Key words: active inference autopoiesis cognitive dynamics free energy epistemic value self-organization.
I am therefore I think Karl Friston, University College London
Free Energy Workshop 2015
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
How can the events in space and time which take place within the spatial boundary of a living organism be accounted for by physics and chemistry? (Erwin Schrdinger 1943)
The Markov blanket as a statistical boundary (parents, children and parents of children)Internal states External statesSensory statesActive statesThe Markov blanket in biotic systems
Active states
External statesInternal statesSensory states
The Fokker-Planck equation
And its solution in terms of curl-free and divergence-free componentslemma: any (ergodic random) dynamical system (m) that possesses a Markov blanket will appear to engage in active inference
5But what about the Markov blanket?Reinforcement learning, optimal control and expected utility theory
Information theory and minimum redundancy
Self-organisation, synergetics and homoeostasis
Bayesian brain, active inference and predictive coding
Value
Surprise
Entropy
Model evidence
PavlovHakenHelmholtz
Barlow
PerceptionAction
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Position
Simulations of a (prebiotic) primordial soupWeak electrochemical attractionStrong repulsionShort-range forces
ElementAdjacency matrix2040608010012020406080100120Markov BlanketHidden statesSensory statesActive statesInternal states
Markov Blanket = [B [eig(B) > ]]Markov blanket matrix: encoding the children, parents and parents of childrenFinding the (principal) Markov blanketADoes action maintain the structural and functional integrity of the Markov blanket (autopoiesis) ?
Do internal states appear to infer the hidden causes of sensory states (active inference) ?
Autopoiesis, oscillator death and simulated brain lesionsDecoding through the Markov blanket and simulated brain activation100200300400500-0.4-0.3-0.2-0.10TimeMotion of external stateTrue and predicted motion -505-8-6-4-20468PositionPositionPredictability2
TimeModesInternal states10020030040050051015202530
Christiaan Huygens
The existence of a Markov blanket necessarily implies a partition of states into internal states, their Markov blanket (sensory and active states) and external or hidden states.
Because active states change but are not changed by external states they minimize the entropy of internal states and their Markov blanket. This means action will appear to maintain the structural and functional integrity of the Markov blanket (autopoiesis).
Internal states appear to infer the hidden causes of sensory states (by maximizing Bayesian evidence) and influence those causes though action (active inference)
Interim summary
res extensa (extensive flow)res cogitans (beliefs)Belief productionFree energy functionalI am [ergodic] therefore I think
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism - von Helmholtz
Thomas BayesGeoffrey HintonRichard FeynmanThe Helmholtz machine and the Bayesian brainRichard Gregory
Hermann von Helmholtz Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism - von Helmholtz Richard Gregory
Hermann von Helmholtz Impressions on the Markov blanket
Bayesian filtering and predictive coding
prediction update
prediction error
Making our own sensations
Changing sensationssensations predictionsPrediction errorChanging predictionsActionPerception
the
DescendingpredictionsAscending prediction errors
A simple hierarchy
whatwhereSensory fluctuations
Hierarchical generative models
frontal eye fieldsgeniculatevisual cortexretinal inputponsoculomotor signalsTop-down or backward predictionsBottom-up or forward prediction errorproprioceptive inputreflex arcPerception
David MumfordPredictive coding with reflexesAction
Prediction error (superficial pyramidal cells)Expectations (deep pyramidal cells)
Biological agents minimize their average surprise (entropy)
They minimize surprise by suppressing prediction error
Prediction error can be reduced by changing predictions (perception)
Prediction error can be reduced by changing sensations (action)
Perception entails recurrent message passing to optimize predictions
Action makes predictions come true (and minimizes surprise)
Interim summary
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
ThalamusArea X
Higher vocal centreHypoglossal Nucleus
creating your own sensations: Listening
time (sec)Frequency (Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000
ThalamusArea X
Higher vocal centreHypoglossal Nucleus
creating your own sensations: SingingMotor commands (proprioceptive predictions)Corollary discharge(exteroceptive predictions)
time (sec)Frequency (Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000
time (sec)Frequency (Hz)percept1234567250030003500400045005000012345678-50050100time (seconds)First level expectations (hidden states)012345678-40-20020406080time (seconds)Second level expectations (hidden states)Singing alone
time (sec)Frequency (Hz)percept1234567250030003500400045005000012345678-50050100time (seconds)First level expectations (hidden states)012345678-40-20020406080time (seconds)Second level expectations (hidden states)Singing together
-20-100102030405060-20-100102030405060Synchronizationsecond level expectations (first bird)second level expectations (second bird)-20-100102030405060-30-20-1001020304050No synchronizationsecond level expectations (first bird)second level expectations (second bird)Mutual prediction and synchronization of chaossynchronization manifold
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
saliencevisual inputstimulussamplingPerception as hypothesis testing saccades as experiments
Sampling the world to minimise expected uncertainty
I am [ergodic] therefore I think I think therefore I am [ergodic]
LikelihoodEmpirical priorsPrior beliefs
Frontal eye fields
Pulvinar salience mapFusiform (what)Superior colliculusVisual cortexoculomotor reflex arc
Parietal (where)
30
Visual samplesConditional expectations about hidden (visual) statesAnd corresponding perceptSaccadic eye movementsHidden (oculomotor) states
Saccadic fixation and salience maps
200400600800100012001400-202Action (EOG)time (ms)200400600800100012001400-505Posterior belieftime (ms)
vs.vs.
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Generative modelsHidden states
Action
Control states
Continuous statesDiscrete statesBayesian filtering (predictive coding)Variational Bayes(belief updating)
Full priors control statesEmpirical priors hidden statesLikelihoodThe (normal form) generative modelHidden states
Action
Control states
KL or risk-sensitive control
In the absence of ambiguity:Expected utility theory
In the absence of uncertainty or risk:Bayesian surprise and Infomax
In the absence of prior beliefs about outcomes:
Prior beliefs about policiesQuality of a policy = (negative) Expected free energy
Extrinsic valueEpistemic value, information gain or reduction of expected uncertaintyPredicted divergenceExtrinsic valueBayesian surprisePredicted mutual information
midbrainmotor Cortexoccipital Cortexstriatum
Variational updatesPerception
Action selection
Incentive salienceFunctional anatomy
prefrontal Cortex
hippocampus
00.20.40.60.81020406080PerformancePrior preferencesuccess rate (%) FEKLRLDASimulated behaviour
Simulating conditioned responses and dopamine release (precision updates)
OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think
The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony
Active inferenceMinimizing expected uncertaintySaccadic searches and salience
Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision
Knowing your place Multi-agent gamesMorphogenesis
Generative modelIntercellular signalingIntrinsic signalsExogenous signalsEndogenous signals
Knowing your place: multi-agent games and morphogenesis
-4-2024-4-3-2-101234Extracellular target signalposition
(Genetic) encoding of target morphologyPosition (place code) Signal expression (genetic code)
-4-2024-4-3-2-101234Target formposition
05101520253035-4-3-2-101234Morphogenesistimelocation-4-2024-4-3-2-101234Solutionlocation51015202530-420-400-380-360-340Free energytimeFree energy
Softmax expectationscellcell123456781234567851015202530-1.5-1-0.500.511.522.53Expectationstime51015202530-2-1.5-1-0.500.511.522.5timeAction
Morphogenesis
DysmorphogenesisGradientGradientIntrinsicExtrinsicExtrinsicTARGET
Regeneration of head05101520253035-4-3-2-101234morphogenesistimelocation-4-2024-4-3-2-101234location05101520253035-4-3-2-101234morphogenesistimelocationRegeneration of tailRegeneration
Each movement we make by which we alter the appearance of objects should be thought of as an experiment designed to test whether we have understood correctly the invariant relations of the phenomena before us, that is, their existence in definite spatial relations.
The Facts of Perception (1878) in The Selected Writings of Hermann von Helmholtz,Ed.R. Karl, Middletown: Wesleyan University Press, 1971 p. 384
Hermann von Helmholtz And thanks to collaborators:
Rick AdamsRyszard AuksztulewiczAndre BastosSven BestmannHarriet BrownJean DaunizeauMark EdwardsChris FrithThomas FitzGeraldXiaosi GuStefan KiebelJames KilnerChristoph MathysJrmie MattoutRosalyn MoranDimitri OgnibeneSasha Ondobaka Will PennyGiovanni PezzuloLisa Quattrocki KnightFrancesco Rigoli Klaas StephanPhilipp Schwartenbeck
And colleagues:
Micah AllenFelix BlankenburgAndy ClarkPeter DayanRay DolanAllan HobsonPaul FletcherPascal FriesGeoffrey HintonJames HopkinsJakob HohwyMateus JoffilyHenry KennedySimon McGregorRead MontagueTobias NolteAnil SethMark SolmsPaul Verschure
And many othersThank you