Cognition Through Imagination and Affect

Murray ShanahanImperial College London

Department of Computing

2

Overview

Brain-inspired architectures

Cognitively mediated action

An internal sensorimotor loop

3

Brain-inspired ArchitecturesProgress towards the vision of human-level AI has been slow

Classical AI has not yet succeeded in devising systems that can match the common sense reasoning skills of a young child

Biologically-inspired AI has been very slow to move beyond trivial motor tasks and tackle the difficult questions of cognition

Some researchers are now turning to the human brain for inspiration, especially to architecture-level theories of its functioning

4

A New VocabularyWe have a whole new set of concepts to explore

But all should be in scare quotes

Many are alien to both top-down classical AI and bottom-up biologically-inspired AI

“Imagination”

“Emotion”

“Consciousness”

Or, more technicallyInternally closed sensorimotor loops

Affect-based mechanisms of selection

Global workspaces

5

Imagination and AffectHere we have an internally-closed sensorimotor loop that can simulate interaction with the environment

It rehearses trajectories through sensorimotor space without having to traverse those trajectories for real

The outcome of various potential trajectories can be evaluated. This where affect comes in

The result impacts on action selection

Motorcortex

Affect

WORLD

Inner sensorimotor loop(the “core circuit”)

Sensorycortex

ACa ACb

6

Why an Internal Loop?The inner sensorimotor loop implements a form of analogical representation

The medium of representation has the same structure as what is being represented – eg: a map

We get spatial properties for free, and complex shapes can be represented

The dynamics of the inner loop has a close relationship to the dynamics of the outer loop

It can realise inner speech as well as mental imagery

Categories become attractors in a state space having same structure as that of sensory input

This addresses the symbol grounding problem

7

A Cognitively-mediated ActionOn sight of green, turn-right is action has highest “salience”

But this reactive response is held on veto while turning right is rehearsed

Sight of red of predicted

But red is aversive

So salience of turn-right is modulated down, resulting in turn-left becoming the action with highest salience

Again this response is held on veto

Now sight of blue is predicted, and blue is associated with reward

So salience of turn-left is modulated up

Eventually it reaches a threshold, veto is released, and robot acts

8

The Core Circuit

VC / IT = visual cortex / inferior temporal cortex

AC = association cortex

GW / BG = global workspace / basal ganglia

Am = amygdala

This “core circuit” combines an internal sensorimotor loop with mechanisms for broadcast and competition, and thereby marries the simulation

hypothesis with global workspace theory

GW / BG

AC1a

AC2a

AC3a

AC1b

AC2b

AC3b

VC / IT

Am

9

Affect Circuitry (Am)

Reward

Punish

Salience1

Salience2

Salience3

VC / IT

GWBG

VC / IT = visual cortex / infero-temporal cortex, BG = basal ganglia,GW = global workspace

10

Motor Circuitry

MC = motor cortex

BG = basal ganglia

Am = amygdala

MC1

MC2

MC3

BG

Veto

Selectedactionbuffer

Motorcommand

VC / IT

Am

Robotmotor

controllers

Newaction

detector

VC / IT

Urgency

11

Concluding RemarksThe brain-inspired approach to building cognitive systems is promising

But it is still relatively unexplored

Affect plays a vital role in the proposed architecture

It is currently a simple scalar value. A vector of “basic emotions” would be interesting to investigate

The relationship to consciousness is very interesting

Too bad there’s no time to talk about it

Shanahan, M.P. (2006). A Cognitive Architecture that Combines Inner Rehearsal with a Global Workspace. Consciousness and Cognition 15, 433–449.