Computers - Using your Brains

Jim Austin

Professor of Neural Computation

Pentium III

So how complex is it ?

– 1012 neurons … 1,000,000,000,000–1000 connections between neurons.

– One brain can hold ...

1,000,000,000,000,000 numbers!

What do 1012 neurons look like ?

• 1600 times Population of the world (6,100,000,000)

• 78,125 times the complexity of the Pentium III

• Equal to the number of stars in our galaxy

4 Meters

4 Meters

4 Meters

The good and the bad

What are computers bad at ?Being reliableFinding information - knowledgeDoing very complex things - recognizing imagesLearning to do the job them selves!

What are computers good at ?Adding up fastStoring data - numbers and factsPushing data around

Why are computers so restricted ?

ACE

Leo - for stock control

Colossus - for breaking codes

Pegusus - for scientific work

Neurons verses Gates

NAND Gate

Input 1Input 2

Output

Boolean Logic - both inputs OK, output not OK

Gates - NAND

Input 1 Input 2 Output

==

=

ALL inputs to be OK for output to be NOT OK

Evolution ?

Should have picked a NAND gate for the brain...

+

“Weights”

Inputs OutputA

B

Threshold logic - threshold 1 - one or more inputs OK output OK

Output = threshold (input A x weight A + input B x weight B)

Neuron

Neuron

==

=

At least three OK’s for output to be OK

At least one OK for output to be OK

Can also alter connections/importance of inputs

using the weights on the inputs

+3.5

1

0

1

1

0.5

1

1

Weights

Why did this difference develop ?

• “The analysis of the operation of a machine using two indication elements and signals can be conveniently be expressed in terms of a diagrammatic notation introduced, in this context, by Von Neuman and extended by Turing. This was adopted from a notation used by Pits and McCulloch as a possible way of analyzing the operation of the nervous system,…” Calculating Instruments & Machines, D Hartree, 1950, Cambridge University Press.

• Probably dropped due to the development of the silicon chip– simpler to build Boolean logic gates rather than neuron

units.

Functional elements.

nz

k inputs

Threshold n gatek n

1z Excitation, “OR”

2z

Excitation, “AND”

ICT Orion Computer• Used ‘Neuron’ logic - 1962

Learning !

Learning at neuron level =

Adjustment of which inputs are important

Conventional computers have no implicit learning ability

Spot the difference

Happy

Hungry

+

+

Threshold = 2

+

+ Hungry

Happy

+

+ hungry

Happy

Can we build useful systems with neurons ?

Better tolerance to failureParallelism/use of threshold logic/distributed memory

Faster operationMassive parallelism

Better access to uncertain informationThreshold logic/neurons

Where the inputs are uncertainThreshold logic/neurons.

Where we want low powerAsynchronous systems

AdaptabilityUse of weights and learning methods.

So what have we done with these ?

Cortex-1

28 Processor cards, each holding 128 hardware neurons.

Each with 1,000,000,000 weights.

16MHz.

PCI based card.

Complete Machine:

400,000,000 neuron evaluations per second28,000 inputs30 bits set on input1,000,000 neurons.

Cortex-1 node

5,120,000,000 neuron weights, 640 neurons.

Recognising Addresses for the Post Office

Recognising trademarks

Text search engines

• Tolerant to spelling errors.

• Finds similar words to those supplied, for example chair, seat, bench.

• Learns these similarities automatically from text.

• Uses neural engine for document storage.

• Estimated 400,000,000 documents searched per second.

Molecular Databases• One of few systems that deal with the full 3D molecule

Query

Good matches

Bad Match

(It’s Brains from Thunderbirds !)

Aaron Turner

Mick Turner

Vicky Hodge

Julian Young

Anthony Moulds

Zyg Ulanowski

Ken Lees

Michael Weeks

Sujeewa Alwis

John Kennedy

David Lomas

and many others ….

Thanks...