Circus, Circuits

Interesting Neural Networks:Some actually occur in brains; some are

hypotheses

Owl Audition• The Barn Owl uses delay lines and coincidence detectors (neurons that only

fire when both pre-synaptic axons are simultaneously depolarized) to locate objects in horizontal and vertical plane.

B

C

D

A

E

From Left Ear

From Right Ear

Far Right

Far Left

Center-Surround Cells

C

C C

C

S

S

S

S

S

S

S

S

SS

Visual Field

S

C Center

Surround

Inhibit

CS

CS firing pattern

Stim

ulu s

“ON center OFF surround” cell

Retina

Strong

Medium

Weak

Ganglia

Brain

On-Center -vs- Off-Center Retinal Ganglion Cells• The primary visual receptors (rods & cones) actually turn OFF when hit by

photons (light) and are ON when they detect dark spots (Hubel, Eye, Brain and Vision, 1988, pg. 54)

Receptors

Bipolar Cells

Retinal Ganglion

Light

Receptors

Light

CC C

C

S

S

S

S

SS

S

SSS

On-Center (Off-Surround) Off-Center (On-Surround)

These are non-intersectingpathways but are drawntogether to illustrate theirsimilarities.

ExciteInhibit

Line Detectors

On-Centers

Off-Centers

45o Line

Retinal Ganglia

Visual Cortex

To higher levelsof the visual cortex

Motion DetectorsRiechard Detector (1961) - based on the fly’s visual system

Delay

Delay

t1 t2 t3+

Delay

450 bar movingleft to right

t2+

t3+2

• Works best when delay = t2 - t1 = t3 - t2• = normal (non-delayed) transmission time

Coincidence detectors =>only fire when all inputsare ON simultaneously.

t3

Lateral Inhibition Lines

• Neurons that stimulate themselves and inhibit their near neighbors function as filters

1 2 3 4 5

Firi n

g R

ate

Neuron

Firin

g R

ate

Input

Output

ExciteInhibit

Lateral Inhibition in Visual Pathways• Grossberg, S. (2003) in The Handbook of Brain Theory and Neural

Networks, pp. 594-600.

Retinal ON Cell

LGN

V1(6)

V1(2/3)

V1(4)

V2(6)

V2(4)

V2(2/3)

ExciteInhibitInterneuron• 6 - 4 - 2/3 pathway/loop is self-excitatory

• Similar lateral inhib topology in V1 & V2

Central Pattern Generators (CPGs)Neural circuits for generating simple, repeated patterns of activity.E.g. gait patterns in N-legged animals.Ian Steward (1998). Life’s other secret. Ch.9

Overhead view ofhorse, goat, dog??

t2

t3t1

t4

Walking gait: First move left rear leg, then left front, then right rear, then right front.

1/4

1/20

3/4

Standard Notation:Fractions = Phase diffs

Generic Gait Generator• Each animal species can perform many different gaits.• Do we need a different wiring pattern for each gait?• No! (Golubitsky, Stewart, Collins, Buono (1997))• Goal: A single circuit with adjustable delay times.• Solution: For an N-legged animal, 2 cross-linked N-neuron loops.

LeftFront

RightRear

RightFront

LeftRear

AR1

AL2

AL1

AR2

Inter-loop delay

Intra-loop delay

By adjusting theseTWO delay times,we can generateall standard gait patterns forN-legged animals!!

1/4

1/2

3/4

0

0

3/4

1/2

1/4

1/41/2

Walking

1/4

0

1/4

0

1/2

3/4

1/2

3/4

1/40

Jumping

1/20

1/2

0

0

1/2

01/2

Pacing

1/2

1/2

0

0

1/2

1/2

0

0

1/21/2

Trotting

0 1/2

Brain Clocks• Wright, Karen,”Times of our Lives”, Scientific American, Sept. 2002• In the cerebral cortex, a collection of neurons with different firing

patterns enables us to record and reuse specific time intervals.

A

B

D

C

t1 t2 t3

Time Signatures

A B C D

t1 0 1 0 1

t2 1 1 1 0

t3 1 1 0 0

t4 0 0 1 1

t4

Timing Circuit

A B

C

D

Cerebral Cortex

Neural Oscillators from 10-40 Hz

S

StriatumSNc

SNr

STN

DopamineSignal => Learn!Excite

Inhibit

1. A start signal (e.g. Dance instructor says ”Begin”): STN excites SNr, which then inhibits all cortical oscillators, so they essentially RESET to off.

2. Oscillators then resume their normal diverse firing patterns, from same init state.

3. A stop signal (e.g. Dance instructor…): SNc releases dopamine into striatum, causing striatal cells to record the current time signature via Hebbian Learning

Learning a Time Signature

• Non-associate Learning: Strengthen pre-synaptic axon since:a) it fired/depolarized, and b) significant event (STOP) signalled.

• After learning, S will only fire when B & D are active (i.e. after a time interval of duration = t1). Details are unclear as to whether A & C develop inhibitory links to S.

• In future (e.g. when repeating the dance), the instructor still says ”Go”, which again resets the cortical oscillators, but now the brain generates its own ”STOP” signal in the striatum, when S fires => student has learned t1!

• Given enough diverse oscillators, student can learn ANY interval.

B C

D

A

S

B C

D

A

S

Low

High Low

High

STOP!!LEARN!!

Cricket Phonotaxis• Webb, B. (2001). Biorobotics: Methods & Applications, Ch. 1.• Female Crickets only respond to songs with particular carrier

frequencies and syllable durations.

LeftEar Drum

RightEar Drum

• Syllable Duration• Carrying Frequency = 1/Inter-syllable period

BugOff!

Preferred Carrier FrequencyDistance between the two ear-drums is the critical determinant. If it’s ONE

QUARTER the song’s inter-syllable wavelength, then the eardrums vibrate most strongly. Here P = period of the sound wave.

Peak

Trough

Time T

Time T+P/2

• From T to T+P/4, the peak travels across the body and meets the right eardrum, causing it to vibrate, thus generating a new peak.

• From T+P/4 to T+P/2, the new peak travels exactly 1/4 wavelength = ear-to-ear distance.

• At time T+P/2, the left ear has a) a trough on the outside, and b) a peak on the inside.

• That’s a max pressure difference => the eardrum is maximally stimulated.

• The cricket is happy!!

Eardrums R L

Preferred Syllable Duration

• Appears to be determined in the brain, but details only partially known.• Biorobotics researchers (Webb et. al.) provide minimal ANNs that are

sufficient explanations.

ANR

RightEar

LeftEar

ANL

MNR MNL

AuditoryNeurons

MotorNeurons

Turn Left

Turn Right

• Each auditory neuron stimulates the corresponding motor neuron and inhibits the opposite motor neuron.

• Each of the 4 neurons has a very detailed (but standard) model: leaky integrate-and-fire

• AN => MN synapses are temporarily depressed after the AN fires

Leaky Integrate-and-Fire Neural Models

tmdVi/dt = b(EL - Vi) + awijzj

zj = (1 + eVi)-1 {Standard sigmoidal transfer function}

Vi = voltage inside the neuronEL = voltage outside the neuron (standard value: -55mV)

zj = firing rate of neuron j

wij = synaptic weight from neuron j to neuron i.

a: excitation factor, b: leakage factor, tm = time scaling factor

Vi

EL

z1*wi1

z2*wi2

z3*wi3

zi

Leak

Leak Integrate

AP = Voltage Spike• Although the voltage of a neuron changes constantly, only large abrupt

changes (action potentials) can be transmitted to other neurons.

0 mV

+40 mV

-65 mVResting Potential

Overshoot

RisingPhase

FallingPhase

Undershoot

Na+ gates open.Na+ enters cell.

K+ gates open.K+ leaves cell.Na+ gates still open

Na+ gates close.K+ gates still open.

K+ gates close.

Habituation

When a neuron fires weakly, but frequently, its axonal synapses weaken.After a little rest, the synapse returns to normal strength.

tmdwij/dt = c(wij(*) - wij) - S(zj)

wij(*): base value for wij

S(zj) = stimulus function; lower zj => higher S

Vj

zj

Vi

S

wij

z j

t

wij

t

Preferred Syllable Duration• Assume a stimulus on the left side of the cricket. • High frequency (short wavelength) sound has a quickly-decaying amplitude

with distance, so the left ear gets a stronger signal than the right.

• The cricket turns left. It is attracted to the song.

ANL Response

Incoming soundSyllable

• Neuron ANL integrates the inputs from the left ear drum and fires groups of pulses with durations = syllable durations.

• This inhibits motor neuron MNR but stimulates MNL, which integrates the inputs from ANL and eventually begins to fire. However, it integrates more slowly than ANL and therefore fires less frequently.

MNL Response

Null Poeng• Stimulus again from left side, but now the syllables are very short and frequent..

• The cricket is not interested.

ANL Response

Incoming soundSyllable

• Neuron ANL integrates the inputs from the left ear drum and fires constantly, with very few significant gaps.

MNL Response

• This inhibits motor neuron MNR and stimulates MNL.• But, now the ANL-MNL synapse habituates due to the constant firing of

ANL (and hence no break in which to regain strength).• So the signals that ANL sends to MNL are WEAK, and MNL never integrates

enough charge to fire.

Another Loser• Stimulus again from left side, but now the syllables are very long, with a large gap between syllables..

• This is cricket is very picky!

ANL Response

Incoming soundSyllable

• Neuron ANL integrates the inputs from the left ear drum and fires long sets of pulses with long gaps.

MNL Response

• This inhibits motor neuron MNR and stimulates MNL.• But, now the gap is too long: MNL almost fires during a syllable, but then a

lot of voltage LEAKS out during the inter-syllable gap. • So although ANL’s signals are strong, MNL leaks too much and can never

integrate enough charge to fire.