Gordon Pipa Institute of Cognitive Science University of

Identification of complex biological Motion patterns

Gordon Pipa

Institute of Cognitive Science University of Osnabrück

With:

Matthias Staib (UOS) Frank Jäckel (UOS) Robert Haslinger (MIT)

The Brain: An ordered Hierarchical System

Blake 2001 Giese 2003

Giese 2003


Giese 2003

RF V1

RF V4

RF IT

Blake 2001


Giese 2003

RF V1

RF V4

RF IT

The Brain: Large, self-organized but mostly random Network

Giese 2003

RF V1

RF V4

RF IT

Hagmann, et al. (2008), ’Mapping the structural core of human

cerebral cortex’ PLoS Biol 6(7): e159

• Complex network with a high degree of randomness, and that is changing at all times

• Intrinsic activity dominates complex neuronal dynamics (Stimulus = Perturbation)

• Haslinger, R., Pipa, G., Lima, B., Singer, W., Brown, E. N., & Neuenschwander, S. (2012). Context Matters:

The Illusive Simplicity of Macaque V1 Receptive Fields. PLoS ONE, 7(7), e39699.

Biological Motion with Reservoir Computing

excitatory

inhibitory

Output

SORN:

• 25 bipolar cells (center surround RF), project randomly on excitatory neurons

• Reservoir: 100 exc. + 20 inh.

• Plasticity for Reservoir learning

• RF covers ~6% of visual field

• A. Lazar, G. Pipa, and J. Triesch, Frontiers Computational Neuroscience 2009

• R. Haslinger, G. Pipa, B. Lima, W. Singer, E.N. Brown, S. Neuenschwander (2012). PLoS ONE, 7(7), e39699

happy,

sad,

angry

Classification

Reservoir Computing

Incoming drive Threshold Input

task: Mood

Bipolar Cells

With center

surround

1

1 ( )N

i ij j i

j

h t f W t x t T t I t

• A. Lazar*, G. Pipa*, and J. Triesch (*authors contributed equally), Neural Networks, 20(3):312--322, 2007


RF covers 6%

Network structure

Network activity

Plasticity

time

un

it

Information processing

Two types of neuronal plasticity:

• Spike timing dependent plasticity (STDP) for network structure formation

• Intrinsic plasticity for homeostasis of average activity of neurons

• Unsupervised learning in the recurrent network

Self-organization of the network and network activity

• V. Gomez, A. Kaltenbrunner, V. Lopez, H. Kappen, ’Self-organization using synaptic plasticity’, NIPS 2008

• Savin C, Joshi P, Triesch J (2010) Independent Component Analysis in Spiking Neurons. PLoS Comput Biol 6(4)

• A. Lazar*, G. Pipa*, and J. Triesch (*authors contributed equally), Fading memory and time series prediction in

recurrent networks with different forms of plasticity, Neural Networks, 20(3):312--322, 2007

• Binarized STDP:

• Intrinsic plasticity:

• Synaptic scaling:

SORN – 3 plasticity mechanisms



bipolar cells

SORN:

• Bachelor Thesis in Cognitive Science, Osnabrück Germany by Matthias Staib (2013)


Cycle:

16 Frames

PreTraining:

500x3 cycles

Training:

100x3 cycles

Testing:

100x3 cycles

from 35 subjects

LSM (shuffeld

connectivity

and

thresholds)

SORN:

Learning differences in temporal patterns

happy

angry

sad

First 3 PCA: 52% of variance

Performance

Performance

• Similar performance for SORN and LSM with

shuffled weights and thresholds.

• RC properties, i.e. feature expansion and

memory are necessary

(Control Logistic classifier: ~55%)

• Why not just a random LSM ?

• What is the advantage of SORN that learns

temporal patterns based on plasticity ?



Occluder Task

Occluder Occluder Occluder sad happy angry

bipolar

cells

SORN:

Performance for occluder Task

• Initial fading memory the same for both the LSM and SORN

• After 15 frames memory maintains to be larger for SORN

• Distance between models (Hamming distance) larger for SORN

Generalisation for other viewing directions

Training orientations: 50°, 75°, 110°, 135°

Interpolation: Radom orientation in [55,° 70°] or [115° 130°]

Extrapolation: Radom orientation in [5,° 50°] or [140° 175°]

Interpolation

Extrapolation

Generalisation for other viewing directions

Conclusions

• Both the hierarchical and the RC based recognition of emotions work

well

• One to one comparison of performance is difficult since models are

rather different in structure

• Most of the activity even in early primary areas is not directly stimulus

related

• Classical approaches using feed forward structures explain this with

noise

• The RC approach provides an alternative that utilizes induced complex

temporal firing sequences to maintain memory, bridge lacking

stimulation, and optimize stimulus presentations.

• Processing a complex stimulus property such a mood can be

implemented with a single recuurent module, for example in V1.

Identification of complex biological Motion patterns

Gordon Pipa

Institute of Cognitive Science University of Osnabrück

With:

Matthias Staib (UOS) Frank Jäckel (UOS) Robert Haslinger (MIT)

Gordon Pipa Institute of Cognitive Science University of

Documents

Transcript of Gordon Pipa Institute of Cognitive Science University of