The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

26
The Perceptron
  • date post

    21-Dec-2015
  • Category

    Documents

  • view

    214
  • download

    2

Transcript of The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Page 1: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The Perceptron

Page 2: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

( )K tD

0 T

Affe

ren

ts

Vthr

V rest

tmax 0 tD

What does a neuron do?

spike

no spike

Page 3: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Affe

ren

ts

0 Ttmax-

VthrNull

We consider a simplified case: input is synchronous

Page 4: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Affe

ren

ts

Alternatively, input is constant

Page 5: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The perceptron

j jj

h W X 11 sgn

2 Y h

1X 2X 3X 4X

Y1W 4W

11 sgn

2

Y W X

Page 6: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Geometrical interpretation

11 sgn

21

sgn cos2

Y W X

1X 2X

Y

1W 2W

W

1W

2W

X

1X

2X

Page 7: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The perceptron

The Perceptron categorizes the space of inputs into inputs that should evoke a response and inputs that should not evoke a response

Page 8: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Constraints on possible categorizations

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

1X

2X

Page 9: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Constraints on possible categorizations

1X

2X

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

Page 10: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Constraints on possible categorizations

1X

2X

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

Page 11: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Constraints on possible categorizations

1X

2X

22 1 0 X X

Page 12: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Solution: change of coordination

21X

2X

22 1 0 X X

Page 13: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Solution: change of coordination

More complicated rules can be realized if an additional non-linear layer is added

Page 14: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.
Page 15: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Deerinck 2002

Page 16: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.
Page 17: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Llinas 1975

Ramon Y Cajal

Page 18: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Llinas 1975

Increasing network capacity?

Page 19: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The perceptron Learning algorithm

1 11 sgn 1 sgn

2 2

i i

i

Y W X W X

Page 20: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The perceptron Learning algorithm

• Algorithm starts with an arbitrary set of weights

• Examples are presented one by one

• If the Perceptron correctly classifies the example no change in synaptic weights

• If the Perceptron does not correctly classify the example then make a Hebbian change in weights:

Page 21: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

The perceptron Learning algorithm

• If the example is to be classifies as ‘1’:

i i iW W X

• If the example is to be classifies as ‘0’:

i i iW W X

Page 22: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Perceptron.m

Page 23: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Hebbian plasticity and unsupervised learning

Page 24: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

1X 2X 3X 4X

Y1W 4W

Unsupervised learning in linear neurons

Y W X

Page 25: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Hebbian plasticity

1 i i iW n W n X n Y n

Wi(n+1) = efficacy of synapse i after n updates

1X 2X 3X 4X

Y1W 4W

Y W X

is the plasticity rate

Page 26: The Perceptron. 0T Afferents V thr V rest t max 0 What does a neuron do? spike no spike.

Geometrical interpretation

1X 2X

Y

1W 2W

W

1W

2W

X

1X

2X