B.Macukow 1 Neural Networks Lecture 4. B.Macukow 2 McCulloch symbolism The symbolism introduced by...

B.Macukow 1

Neural Networks Neural Networks

Lecture 4

B.Macukow 2

McCulloch symbolismMcCulloch symbolism

The symbolism introduced by McCulloch at the basis of simplified Venn diagrams is very useful in the analysis of logical networks

Two areas X1 i X2 correspond to two argument logic function. Symbol X1 means the input signal x1 = 1, its complement - signal x1 = 0.The same for X2.

B.Macukow 3

We have four fragments denoted:

X1X2 , X1 ~X2 , ~ X1 X2 and ~ X1 ~ X2


B.Macukow 4

Instead of circles we can used crosses


B.Macukow 5

Symbolic notation – cross with dots

X1X2

X1X2= (X1 ~X2 ) ( ~ X1 X2 ) ( X1X2)

conjunction – AND operation

disjunction -

OR operation


B.Macukow 6

Symbolic notation – cross with dots

~X2

X1X2

negation –

NOT operation

implication


B.Macukow 7

The function depending of parameters

x1 x1x2 x2

x1 x2 ~x1 x2


B.Macukow 8

Operations performed

•

•

• •••

x1

x1

x2

(

(

x1 x1x2 x2 ~~(( ) )

)x3x2 x2 ~~ ~ )x3x2x1 ((([ x3x3 ]) ) x2 ~~

[x1 ~ ~ x2 x3( ) ]

x1 ~ )x3x2~~


B.Macukow 9

The whole algebra

•

•••

••

Proof:

••

x1 x2 x1x1 x2 ~~( ) x1 ~( )x2~

x2x1 x2 ~( ) x1 ~~( )x2~x1 x2 •

•

x2x1~•

~ x2x1~ ~ x1 •x2


B.Macukow 10

Analysis of the simple nets composed from logical neurons

x1

x4x3

x2

xout


B.Macukow 11

Simplified notation

x1

x4x3

x2

xout


B.Macukow 12

The middle cross denotes an operation performed on the two symbols on either side. For example, the operation below means the operation which is not entered in either symbol on the left or symbol on the right should be written down as the result.

••• •

•

••

•••

Proof •)x1 x2•

•(

x3

x1 x2•( )

x4


B.Macukow 13

x3 x2~ x4 ~ x1 x2

x3•

x4 ~~ x3 x4

~ ( (x2~ ) ) ~ ~ x1 x2

x2x1

x2 ( )x1 x2~

(( ) )x2 x1 x2 x2~

)x1 x2••

(

x3

x1 x2•( )

x4


B.Macukow 14

The net with the loops


B.Macukow 15

Threshold influence for the neuron reaction


B.Macukow 16

Applications of McCulloch symbols, operation on the symbols


B.Macukow 17

Use of McCulloch symbols to denote the function of a neuron


B.Macukow 18

Network for modeling the conditioned reflex network realized by a single unconditioned reflex (UR) and a

conditioned reflex (CR)


B.Macukow 19

Venn diagram and McCulloch symbols for three outputs. Unknown are marked by A, B and C.

McCulloch symbolism for three outputs

McCulloch symbolism for three outputs

B.Macukow 20

McCulloch symbolism four outputs

McCulloch symbolism four outputs

Venn diagram and McCulloch symbols for four outputs. Unknown are marked by A, B, C and D.

B.Macukow 21

Logical functions of two unknown

Logical functions of two unknown

Logical functions of two unknownLogical functions of two unknown

Function Formula Description Diagram 00 01 10 11

Const 1 1 (AB)(A~B)(~AB)(~A~B)

1 1 1 1

NAND ~(AB) (A~B)(~AB)(~A~B)

1 1 1 0

Implication AB (AB)(~AB)(~A~B)

1

1

0

1

Negation A ~A (~AB)(~A~B) 1 1 0 0


Implication BA (AB)(A~B)(~A~B)

1 0 1 1

Negation B ~B (A~B)(~A~B) 1 0 1 0

equivalence AB (AB) (~A~B) 1

0

0

1

NOR ~(AB) (~A~B) 1 0 0 0



disjunction AB (AB)(A~B)

(~AB)

0 1 1 1

non-equivalence

~(AB) (A~B)(~AB) 0 1 1 0

B B (AB)(~AB) 0

1

0

1

negation of implication ~AB (~AB) 0 1 0 0



A A (AB)(A~B) 0 0 1 1

negation of implication

A ~ B (A~B) 0 0 1 0

conjunction A B (AB) 0

0

0

1

constant 0 0 0 0 0 0


B.Macukow 26

Neural NetworksNeural Networks

B.Macukow 27


At the beginning was the idea that it is enough to build the net of many randomly connected elements to get the model of the brain operation.

Question: how many element is necessary for the process of self organization ??

B.Macukow 28

Research of McCulloch, Lettvin, Maturana, Hartlin and Ratliff.

Research on the frog’s eye and specially on the compound eye of the horseshoe cram - Limulus.

Hubel and Wiesel research on the mammals visual system.

Some parts are constructed in the very special, regular way.


B.Macukow 29

Two – layers chain structure


B.Macukow 30

The input layer of photoreceptors and the layer of processing elements which will locate the possible changes in the excitation distribution.

Connection rule:

Each receptor cell is to excite one element (exactly below). In addition to the excitatory connections there are also inhibitory connections (for the simplicity - to the adjacent cells only) which reduce the signal to the neighbors.


B.Macukow 31

The inhibition range can differs.

This is known as the of lateral inhibition


B.Macukow 32

As can be easily seen the uniform excitation of the first layer will not excite the second layer. The excitatory and inhibitory signals will be balanced.

A step signal is a step change in the spatial distribution. The distribution of output signal is not a copy of the input signal distribution but is the convolution of the input signal and the weighting function.


B.Macukow 33

The point in which the step change occurs is exaggerated at each side by increasing and decreasing the signal resulting in the single signal at the point of the this step.


B.Macukow 34

Input Signals

Output Signals

1

Elements’ transfer function


B.Macukow 35

As you can see such a network gives the possibility to locate the point where the changes in the excitation were enough high (terminations, inflections, bends etc.).

From the neurophysiology we know on the existence of the opposite operation lateral excitation.

These nets allows to detect the points of crossing or branchings etc.


B.Macukow 36

The lateral inhibition rule can be realized be the one dimensional net with negative feedback

Attention: elements are nonlinear and the feedback loops make analysis difficult; such the networks can be non stable and the distribution of the input signals does not depends univocally from the input signals.


Another simple neural netsAnother simple neural nets

B.Macukow 1 Neural Networks Lecture 4. B.Macukow 2 McCulloch symbolism The symbolism introduced by...

Documents

Transcript of B.Macukow 1 Neural Networks Lecture 4. B.Macukow 2 McCulloch symbolism The symbolism introduced by...