Ann Assignment 2ashish

Assignment 2(Machine Intelligence)

Ashish KushwahaResearch Scholar,

Deptt. Of EE, SOE,Shiv Nadar University

1Ashish Kushwaha, Ph.D Research Scholar (EE), Deptt. Of EE, SOE, Shiv Nadar University

Answer 1.1 The function is

φ ( v )= 1

1+e−av

The derivative is

dφdv

=ddv ( 1

1+e−av )dφdv

=a (1+e−av)−2 e−av

dφdv

=a ( 11+e−av )( e−av

1+e−av )Or we can write

dφdv

=a φ(v)(1−φ (v ))

Since we need to find the value of this derivative at the origin, therefore by putting v=0 in the derivative, we get

dφdv v=0

=a(1+e−a0)−2 e−a0

Or

dφdv v=0

=a(1+1)−21

i.e.

dφdv v=0

=a4

Answer 1.2

φ ( v )=1−e−av

1+e−av =tanhav2

The derivative of the above function will be


d φdv

= ddv ( 1−e−av

1+e−av )dφdv

=(1−e−av ) a ( 1+e

−av )−2e−av+( 1

1+e−av )ae−av

The above equation can be written as

dφdv

=( ae−av

1+e−av )( 1−e−av

1+e−av +1)As 1−φ (v )= 2e−av

1+e−av

So,

Derivative can be written as

dφdv

=a2 (1−1−e−av

1+e−av )( 1−e−av

1+e−av +1)dφdv

=a2

( 1−φ ( v ) ) ( φ (v )+1 )

dφdv

=a2

(1−φ2(v ))

On putting the v=0, the value of derivative at origin is

dφdv

=a2 (1−1−e−a0

1+e−a0 )( 1−e−a0

1+e−a0 +1)dφdv

=a2

Now if the slope parameter ‘a’ is made infinitely large, the odd sigmoid function will become the hard limiter threshold function.

Answer 1.3 - Algebraic odd sigmoid function is given by

φ ( v )= v

√1+v2


The derivative of the given function is given by

dφdv

=ddv ( v

√1+v2 )dφdv

=( 1

√1+v2 )−v12(1+v2)

−32 2v

dφdv

=( 1

√1+v2 )(1− v2

1+v2 )dφdv

=( 1

√1+v2 )( 1

1+v2 )As

φ3 ( v )= v3

(1+v2)√1+v2

So

dφdv

=( φ3 (v )v3 )

On putting the v=0, the value of derivative at origin is

dφdv

=( 1

√1+02 )( 1

1+02 )dφdv

=1

Answer- 1.4(i)

φ ( v )= 1√2 π

∫−∞

v

e−x2

2 dx

So, we know that

∫−∞

v

e−c x2

dx=√ π4 c

erf ¿


Φ(v) includes error function and error function is the part of sigmoid function. So, it fits into the requirements of a sigmoid function.

(ii)

φ ( v )= 2π

tan−1(v )

It is the arc tangent function. Sigmoid function also contains arc tangent function. So we can say that is a sigmoid function.

Difference between two functions: Both the functions are sigmoid but they have different varieties means first one is the error function and second one is the arc tangent function.

Answer 1.5

Function in Q. 1.4

φ ( v )= 1√2 π

∫−∞

v

e−x2

2 dx

Is cumulative (probability) distribution function (CDF) because,

∫−∞

v

e−c x2

dx=12(1+erf ( x

√2 ))CDF describe probability of a random variable falling in interval (−∞,v).

Answer 1.6 (a)


The function is linear between -0.5a to 0.5a.So, the function is given by

φ (v )={ 0 , for v≤−0.5aba

v+0.5b , for−0.5a<v<0.5 a

b , for v≥ 0.5a

(b) If a is allowed to be zero, the slope will become infinite and the function will become unipolar hard limiter threshold function.

Answer -1.7

The function is given by

φ (v )={ −b , for v ≤−aba

v , for−a<v<a

b , for v≥ 0.5a

if a is allowed to be zero, the slope will become infinite and the function will become bipolar hard limiter threshold function.

Answer 1.8. If we consider the first function i.e.

φ ( v )= 1

1+e−av

We can write it as

φ ( v )= 1

1+e−a(w1 x1+w2 x2+w3 x3+…+wN xN )


Similarily the function given in this problem could be written as

φ ( v )= 1

1+e−(w1 x1+w2 x2+w3 x3+…+wN xN )

Equating these above two equations i.e.

φ ( v )= 1

1+e−a (w1 x1+w2 x2+w3 x3+…+w N xN )

φ ( v )= 1

1+e−(w1 x1+w2 x2+w3 x3+…+wN xN )

Taking log on both sides i.e.

φ ( v )=loge( 1

1+e−a ( w1 x1+w2 x2+w3 x3+…+wN xN ) )

φ ( v )=loge(1

1+e−( w1 x1' +w2 x2 '+w3 x3 '+…+wN xN ' )

)

And now solving and equating we get

x1' =ax1 , x2

' =ax2 , x3' =ax3 …….x N

' =axN

So we conclude that the the weights should be multiplied by ‘a’ to get the same inputs as before.

Answer-1.9

In this case x1=10, x2=-20, x3=4 amd x4=-2

And w1=0.8,w2=0.2,w3=-1.0 w4=-0.9

So in this case

Output of neuron will ber given by w1 x1+w2 x2+w3 x3+…+wN x N

Therefore we have 10x0.8+( -20)x 0.2+4 x-1+-2X-0.9=1.8


(a) In case of a linear neuron we will have the output to be the same as the sum of products as done above i.e. 1.8

(b) In case of a McCulloch Pitts model, since the output of a neuron is greater than 0 i.e. 1.8 therefore it is a positive value means greater than T so the final output will be 1

Answer 1.10.

(a)In this case the logistic function given is that of unipolar sigmoid. The value of v is 1.8 therefore by using

φ ( v )= 1

1+e−v

We have

φ ( v )= 1

1+e−(1.8)=0.605

(b)For a McCulloch Pitts model

φ ( v )= 1

1+e−1.8

Here φ ( v )>0

Here the output is greater than threshold value so the output will be one.

Answer 1.11 (a) and (b)

For the McCulloch–Pitts model of a single neuron we can think about the calculation as proceeding in two parts, input processing then the calculation of the output.

The inputs to the neuron body are shown as x1, x2, … xN . When the input (from the previous neurons) reaches this neuron, then their values are multiplied by the synaptic strength. These strengths are dependent on how much learning has occurred. These strengths are called weights and are represented by w1 etc. So a single input is calculated as w1 x1 and the total weighted input to the neuron is


Sigmoid Functions. Left has k = 10, right has k = 5. Large k means steep slope.

w1 x1+w2 x2+w3 x3+…+wN x N

This total input is then passed on to the output-calculation part of the neuron. Here, if the total input is greater than some thresholdθ, then the neuron will ‘fire’ and produce an output. Note, that the output of the neuron is effectively binary; fire or no-fire i.e 0 or 1.

It is possible to relax the binary nature of the output, to obtain real numbers (such as decimals). This is done using the ‘sigmoid’ function. The parameter k adjusts the slope of the transition between fire and no-fire states. The mathematical description of the sigmoid function is

f ( a )= 1

1+e−ka

It is clear that choosing a large value of k approximates the binary McCulloch-Pitts neuron more closely.


Ann Assignment 2ashish

Documents

Transcript of Ann Assignment 2ashish