7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
1/16
Scholastic Video Book Series
Artificial Neural Networks
Part 3
(Back Propagation)
(with English Narrations)
http://scholastictutors.webs.com
(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdf)
1
Scholastic Tutors (Oct, 2013)ISVT 911-0-20-131022-1
ARTIFICIAL NEURAL
NETWORKS
http://scholastictutors.webs.com/http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdfhttp://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
2/16
International Baccalaureate (IB)
2
Artificial Neural Networks - #3Back-Propagation
http://scholastictutors.webs.com
(ANN-003)
http://youtu.be/I2I5ztVfUSE
Click here to see the video
http://scholastictutors.webs.com/http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSEhttp://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
3/16
Step 1:Calculate the local gradients (do1, do2, dh1 and dh2) for the nodes
in the network.
n1
n2
n3
n4
w31
y4
w41
b1
x1
x2
y3
do1b3
do2
dh1
dh2
1
( )1 exp
v
v
w32
w42
b2
w11
w21
w12
w22
1 = 1 3 1 31 2 32 (3 3)
2 = 1 4 1 41 2 42 (4 4)
1 = (1 1 1 11 2 12) 1 31 2 41
y1
y2
2 = (1 2 1 21 2 22) 1 32 2 42
= () 1 () +1
+1
+1
http://scholastictutors.webs.com
Back-Propagation Algorithm
b4+1
Desired = d3
Desired = d4
http://scholastictutors.webs.com/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
4/16
http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
5/16
5
Lets perform a complete forward and backward sweep of the feedforward
network (2-2-1 architecture) shown below using the back propagation
algorithm discussed.Assume,
target output = 0.9,
= 0.25, and =0.0001
b1=0.1
Output = y3
w21= -0.1
Neuron 1
Neuron 2
Neuron 3
x1=0.1
x2=0.9
+1
b2= 0.1
+1
+1
w11= -0.2
w22= 0.3
w12= 0.1
w32= 0.3
w31= 0.2
b3= 0.2
y1
y2
v1
v2
v3
Also assume that the
activation function in all 3
neurons are sigmoid functions
given by:
1
( )1 exp
v
v
Back-Propagation - Example
dh1
dh2
do1
http://scholastictutors.webs.com
http://scholastictutors.webs.com/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
6/16
1 = 1 1 1 11 2 12 = 1 0.1 0.1 0.2 0.9 0.1 = 0.17
1 = 1 = 0.17 =
+exp(.)= 0.542
2 = 1 2 1 21 2 22 = 1 0.1 0.1 0.1 0.9 0.3 = 0.36
2 = 2 = 0.36 =
+exp(.)= 0.589
3 = 1 3 1 31 2 32 = 1 0.2 0.542 0.2 0.589 0.3 = 0.485
3 = 3 = 0.485 =
+exp(.)= 0.619
Therefore: = 3 3 = 0.9 0.619 = 0.281
Forward Pass
http://scholastictutors.webs.com
http://scholastictutors.webs.com/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
7/16
1 = 3 3 3 = 0.4851 0.281 = 0.4851 1 0.4851 0.281 = 0.619 1 0.619 0.281 = 0.0663
1 = 1 1 31 = 0.17 0.0663 0.2 = 0.17 1 0.17 0.01362 = 0.542 1 0.542 0.01362=0.0033
2 = 2 1 32 = 0.36 0.0663 0.3 = 0.36 1 0.36 0.01989 = 0.589 1 0.589 0.01989=0.0049
Backward Pass
1 = 0.17, 1 = 0.542
2 = 0.36, 2 = 0.589
3 = 0.485, 3 = 0.619
= 0.281
http://scholastictutors.webs.com
http://scholastictutors.webs.com/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
8/16
Backward Pass
1 = 0.542, 2 = 0.589
Step 2:Adjust the weights of the network using
the learning rule:
=
31 1 = 31 31 1 1 1
31 1 = 0.2 0.0001 0.2 0.25 0.0663 0.542 = 0.2090
32 1 = 32 32 1 1 2
32 1 = 0.3 0.0001 0.3 0.25 0.0663 0.589 = 0.3098
1 = 0.0663,
1 = 0.0033
2 = 0.0049
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
9/16
Backward Pass
1 = 0.542, 2 = 0.589
Step 2:Adjust the weights of the network using
the learning rule:
=
11 1 = 11 11 1 1 1
11 1 = 0.2 0.0001 0.2 0.25 0.0033 0.1 = 0.1999
21 1 = 21 21 1 2 1
21 1 = (0.1) 0.0001 0.1 0.25 0.0049 0.1 = 0.0999
1 = 0.0663,
1 = 0.0033
2 = 0.0049
http://scholastictutors.webs.com
http://scholastictutors.webs.com/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
10/16
Backward Pass
1 = 0.542, 2 = 0.589
Step 2:Adjust the weights of the network using
the learning rule:
=
12 1 = 12 12 1 1 2
12 1 = 0.1 0.0001 0.1 0.25 0.0033 0.9 = 0.1008
22 1 = 22 22 1 2 2
22 1 = 0.3 0.0001 0.3 0.25 0.0049 0.9 = 0.3011
1 = 0.0663,
1 = 0.0033
2 = 0.0049
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
11/16
Backward Pass
1 = 0.542, 2 = 0.589
Step 2:Adjust the weights of the network using the
learning rule:
=
3 1 = 3 3 1 1 1
3 1 = 0.2 0.0001 0.2 0.25 0.0663 1 = 0.2166
1 1 = 1 1 1 1 1
1 1 = 0.1 0.0001 0.2 0.25 0.0033 1 = 0.1008
3 1 = 3 3 1 2 1
3 1 = 0.1 0.0001 0.1 0.25 0.0049 1 = 0.1012
1 = 0.0663,
1 = 0.0033
2 = 0.0049
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
12/16
12
b1=0.1008
Output = y3
w21= -0.0999
Neuron 1
Neuron 2
Neuron 3
x1=0.1
x2=0.9
+1
b2= 0.1012
+1
+1
w11= -0.1999
w22= 0.3011
w12= 0.1008
w32= 0.3098
w31= 0.2090
b3= 0.2166
y1
y2
v1
v2
v3
After one Complete Forward &
Backward Pass
dh1
dh2
do1
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
13/16
1 = 0.17 1 = 0.1715
1 = 0.542 1 = 0.5428
2 = 0.36 2 = 0.3622
2 = 0.589 2 = 0.5896
3 = 0.4851 3 = 0.5127
3 = 0.619 3 = 0.6254
= 3 3 = 0.9 0.619 = 0.281 = 3 3 = 0.9 0.6254 =0.2746
After one Complete Forward &
Backward Pass
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
14/16
After second pass = 0.2683 After third pass = 0.2623
After forth pass = 0.2565 After 100 passes = 0.0693 After 200 passes = 0.0319 After 500 passes = 0.0038
Error is getting reduced after each pass.
After a few more Complete
Forward & Backward Passes
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSE7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
15/16
International Baccalaureate (IB)
15
Artificial Neural Networks - #3Back Propagation
http://scholastictutors.webs.com
(ANN-003)
END of the Book
If you like to see similar solutions to any Mathematics problems please
contact us at: [email protected] your request.
http://youtu.be/I2I5ztVfUSEClick here to see the video
http://scholastictutors.webs.com/http://[email protected]/http://youtu.be/I2I5ztVfUSEhttp://youtu.be/I2I5ztVfUSEhttp://[email protected]/http://scholastictutors.webs.com/7/25/2019 Scholastic Book NefghuralNetworks Part03 2013-10-22
16/16
Videos at: http://www.youtube.com/user/homevideotutor
16
(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-10-22.pdf)
Scholastic Video Book Series
Artificial Neural Networks
Part 3
(Back Propagation)
(with English Narrations)
(END)
Scholastic Tutors (Sep, 2013)ISVT 911-0-20-131022-1
http://www.youtube.com/user/homevideotutorhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part03-2013-09-23.pdfhttp://www.youtube.com/user/homevideotutorhttp://www.youtube.com/user/homevideotutorTop Related