Backpropagation algo

BACKPROPAGATION ALGORITHM

BY:Amit kumar

CONTENTS:

1.Introduction2. Example of Backpropagation4.Algorithm6. Advantages7.Disadvantages8. Application9. Conclusion

Blackcollar4/23/2015 2

INTRODUCTION

Backpropagation, an abbreviation for "backward propagation of errors" is a common method of training artificial neural networks.

The method calculates the gradient of a loss function with respects to all the weights in the network.

The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the loss function.


Backpropagation requires a known, desired output for each input value in order to calculate the loss function gradient.

The backpropagation learning algorithm can be divided into two phases:

Propagation

Weight update

In Propagation neural network using the training pattern target in order to generate the deltas of all

output and hidden neurons.

Multiply its output delta and input activation to get the gradient of the weight.


EXAMPLE OF BACKPROPAGATION

Inputs xi arrive through pre-connected path

Input is modeled using real weights wi

The response of the neuron is a nonlinear function f of its weighted inputs


Learning is the process of modifying the weights in order to produce a network that performs some function.


ALGORITHMThe following steps is the recursive definition of

algorithm:Step:-1.Randomly choose the initial weights.

2.For each training pattern apply the inputs to the network.

3.Calculate the output for every neuron from the inputlayer, through the hidden layer(s), to the output layer.

4.Calculate the error at the outputs:Use the output error to compute error signals for pre-output layers.

ErrorB= OutputB(1-OutputB) (TargetB-OutputB)Blackcollar4/23/2015 7

use the error signals to compute weight adjustments.

W+AB = WAB + (ErrorB x OutputA)

Apply the weight adjustments.

Where

W+AB is New weight and WAB is initial weight

Output(1-Output)- the Sigmoid Function .


Advantages

Backpropagation has many advantages:-

It is fast, simple and easy to program.

It has no parameters to tune (except for the number of input) .

This is a shift in mind set for the learning-system designer instead of trying to design a learning algorithm that is accurate over the entire space

It requires no prior knowledge about the weak learner and so can be flexible.


Disadvantages

Disadvantages are:-

The actual performance of Backpropagation on a particular problem is clearly dependent on the input data.

Backpropagation can be sensitive to noisy data and outliers.

Fully matrix-based approach to backpropagation over a mini-batch .


Application

Mapping character strings into phonemes so they can be pronounced by a computer.

Neural network trained how to pronounce each letter in a word in a sentence, given the three letters before and three letters after it in a window

In the field of Speech Recognition.

In the field of Character Recognition.

In the field of Face Recognition.


Conclusion

The backpropagation algorithm normallyconverges reasonably fast However, the actualspeed depends very much on the simulationparameters on the initial weight values.


Thank you !


Backpropagation algo

Engineering

Transcript of Backpropagation algo