Intro to Neural Networks
description
Transcript of Intro to Neural Networks
![Page 1: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/1.jpg)
Intro to Neural Networks
Supervised Learning: Perceptrons and Backpropagation
![Page 2: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/2.jpg)
Neural Network ==
Connectionist /ism==Parallel Distributed Processing (PDP)
![Page 3: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/3.jpg)
Neural Networks assume
Intelligence is emergent
![Page 4: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/4.jpg)
1943 - McCullough Pitts Artificial Neuron
![Page 5: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/5.jpg)
1943 - McCullough Pitts Artificial Neuron
![Page 6: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/6.jpg)
Perceptron Learning 1958
![Page 7: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/7.jpg)
Perceptron Learning 1958
![Page 8: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/8.jpg)
Perceptron Learning 1958
![Page 9: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/9.jpg)
Perceptron Learning 1958
![Page 10: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/10.jpg)
Perceptron Learning 1958
![Page 11: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/11.jpg)
Linear Seperability Problem 1965
![Page 12: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/12.jpg)
Linear Seperability Problem 1965
![Page 13: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/13.jpg)
Backpropagation
Used to train multilayer feedforward networks
![Page 14: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/14.jpg)
Backpropagation
![Page 15: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/15.jpg)
Backpropagation
Used to train multilayer feedforward networks
Assumes a continuous activation function
![Page 16: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/16.jpg)
Backpropagation - Activation
![Page 17: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/17.jpg)
Backpropagation
Used to train multilayer feedforward networks
Assumes a continuous activation function
Delta rule
![Page 18: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/18.jpg)
Backpropagation Delta rulePerceptron update rule was:
Backprop update rule is:
€
Δw = c(desired − sign(actual))x
€
Δw = c(error)x
![Page 19: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/19.jpg)
Backpropagation Delta ruleError of an output node:
€
error j = (1−output j2)(desired j − actual j )
![Page 20: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/20.jpg)
Backpropagation Delta ruleError of a hidden node:
€
errori = (1−outputi2)( error j *wij
j∑ )
![Page 21: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/21.jpg)
Backpropagation Delta rule
![Page 22: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/22.jpg)
Backpropagation Delta rule
![Page 23: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/23.jpg)
Backpropagation Delta rule
![Page 24: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/24.jpg)
Backpropagation Delta rule
![Page 25: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/25.jpg)
Backpropagation
demo
![Page 26: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/26.jpg)
Inductive Bias
![Page 27: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/27.jpg)
Inductive Bias
Encoding / Feature Extraction# neurons used# layers usedOutput mapping
![Page 28: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/28.jpg)
Domains
Classification
![Page 29: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/29.jpg)
Domains
ClassificationPattern Recognition
![Page 30: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/30.jpg)
Domains
ClassificationPattern RecognitionContent Addressable Memory
![Page 31: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/31.jpg)
Domains
ClassificationPattern RecognitionContent Addressable MemoryPrediction
![Page 32: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/32.jpg)
Domains
ClassificationPattern RecognitionContent Addressable MemoryPredictionOptimization
![Page 33: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/33.jpg)
Domains
ClassificationPattern RecognitionContent Addressable MemoryPredictionOptimizationFiltering
![Page 34: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/34.jpg)
The good
Degrade gracefully
![Page 35: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/35.jpg)
The good
Degrade gracefullySolve ill-defined problems
![Page 36: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/36.jpg)
The good
Degrade gracefullySolve ill-defined problemsFlexible
![Page 37: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/37.jpg)
The good
Degrade gracefullySolve ill-defined problemsFlexibleGeneralization
![Page 38: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/38.jpg)
The bad
Time & Memory
![Page 39: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/39.jpg)
The bad
Time & MemoryBlack box
![Page 40: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/40.jpg)
The bad
Time & MemoryBlack boxTrial and Error
![Page 41: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/41.jpg)
When not to use Feedforward net If you can draw a flow chart or
formula
![Page 42: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/42.jpg)
When not to use Feedforward net If you can draw a flow chart or
formula If a piece of hardware or software
already exists that does what you want
![Page 43: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/43.jpg)
When not to use Feedforward net If you can draw a flow chart or
formula If a piece of hardware or software
already exists that does what you want
If you want to functionality to evolve
![Page 44: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/44.jpg)
When not to use Feedforward net If you can draw a flow chart or
formula If a piece of hardware or software
already exists that does what you want
If you want to functionality to evolvePrecise answers are required
![Page 45: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/45.jpg)
When not to use Feedforward net If you can draw a flow chart or
formula If a piece of hardware or software
already exists that does what you want
If you want to functionality to evolvePrecise answers are requiredThe problem could be described in a
lookup table
![Page 46: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/46.jpg)
When to use feedforward netYou can define a correct answer
![Page 47: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/47.jpg)
When to use feedforward netYou can define a correct answerYou have a lot of training data with
examples of right and wrong answers
![Page 48: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/48.jpg)
When to use feedforward netYou can define a correct answerYou have a lot of training data with
examples of right and wrong answers
You have lots of data but can’t figure how to map it to output
![Page 49: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/49.jpg)
When to use feedforward netYou can define a correct answerYou have a lot of training data with
examples of right and wrong answers
You have lots of data but can’t figure how to map it to output
The problem is complex but solvable
![Page 50: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/50.jpg)
When to use feedforward netYou can define a correct answerYou have a lot of training data with
examples of right and wrong answers
You have lots of data but can’t figure how to map it to output
The problem is complex but solvableThe solution is fuzzy or might change
slightly
![Page 52: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/52.jpg)
Examples
Jonathan McCabe’sNervous States 2006Each pixel is the Output state of aNeural network givenDifferent inputs
![Page 53: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/53.jpg)
Examples
2007 Phillip StearnsAANN: Artificial Analog Neural Network
![Page 54: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/54.jpg)
Examples
![Page 55: Intro to Neural Networks](https://reader035.fdocuments.us/reader035/viewer/2022062302/56816384550346895dd4678a/html5/thumbnails/55.jpg)
Examples
Ted?