2012-1155. Intro NN Perceptrons
-
Upload
praveen-nayak-bhukya -
Category
Documents
-
view
225 -
download
0
description
Transcript of 2012-1155. Intro NN Perceptrons
![Page 1: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/1.jpg)
Some more Some more Artificial Artificial
IntelligenceIntelligence• Neural Networks please read chapter 19.please read chapter 19.• Genetic Algorithms• Genetic Programming• Behavior-Based Systems
![Page 2: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/2.jpg)
Background
- Neural Networks can be :
- Biological Biological models
- ArtificialArtificial models
- Desire to produce artificial systems capable of sophisticated computations similar to the human brain.
![Page 3: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/3.jpg)
Biological analogy and some main Biological analogy and some main ideasideas
• The brain is composed of a mass of interconnected neurons– each neuron is connected to many other neurons
• Neurons transmit signals to each other
• Whether a signal is transmitted is an all-or-nothing event (the electrical potential in the cell body of the neuron is thresholded)
• Whether a signal is sent, depends on the strength of the bond (synapse) between two neurons
![Page 4: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/4.jpg)
How Does the Brain Work ? (1)
NEURON
- The cell that performs information processing in the brain.
- Fundamental functional unit of all nervous system tissue.
![Page 5: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/5.jpg)
Each consists of :SOMA, DENDRITES, AXON, and SYNAPSE.
How Does the Brain Work ? (2)
![Page 6: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/6.jpg)
Brain vs. Digital Computers (1)
- Computers require hundreds of cycles to simulate a firing of a neuron.
- The brain can fire all the neurons in a single step. ParallelismParallelism
- Serial computers require billions of cycles to perform some tasks but the brain takes less than a second.
e.g. Face Recognition
![Page 7: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/7.jpg)
Comparison of Brain and computer
Human Computer
ProcessingElements
100 Billionneurons
10 Milliongates
Interconnects 1000 perneuron
A few
Cycles per sec 1000 500 Million
2Ximprovement
200,000Years
2 Years
![Page 8: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/8.jpg)
Brain vs. Digital Computers (2)Brain vs. Digital Computers (2)
Future : combine parallelism of the brain with the switching speed of the computer.
![Page 9: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/9.jpg)
HistoryHistory
• 1943:1943: McCulloch & Pitts show that neurons can be combined to construct a Turing machine (using ANDs, Ors, & NOTs)
• 1958:1958: Rosenblatt shows that perceptrons will converge if what they are trying to learn can be represented
• 1969:1969: Minsky & Papert showed the limitations of perceptrons, killing research for a decade
• 1985:1985: backpropagation backpropagation algorithm revitalizes the field
![Page 10: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/10.jpg)
Definition of Neural Network
A Neural Network is a system composed of
many simple processing elements operating in
parallel which can acquire, store, and utilize
experiential knowledge.
![Page 11: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/11.jpg)
What is What is Artificial Artificial Neural Neural
Network?Network?
![Page 12: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/12.jpg)
Neurons vs. Units (1)
- Each element of NN is a node called unit.
- Units are connected by links.
- Each link has a numeric weight.
![Page 13: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/13.jpg)
Neurons vs units (2)Real neuron is far away from our simplified model - unit
Chemistry, biochemistry, quantumness.
![Page 14: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/14.jpg)
Computing Elements
A typical unit:
![Page 15: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/15.jpg)
Planning in building a Neural Network
Decisions must be taken on the following:
- The number of units to use.
- The type of units required.
- Connection between the units.
![Page 16: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/16.jpg)
How NN learns a task. How NN learns a task.
Issues to be discussedIssues to be discussed
- Initializing the weights.
- Use of a learning algorithm.
- Set of training examples.
- Encode the examples as inputs.
- Convert output into meaningful results.
![Page 17: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/17.jpg)
Neural Network ExampleNeural Network Example
Figure 19.7. A very simple, two-layer, feed-forward network with two inputs, two hidden nodes, and one output node.
![Page 18: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/18.jpg)
Simple Computations in this network
- There are 2 types of components: Linear and Non-linear.
- Linear: Input function- calculate weighted sum of all inputs.
- Non-linear: Activation function- transform sum into activation level.
![Page 19: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/19.jpg)
Calculations
Input function:
Activation function gg:
![Page 20: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/20.jpg)
A Computing Unit. A Computing Unit.
Now in more detail but for a Now in more detail but for a particular model onlyparticular model only
Figure 19.4. A unit
![Page 21: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/21.jpg)
ActivationActivation Functions
- Use different functions to obtain different models.
- 3 most common choices :
1) Step function2) Sign function3) Sigmoid function
- An output of 1 represents firing of a neuron down the axon.
![Page 22: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/22.jpg)
Step Function Perceptrons
![Page 23: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/23.jpg)
3 Activation Functions
![Page 24: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/24.jpg)
Are current computer a wrong model of thinking?
• Humans can’t be doing the sequential analysis we are studying
– Neurons are a million times slower than gates
– Humans don’t need to be rebooted or debugged when one bit dies.
![Page 25: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/25.jpg)
100-step program constraint• Neurons operate on the order of 10-3 seconds
• Humans can process information in a fraction of a second (face recognition)
• Hence, at most a couple of hundred serial operations are possible
• That is, even in parallel, no “chain of reasoning” can involve more than 100 -1000 steps
![Page 26: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/26.jpg)
Standard structure of an artificial neural network
• Input units– represents the input as a fixed-length vector of numbers
(user defined)
• Hidden units– calculate thresholded weighted sums of the inputs– represent intermediate calculations that the network
learns
• Output units– represent the output as a fixed length vector of numbers
![Page 27: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/27.jpg)
Representations• Logic rules
– If color = red ^ shape = square then +
• Decision trees– tree
• Nearest neighbor– training examples
• Probabilities– table of probabilities
• Neural networks– inputs in [0, 1]
Can be used for all of them
Many variants exist
![Page 28: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/28.jpg)
NotationsNotations
![Page 29: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/29.jpg)
Notation
![Page 30: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/30.jpg)
Notation (cont.)
![Page 31: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/31.jpg)
Operation of individual units
• Outputi = f(Wi,j * Inputj + Wi,k * Inputk + Wi,l * Inputl)
– where f(x) is a threshold (activation) function– f(x) = 1 / (1 + e-Output)
• “sigmoid”
– f(x) = step function
![Page 32: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/32.jpg)
Artificial Neural Artificial Neural NetworksNetworks
![Page 33: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/33.jpg)
Units in Action
- Individual units representing Boolean functions
![Page 34: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/34.jpg)
Network StructuresNetwork Structures
Feed-forward neural netsFeed-forward neural nets::
Links can only go in one direction.
Recurrent neural nets:Recurrent neural nets:
Links can go anywhere and form arbitrarytopologies.
![Page 35: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/35.jpg)
Feed-forward NetworksFeed-forward Networks
- Arranged in layerslayers.
- Each unit is linked only in the unit in next layer.
- No units are linked between the same layer, back to the previous layer or skipping a layer.
- Computations can proceed uniformly from input to output units.
- No internal state exists.
![Page 36: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/36.jpg)
Feed-Forward Example
I1
I2
t = -0.5W24= -1
H4
W46 = 1t = 1.5
H6
W67 = 1
t = 0.5
I1
t = -0.5W13 = -1
H3
W35 = 1t = 1.5
H5
O7
W57 = 1
W25 = 1
W16 = 1
Inputs skip the layer in this case
![Page 37: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/37.jpg)
Multi-layer Networks and PerceptronsPerceptrons
- Have one or more layers of hidden units.
- With two possibly very large hidden layers, it is possible possible to implement any to implement any functionfunction.
- Networks without hidden layer are called perceptrons.
- Perceptrons are very limited in what they can represent, but this makes their learning problem much simpler.
![Page 38: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/38.jpg)
Recurrent Network (1)Recurrent Network (1)
- The brain is not and cannot be a feed-forward network.
- Allows activation to be fed back to the previous unit.
- Internal state is stored in its activation level.
- Can become unstable
-Can oscillate.
![Page 39: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/39.jpg)
Recurrent Network (2)
- May take long time to compute a stable output.
- Learning process is much more difficult.
- Can implement more complex designs.
- Can model certain systems with internal states.
![Page 40: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/40.jpg)
PerceptronsPerceptrons
![Page 41: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/41.jpg)
PerceptronsPerceptrons
- First studied in the late 1950s.
- Also known as Layered Feed-Forward Networks.
- The only efficient learning element at that time was for single-layered networks.
- Today, used as a synonym for a single-layer, feed-forward network.
![Page 42: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/42.jpg)
Fig. 19.8. PerceptronsFig. 19.8. Perceptrons
![Page 43: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/43.jpg)
Perceptrons Perceptrons
![Page 44: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/44.jpg)
Sigmoid Perceptron
![Page 45: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/45.jpg)
Perceptron Perceptron learning rulelearning rule
• Teacher specifies the desired outputdesired output for a given input
• Network calculates what it thinks the output should be
• Network changes its weights in proportion to the error between the desired & calculated results
wi,j = * [teacheri - outputi] * inputj– where: is the learning rate;– teacheri - outputi is the error term; – and inputj is the input activation
• wi,j = wi,j + wi,j
Delta rule
![Page 46: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/46.jpg)
Adjusting perceptron weights
wi,j = * [teacheri - outputi] * inputj
• missi is (teacheri - outputi)
• Adjust each wi,j based on inputj and missi
• The above table shows adaptation.
• Incremental learning.
miss<0 miss=0 miss>0
input < 0 alpha 0 -alphainput = 0 0 0 0input > 0 -alpha 0 alpha
![Page 47: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/47.jpg)
Node biases
• A node’s output is a weighted function of its inputs
• What is a bias?What is a bias?
• How can we learn the bias value?
• Answer: treat them like just another weight
![Page 48: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/48.jpg)
Training biases ()• A node’s output:
– 1 if w1x1 + w2x2 + … + wnxn >=
– 0 otherwise
• Rewrite– w1x1 + w2x2 + … + wnxn - >= 0
– w1x1 + w2x2 + … + wnxn + (-1) >= 0
• Hence, the bias is just another weight whose activation is always -1
• Just add one more input unit to the network topology
bias
![Page 49: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/49.jpg)
Perceptron convergence theorem
• If a set of <input, output> pairs are learnable (representable), the delta rule will find the necessary weights– in a finite number of steps– independent of initial weights
• However, a single layer perceptron can only learn linearly separable concepts– it works iff iff gradient descent works
![Page 50: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/50.jpg)
Linear separability
• Consider a perceptron
• Its output is– 1, if W1X1 + W2X2 >
– 0, otherwise
• In terms of feature space– hence, it can only classify examples if a line (hyperplane
more generally) can separate the positive examples from the negative examples
![Page 51: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/51.jpg)
What can Perceptrons Represent ?
- Some complex Boolean function can be represented.
For example: Majority function - will be covered in this lecture.
- Perceptrons are limited in the Boolean functions they can represent.
![Page 52: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/52.jpg)
Figure 19.9. Linear Separability in Perceptrons
The Separability Problem and EXOR The Separability Problem and EXOR troubletrouble
![Page 53: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/53.jpg)
AND and OR linear Separators
![Page 54: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/54.jpg)
Separation in n-1 dimensions
Example of 3Dimensional space
majority
![Page 55: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/55.jpg)
Perceptrons & XORPerceptrons & XOR• XOR function
– no way to draw a line to separate the positive from negative examples
Input1 Input2 Output
0 0 00 1 11 0 11 1 0
![Page 56: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/56.jpg)
How do we compute XOR?
![Page 57: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/57.jpg)
Learning Linearly SeparableLearning Linearly Separable Functions (1)
What can these functions learn ?
Bad news:- There are not many linearly separable functions.
Good news:- There is a perceptron algorithm that will learn any linearly separable function, given enough training examples.
![Page 58: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/58.jpg)
Most neural network learning algorithms, including the
perceptrons learning method, follow the current-best-
hypothesis (CBH) scheme.
Learning Linearly Separable Functions (2)
![Page 59: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/59.jpg)
- Initial network has a randomly assigned weights.
- Learning is done by making small adjustments in the weights to reduce the difference between the observed and predicted values.
- Main difference from the logical algorithms is the need to repeat the update phase several times in order to achieve convergence.
-Updating process is divided into epochs.
-Each epoch updates all the weights of the process.
Learning Linearly Separable Functions (3)
![Page 60: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/60.jpg)
Figure 19.11. The GenericGeneric Neural Network Learning Method: adjust the weights until predicted
output values O and true values T agreee are examples from set examples
![Page 61: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/61.jpg)
Examples of Feed-Forward Learning
Two types of networks were compared for the restaurant problem
![Page 62: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/62.jpg)
Multi-Layer Multi-Layer Neural NetsNeural Nets
![Page 63: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/63.jpg)
Feed Forward Feed Forward NetworksNetworks
![Page 64: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/64.jpg)
2-layer Feed Forward example
![Page 65: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/65.jpg)
Need for hidden unitshidden units
• If there is one layer of enough hidden units, the input can be recoded (perhaps just memorized; example)
• This recoding allows any mapping to be represented
• Problem: how can the weights of the hidden units be trained?
![Page 66: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/66.jpg)
XOR Solution
![Page 67: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/67.jpg)
Majority of 11 Inputs(any 6 or more)
Perceptron is better than DT on majority
Constructive induction is even better than NN
How many times in battlefield the robot recognizes majority?
![Page 68: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/68.jpg)
Other Examples• Need more than a 1-layer network for:
– Parity– Error Correction– Connected Paths
• Neural nets do well with– continuous inputs and outputs
• But poorly with– logical combinations of boolean inputs
Give DT brain to a mathematician robot and a NN brain to a soldier robot
![Page 69: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/69.jpg)
WillWait Restaurant example
Let us not dramatize “universal” benchmarks too much
Here decision tree is better than perceptron
![Page 70: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/70.jpg)
N-layer FeedForward N-layer FeedForward NetworkNetwork
• Layer 0 is input nodes
• Layers 1 to N-1 are hidden nodes
• Layer N is output nodes
• All nodesAll nodes at any layer k are connected to all nodes at layer k+1
• There are no cycles
![Page 71: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/71.jpg)
2 Layer FF net with LTUs
• 1 output layer + 1 hidden layer– Therefore, 2 stages to “assign reward”
• Can compute functions with convex regions
• Each hidden node acts like a perceptron, learning a separating line
• Output units can compute intersections of half-planes given by hidden units
Linear Threshold Units
![Page 72: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/72.jpg)
Feed-forward NN with Feed-forward NN with hidden hidden layerlayer
![Page 73: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/73.jpg)
• Braitenberg Vehicles• Quantum Neural BV
Reactive architecture based Reactive architecture based on NN for a simple roboton NN for a simple robot
![Page 74: 2012-1155. Intro NN Perceptrons](https://reader036.fdocuments.us/reader036/viewer/2022081511/5695d2361a28ab9b02998647/html5/thumbnails/74.jpg)
Evaluation Evaluation of a Feedforward NNFeedforward NN using software is easy
Calculate output neurons
Calculate activation of hidden neurons
Set bias input neuron
Take from hidden neurons and multiply by weights