CS536: Machine Learning Artificial Neural Networks Neural Networks
Outline What Neural Networks are and why they are desirable Historical background Applications...
-
Upload
beatrice-cunningham -
Category
Documents
-
view
216 -
download
0
Transcript of Outline What Neural Networks are and why they are desirable Historical background Applications...
![Page 1: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/1.jpg)
![Page 2: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/2.jpg)
OutlineOutline What Neural Networks are and why they are
desirable Historical background Applications Strengths neural networks and advantages Status N.N and evolving N.N Basic Architectures and algorithms Applications Drawbacks and limitations
![Page 3: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/3.jpg)
What are Neural Networks?What are Neural Networks?
information processing paradigm inspired by biological nervous systems, such as our brain
Structure: large number of highly interconnected processing elements (neurons) working together
Like people, they learn from experience (by example)
![Page 4: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/4.jpg)
Historical backgroundHistorical background
Neural network simulations appear to be a recent development. However, this field was established before the advent of computers, and has survived at least one major setback and several eras.
These pioneers were able to develop convincing technology which surpassed the limitations identified by Minsky and Papert. Minsky and Papert, published a book (in 1969)
![Page 5: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/5.jpg)
Human Brain FunctionHuman Brain Function
![Page 6: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/6.jpg)
Neural Network FunctionNeural Network Function
![Page 7: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/7.jpg)
TerminologyTerminologyInput: Explanatory variables also referred
to as “predictors”.Neuron: Individual units in the hidden
layer(s) of a neural network.Output: Response variables also called
“predictions”.Hidden Layers: Layers between input and
output that an apply activation function.
![Page 8: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/8.jpg)
Some SimilaritiesSome Similarities
![Page 9: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/9.jpg)
Why Neural Networks are desirableWhy Neural Networks are desirable
Human brain can generalize from abstract
Recognize patterns in the presence of noise
Recall memoriesMake decisions for current problems
based on prior experience
![Page 10: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/10.jpg)
When to use neural networksWhen to use neural networks
Use for huge data sets (i.e. 50 predictors and 15,000 observations) with unknown distributions
Smaller data sets with outliers as neural networks are very resistant to outliers
![Page 11: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/11.jpg)
Why Neural Networks in Statistics?Why Neural Networks in Statistics?
The methodology is seen as a new paradigm for data analysis where models are not explicitly stated but rather implicitly defined by the network.
Allows for analysis where traditional methods might be extremely tedious or nearly impossible to interpret.
![Page 12: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/12.jpg)
Difference in Neural NetworksDifference in Neural Networks
The difference in the two approaches is that multiple linear regression has a closed form solution for the coefficients, while neural networks use an iterative process.
![Page 13: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/13.jpg)
ApplicationsApplications Data Conceptualization– infer grouping relationships
e.g. extract from a database the names of those most likely to buy a particular product.
Data Filtering– e.g. take the noise out of a telephone signal,
signal smoothing Planning– Unknown environments– Sensor data is noisy– Fairly new approach to planning
![Page 14: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/14.jpg)
ApplicationsApplications
Prediction: learning from past experience– pick the best stocks in the market– predict weather– identify people with cancer risk
Classification– Image processing– Predict bankruptcy for credit card
companies– Risk assessment
![Page 15: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/15.jpg)
Strengths of a Neural NetworkStrengths of a Neural Network
Power: Model complex functions, nonlinearity built into the network
Ease of use:–Learn by example–Very little user domain-specific
expertise neededIntuitively appealing: based on model
of biology, will it lead to genuinely intelligent computers/robots?
![Page 16: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/16.jpg)
General AdvantagesGeneral Advantages
Advantages– Adapt to unknown situations– Robustness: fault tolerance due to
network redundancy– Autonomous learning and generalization
Disadvantages– Not exact– Large complexity of the network structure
For motion planning?
![Page 17: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/17.jpg)
Status of Neural NetworksStatus of Neural Networks
Most of the reported applications are still in research stage
No formal proofs, but they seem to have useful applications that work
![Page 18: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/18.jpg)
Evolving networksEvolving networks
Continuous process of:– Evaluate output – Adapt weights– Take new inputs
ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem
![Page 19: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/19.jpg)
Where are NN used?Where are NN used?
Recognizing and matching complicated, vague, or incomplete patterns
Data is unreliable Problems with noisy data
–Prediction–Classification–Data association–Data conceptualization–Planning
![Page 20: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/20.jpg)
Back propagationBack propagation
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error sizeCalculate output layer error , then
propagate back to previous layerImproved performance, very common!
![Page 21: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/21.jpg)
Activation FunctionActivation Function
The only practical requirement for an activation function is that it be differentiable
Sigmoid function is commonly used g(netinput) = 1/(1+ exp-(netinput))Or a simple binary threshold unitӨ(netinput) = {1 ,if netinput ≥ 0 ; 0 ,
otherwise}
![Page 22: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/22.jpg)
Training the NetworkTraining the Network
Neural Networks must be first trained before being used to analyze new data
Process entails running patterns through the network until the network has “learned” the model to apply to future data
Can take a long time for noisy data
![Page 23: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/23.jpg)
New DataNew Data
Once the network is trained new data can be run through it
The network will classify new data based on the previous data it trained with
If an exact match can not be found it will match with the closest found in memory
![Page 24: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/24.jpg)
Regression and Neural NetworksRegression and Neural Networks
Objective of regression problem is to find coefficients that minimize sum of errors
To find coefficients we must have a dataset that includes the independent variable and associated values of the dependent variable. (very similar to training the network)
Equivalent to a single layer feed forward network
![Page 25: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/25.jpg)
Drawbacks and LimitationsDrawbacks and Limitations
Neural Networks can be extremely hard to use
The programs are filled with settings you must input and a small error will cause your predictions to have error also
The results can be very hard to interpret as well
![Page 26: Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.](https://reader036.fdocuments.us/reader036/viewer/2022070409/56649e9d5503460f94b9ec3b/html5/thumbnails/26.jpg)