Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.
-
Upload
darcy-palmer -
Category
Documents
-
view
218 -
download
1
Transcript of Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.
![Page 1: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/1.jpg)
Artificial Neural Network Theory and ApplicationAshish Venugopal
Sriram Gollapalli
Ulas Bardak
![Page 2: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/2.jpg)
Presentation Overview
Artificial Neural Networks NASA profiles as a Pattern
Classification Task Implementing Pattern Classification
on the Silicon Recognition Neuron Board
![Page 3: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/3.jpg)
ANN General Overview
Inspired by biological neuron modelsDecision are distributed throughout
systemComponents interacts at several stagesComponents work in unison to solve
one problem ANN well suited to problems that don’t
have closed form algorithmic solution Black box model – hard to interpret
![Page 4: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/4.jpg)
ANNs Applied
Driving a car. Task involves determining the direction the road is curving based on an imageImage pixels are inputCurve angle is outputLearned non-linear relationship
between pixels and roads• hard to understand the model
![Page 5: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/5.jpg)
Theory
Input layer, Hidden layer, Output LayerInput – one “neuron” for each pixelHidden – set of neurons to store what
was learned.Output layer – one neuron for each of 5
turn directions – ie straight, slight left, hard right..
But what is a neuron?
![Page 6: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/6.jpg)
Neurons
Math terms: Binary decision makers English terms: take in some input, decide
whether is “high” or “low”, and pass this value on.
Similar to neurons in the brain propagating signals.
![Page 7: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/7.jpg)
How is the model learned?
Once you have training examplesPair (feature set, classification)Run the back propagation algorithms
Learns connection weights between neurons, as well as within neuron transfer weights.
Network topology changes affects learning behavior!
![Page 8: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/8.jpg)
Unsupervised Networks
So far we have used labeled training examples to learn relationship between features and response variable.
What if we are just exploring patterns that exist in the features?
Unsupervised learning / clustering…
![Page 9: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/9.jpg)
Unsupervised Goal
Find prototypical points within data set that can be used to approximate distribution of the whole data set.
Compression / Visualization Typical methods
Define a measure of similarity between data points
Select initial prototypes Move prototypes to best fit the data –
minimize error
![Page 10: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/10.jpg)
Kohonen Net
Unsupervised technique
Each output neuron represents a prototype point, input layer is presented with data
![Page 11: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/11.jpg)
Self Organizing Map
Each prototype is pulled towards that data, and brings its closest prototypes a little closer as well…
SOM Demo
![Page 12: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/12.jpg)
How is it relevant to our problem?
We will go through:Definition of what we are facingHow we can use neural netsHow we can improve on this method
![Page 13: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/13.jpg)
LIDAR in Space
Data from LITE projectSend a laser beam down from a
shuttle and get reflection readings.Data gathered for 53 hours in 1994.40GB of data
![Page 14: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/14.jpg)
Input
![Page 15: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/15.jpg)
Problem
There is too much data to send and processEach pixel is a number to be
transmitted.Each reading (one column) contains
3000 pixels. Need a way to transmit the
information in a more compact fashion.
![Page 16: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/16.jpg)
Applying Neural Nets
Some readings are very similar to each other.
Define classes that will contain multiple readings.
Define a representative for the class which is close enough to all the class members.
![Page 17: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/17.jpg)
Applying Neural Nets (cont.)
In order to get class Train Kohonen Nets on data for
specific number of classes.Once trained, just pass in a new
reading (3000 data points) and get the class id. Just transfer the class id.
Go from 3000x3000 numbers to be transferred to 3000 numbers
![Page 18: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/18.jpg)
Example
![Page 19: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/19.jpg)
Result – 8 classes
![Page 20: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/20.jpg)
Result – 16 Classes
![Page 21: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/21.jpg)
Result – 64 Classes
![Page 22: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/22.jpg)
Result – 128 Classes
![Page 23: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/23.jpg)
Result – 256 Classes
![Page 24: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/24.jpg)
Problems
Takes a while to train 256 classes took 44 hours on a dual
P3 1GHz computer with 1.5GB of RAM.
Not optimalClassifying by the complete reading is
wasteful.• Need one number for each reading
![Page 25: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/25.jpg)
Improving NN
sky
![Page 26: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/26.jpg)
Improving NN (cont.)
Easier version:Look at blocks of 50x50 pixels instead
of just 1 pixel.Classify as sky, etc.Can preprocess using kNN, basic
filters to get rid of noise in data.Classes can be formed from
sequence of 50 pixel blocks.
![Page 27: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/27.jpg)
Improving NN (cont.)
Harder version:Use variable sized mesh
• In parts with just the sky use a big block.• In detailed parts use smaller blocks.
Something to think about…
![Page 28: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/28.jpg)
Muren Board
Silicon Recognition, Inc. 2 ZISC078 Chips (156 neurons in
parallel) 1 Mb of memory Up to 1 million recognitions/second
![Page 29: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/29.jpg)
ZISC
ZISC, the cornerstone of the Muren board, completely overcomes the three major limitations of software-based pattern recognition applications: ZISC fully handles non-linearity and
fuzziness ZISC is a fully massively parallel processing
system. Recognition speed is consistent, regardless of the number of stored patterns
ZISC does not use any algorithms for programming the solution, ZISC is taught
![Page 30: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/30.jpg)
RBF Space mapping
![Page 31: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/31.jpg)
Example
Source picture
(circuit board)
Result (after processing)
(shows good and bad points)
![Page 32: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/32.jpg)
Process
Learning Classifying Training Testing with new data
![Page 33: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/33.jpg)
NASA Data
![Page 34: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/34.jpg)
Further investigation
Can we write directly to the board? Ability to preprocess data?
![Page 35: Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649e7c5503460f94b7dc96/html5/thumbnails/35.jpg)
Questions?