Parallel Artificial Neural Networks Ian Wesley-Smith Frameworks Division Center for Computation and...
-
Upload
kristin-spencer -
Category
Documents
-
view
214 -
download
0
Transcript of Parallel Artificial Neural Networks Ian Wesley-Smith Frameworks Division Center for Computation and...
Parallel Artificial Neural Networks
Ian Wesley-SmithFrameworks DivisionCenter for Computation and TechnologyLouisiana State Universityhttp://cct.lsu.edu/~iwsmith
Basics of ANNs
• Vague model of biological neural network
• No authoritative definition for ANNs– Group of small computational
components (neurons) networked together
Strengths of ANNs
– Inherently Non-Linear– Learn
• Supervised• Input-Output Mapping
– Simple enough to implement in hardware
Good at:– Optimization problems (traveling
salesman)– Pattern classification (handwriting
analysis)
Examples of ANNs
• Digital Signal Processing (DSP)• Optical Character Recognition (OCR)• Sales Forecasting• Industrial Process Control• SONAR/RADAR• Medical Assessment• Games
Examples of ANNs
• Robot Army
Components of Neurons
• Inputs– Vector
• Weights– Matrix [inputs x outputs]
• Output (activation) Function– Threshold– Sigmoid
Example Neuron
x1
x2
xn
.
.
.
f(x)
WeightsInputs Output Function
A Single Neuron
Perceptrons• Simplest ANN• Single Layer• Single Neuron
– Can be more
• Simple pattern classifiers– Only classify linearly separable sets
• Learning with the delta-rule • Output function is threshold
Sample Data
Perceptrons
• Computation
x1
x2
xn
.
.
.
f(x)
WeightsInputs Output Function
A Single Neuron
w=Weight Column Vectori= Input Vectorf i⋅w
Perceptrons
• Delta-Learning
n= Indexw=Weight Column Vectori= Input Vectorz=Learning Constante=Errorerror=desired response−actual response
w n=w n z∗e∗in
Methodology
• Implicit parallelism• Neurons are independent of one another• Calculations are relatively simple• Process large datasets faster
• Implementation– Serial to Parallel Implementation– PETSc
• Portable Extensible Toolkit for Scientific Computing
– Run Details• AMD Dual Opteron• 2 Processor Run• Varying Sized Data Sets (100-10 million)
Results
• Parallel ANN was functional• Parallel implementation performs slower
than serial– This is expected
• Possible Reasons– Single Neuron Problem– PETSc/MPI Overhead
Future Work
• Implement more advanced (recurrent) networks
• Hand code MPI instead of relying on PETSc
• Test in larger environments– 32 processors minimum
Acknowledgments
• Yaakoub Y. El-Khamra• Dr. Gabrielle Allen• Dr. Ed Seidel• Kathy Traxler• Louisiana State University
– Center for Computation and Technology– Computer Science Department