Pattern Recognition: Statistical and Neural
35
1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 19 Oct 26, 2005 Nanjing University of Science & Technology
description
Nanjing University of Science & Technology. Pattern Recognition: Statistical and Neural. Lonnie C. Ludeman Lecture 19 Oct 26, 2005. Lecture 19 Topics. 1. Structures of Optimal Statistical Classifiers 2. Neural Network History 3. Biological Neural Networks - PowerPoint PPT Presentation
Transcript of Pattern Recognition: Statistical and Neural
1
Pattern Recognition:Statistical and Neural
Lonnie C. Ludeman
Lecture 19
Oct 26, 2005
Nanjing University of Science & Technology
2
Lecture 19 Topics
1. Structures of Optimal Statistical Classifiers
2. Neural Network History
3. Biological Neural Networks
4. Modified McColloch Pitts Model – Example
5. Artificial Neural Element - Definition
3
Structures from Statistical Decision Theory and Perceptron Algorithm
(a). Two Class Likelihood Ratio test
(b) Structures to Motivate Neural Networks
(c) Two Class General Linear
(d). N Class General minimum P(error)
(e) N Class Gaussian
4
if l(x) > T decide x from C1 < T decide x from C2 = T decide x randomly
Decision Rule
between C1 and C2
5
if f(x) > T<C2
C1Decision Rule
6
if f(x) > T<C2
C1Decision Rule
7
if p(x | Ck) > p(x | Cj ) for all j ≠ k decide Ck
Decision Rule
8(e) M Class Gaussian
9
These Structures will be seen to be similar to the structures used in designs with Neural Networks
10
Important Events in Neural Networks History
1943 McColloch Pitts Model
1958 Perceptron Algorithm- Rosenblatt
1960-62 ADALINE – Widrow and Hoff
1969 Minsky and Papert- Limitations
1980’s Grossberg , Hopfield, and Rumelhart – Backpropagation algorithms, Adaptive Resonance Theory
Period of Revival
1990’s Maturation of Field
11
IMPORTANT BOOKS
1990 Artificial Neural Systems- Jacek M. Zurada
1992 Neural Networks and Fuzzy Systems- Bart Kosco
1994 Neural Networks: A Comprehensive Foundation- Simon Haykin
12
Neural Networks
Biological (Real)
Mathematical (Artificial)
How do you tell them apart ???
Squeeze them !!!
Excite them !!!
13
Biological Neuron
14
Action Potential
15
16
Biological Neural Network
17
End Bulb Connection
18
Integration at axon-dendrite junction
19
Modified McColloch Pitts Neuronal Model
(Threshold Logic Unit (TLU) )
20
What can we do with a Modified McColloch-Pitts Model ???
Question
Answer
Surprisingly we can model all logical expressions !!!
21
T = n - 1/2 T = 1/2
Implementation of Logical AND and OR
22
Since we can model logical AND, OR and
NOT we can model all logical expressions
Implementation of Logical NOT
-1/2
23
Example – Implementation of a given logic expression
Given:
Implement f(x) using Modified McColloch-Pitts Neurons
24
Solution:
-1/2
-1/2
-1/2
-1/2
2.5
3.5
T= 1/2
-1
25
Artificial Neural Element (ANE)
Node
netInput Vector
26
net = w1x1 + w2x2 + …+ wnxn + wn+1
= wTx
y = f(net) = f( wTx)
Artificial Neural Element Mathematical Model
Nonlinear Activation Function
Linear Operation on x
NonlinearOperation on x
27
Artificial Neural Element Nodal Representation
Vector Notation
28
Some Activation Functions
29
30
Hyperbolic Tangent Activation Function
f(net) Equivalent Form
Definition
31
Hyperbolic Activation Function
32
33
Non Monitonic Activation Functions can be very Useful
Examples:
Potential Function Approach
Radial Basis Functions
34
Summary Lecture 19
1. Structures of Optimal Statistical Classifiers
2. Neural Network History
3. Biological Neural Networks
4. Modified McColloch Pitts Model – Example
5. Artificial Neural Element - Definition
35
End of Lecture 19
