ADAPTIVE RESONANCE THEORY (ART)

PPT. BYASHUTOSH

B.TECH COMPUTER SCIENCE ENGINEERINGUIET, KURUKSHETRA

CONTENTS

• INTRODUCTION• STABILITY-PLASTICITY DILEMMA• BASIC ARCHITECTURE• DIAGRAM FOR ARCHITECTURE• DIAGRAM FOR SUPPLEMENTAL UNIT• ALGORITHM• APPLICATION

Introduction to ARTIt is a self organising neural network architecture.It is an unsupervised learning.This type of network remains open to new learnings without washing

away previously learned code.ART n/w and algo. Maintain the plasticity required to learn new patterns,

while preventing the modification of pattern that have been learned previously.

ART is a vector classifier, it accepts input vector and categorise depending upon which of stored pattern it match most or resembles the most.

If input is not match to any pattern than a new category is made by storing this new pattern.

No stored pattern is modified if i/p is not matched with any pattern.

Stability-Plasticity-Dilemma

• Real world facing a situation where data is continuously changing.• It produce stability-plasticity dilemma• Stability – system behaviour does not changes after irrelevant events.• Plasticity – system adapts its behaviour according to significant

events.• Dilemma – Preservation of learned Knowledge.

Basic Architecture

1. Input processing field – F1 LAYER2. Cluster unit – F2 LAYER3. Reset mechanism – It controls degree of similarity of patterns placed

on some cluster.The input processing layer is divided into two parts F1(a) – It present input vector given. F1(b) – It is a interface portion between F1(a) and F2The signal b/w F1 and F2 is used to comparing similarity of input signal to weight of vector unit which is selected for storing pattern or for recognising pattern.

Continue…• F2 is connected to F1(b) through top down weights t(ji).• F(b) is connected to F2 through bottom up weights b(ij).• Cluster unit with largest i/p weight is selected for learning the new

pattern and all other cluster’s activation is set to zero.• Each F2 layer neuron/cluster have associated weight with it, that

cluster will be allowe to learn a new pattern whose dot product of own weight and input vector is maximal. Only that neuron will respond back.• Reset: Information from the input unit is combined with interface unit

, depending upon similarity b/w top down weight and i/p vector cluster may or may not be allowed to learn a new pattern.

Diagram for architecture

Continue…

The above shown diagram is computational unit(F1(a),F2(b),F2, but it have drawback. i.e :-1. These units have to behave differently at different stages.2. Also problem with reset mechanism is that F2 unit has to be inhibited in certain conditions but have to be returned to availability on subsequent learning trails.So, Supplemental unit (G1 and G2) is introduced in addition to reset. These units send their signal from and to all others units in structure.• Inhibitory signals shown by “-”• Excitory signal shown by “+”

Diagram for supplemental unit.

AlgorithmParameters used in algorithm are :n – Number of component in i/p vector.m – Maximum number of clusters that can be formed.b(ij) – Bottom up weight.t (ji) – Top down weight.p – Vigilance parameter ( set by user tester for testing the acceptance level for cluster to train if net i/p is less than p then it will not trained and vice-versa.) s – Binary input vector.x – Activation vector for F1(b).||x|| - norm of vector x ( sum of all x(i)).

Description of Algorithm.

1. Binary i/p vector is applied to F1(a) then it is passed to F1(b) and it is send to F2 over weighted interconnections.

2. Each neuron at F2 will calculate the net i/p i.e dot product of inpuct vector and its own weight.

3. The neuron with highest net i/p will win and learn new pattern and activation d=1 for that neuron.

Steps to Perform.

Step 1 : Initialize parameterL>1 and 0<p<=1Initialize weights0<b(ij)<L/(L-1+N), t(ji)(0)=1Step 2 : while stoping condition is false repeat step 3 to14Step 3 : for each testing i/p repeat step 3 to 13 Step 4 : Set activation of all F2 units to zero

set F1(a) activation to input vector s.

Continue…

Step 5 : compute norm of s.||s||= S(i)

Step 6 : set i/p signal from F1(a) to F1(b)x(i)=s(i)

Step 7 : for each node of F2 that is not inhibited,if y(j)≠-1, y(j)=

Continue…

Step 8 : while reset is true do step 9 to 12Step 9 : find J such that y(J) ≥ y(j) for all J.

if y(j) = -1, then all oods are inhibited and the selected cluster can not be trained.Step 10 : recompute activation of x F1(b).

x(i)= s(i)t(ji)Step 11 : compute norm of x.

||x||=

Conitnue…

Step 12 : Test for reset||x||/||s|| < p then,

y(j)=-1 cluster can not be trained , do step 8 again. else, proceed to step 13.Step 13 : Update weights of node j

b(ij) new = Lx(i)/(L-1+||x||)t(ji) new = x(i)

Step 14 : Test for stopping condition.

Applications of ART

• Face recognition• Image compression• Signature Verification• Medical diagnoses• Target recognition• Land cover classification

References

• Introduction to Neural Networks Using Matlab 6.0 By S. N. Sivanandam, S. N Deepa• Wikipedia

THANK YOU