Abstract This poster presents results of three studies dealing with application of ARTMAP neural networks for classification of remotely sensed multispectral images. 1. Comparison of performance of ARTMAP classifiers with different types of cluster representation. In this study, the best results were obtained using Extended Gaussian ARTMAP.2. A method for computation of classification accuracy index for Gaussian ARTMAP neural network. This method can be used to generate maps containing only pixels with prescribed minimum accuracy of classification.

Methods for analysis and enhancement of neural network classification of remotely

sensed images

Norbert Kopčo, Peter Sinčák, and Rudolf JakšaDepartment of Cognitive and Neural Szstems, Boston University

Computational Intelligence Group, Technical University of Košice, Slovakia

3. Preliminary results obtained on a hierarchical ARTMAP architecture based on set of dichotomical classifiers. This neural network is suitable for parallel processing of large data sets.

Refrences

Sinčák, P., Veregin, H., and Kopčo, N.: Computational intelligence for classification of remotely sensed images. Neural Network World, v. 5, 1998, pp. 577-593.Sinčák, P., Kopčo, N., and Veregin H. (unpublished) Conflation Techniques to Improve Image Classification Accuracy. Submitted to Photogrametric Engineering and Remote Sensing.Carpenter, G. and Grossberg, S. (1992) Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps. IEEE Trans. on Neural Networks, 3: 197-214.Williamson, J.R. (1996) Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps, Neural Networks, pp. 881-897.Cunningham, R. K. (1998) Learning and recognizing patterns of visual motion, color, and form. Unpublished Ph.D. thesis. Boston University.

1. Evaluation of dependence on cluster representation in ARTMAP classifiers

• There is a large number of classification algorithms available without detailed knowledge of their properties/performance.

• Neural networks are considered to be assumption-less classifiers (for a successful classification they do not require explicit assumptions about the data distribution).

• But these systems have implicit assumptions built into them. These assumptions are related to the data representation and algorithmic properties of the system.

• In the present study, the dependence of the ARTMAP classifiers performance on the internal cluster representation is analyzed for image data from remote sensing.

Data-set• Seven-dimensional Landsat TM image of the city of Košice

(Figure 1)• Size of image: 368,125 pixels, out of which 6,331 classified

by an expert into seven categories (A - urban area, B - barren fields, C - bushes, D - agricultural fields, E - Meadows, F - Woods, and G - water)

Method of analysis• The performance is compared in terms of weighted PCC

(Percent of Correctly Classified) and the contingency tables.

Compared systems• Fuzzy ARTMAP - FA (Carpenter et al., 1992)

Gaussian ARTMAP - GA (Williamson, 1996)Extended Gaussian ARTMAP - EGA (Cunningham, 1997)

Figure 1: Original image

ARTMAP classifier topology

Input OutputMapField(Labeling)

Recognition(Clustering)

Comparison

Cluster representation:• Fuzzy ARTMAP: Hyper-rectangles• Gaussian ARTMAP: Gaussian

distributions without covariance• Extended Gaussian ARTMAP:

Gaussian distribution with covariance

F0 F1 F2 MF OL

FA

EGA

GA

Results

Set #1 Set #2 Set #3 Set #4 Set #5 VotingFuzzy ARTMAP 93.72 91.48 90.82 90.82 92.16 93.95Gauss. ARTMAP 93.90 93.57 93.49 94.24 93.09 94.04Ext. Gauss. ARTMAP 93.82 94.14 94.14 94.07 93.88 94.23

Weighted PCC for five permutations of the training set and voting

• Gaussian distribution more suitable as representation of the clusters in image classification tasks (see Fig. 2)

• Best performance achieved by Extended Gaussian ARTMAP (although differences not very significant)

Actual classA B C D E F G TOTAL

A 91.55 1.41 5.63 1.41 0.00 0.00 0.00 4.93B 0.77 99.23 0.00 0.00 0.00 0.00 0.00 37.03C 0.00 0.00 86.74 0.00 1.10 9.94 2.21 0.06D 0.00 0.00 0.00 96.33 0.00 3.67 0.00 42.84E 0.00 0.00 .49 0.00 9.51 0.00 0.00 6.54F 0.00 0.00 .85 8.83 0.00 89.32 0.00 2.56G 0.00 0.00 4.49 0.64 0.00 0.00 94.87 5.43

TOTAL 2.24 36.87 5.72 28.37 6.48 15.39 4.93 100.00

Contingency table for Extended Gaussian ARTMAP

Pre

dict

ed c

lass

Figure 2: Image classified by Extended Gaussian ARTMAP

• Sensitivity to the ordering of the training set smaller for GA and EGA than for the FA

• During learning, FA always reached 100% accuracy. GA/EGA accuracy was around 97.7%, which means stronger generalization.

Title:KOVARVO3.EPSCreator:CorelDRAW!Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

2. Confidence Index

• In remote sensing, a large amount of data is produced, which needs to be quickly and reliably classified.

• Often, explicit assessment of the reliability is needed.

Goals:

• Develop a method for simple and fast assessment of the accuracy of classification.

• For the accuracy assessment, exploit computations done during the classification process.

• Develop the assessment method for the Gaussian ARTMAP algorithm, which achieved the best performance in the previous study.

Method• GA uses Bayes discrimination function:

(1)

• The pattern is classified into the category with the largest probability measure:

(2)

)(

)(exp)(kj

jk IgIR

• If voting is used, the probability measures are evaluated over all the networks:

(3)

V

l kjljk IgIR

1 )(, )(exp)(

• The confidence index is then defined as:

(4)

M

k k

K

IR

IRIc

1)(

)()(

)(log)|(log)( jjj PIpIg

Figure 3: Confidence map for Gaussian ARTMAP classification (expressed in %)

Confidence threshold• A confidence threshold can be defined, which, in combination with the confidence index, can be used to generate maps with arbitrary accuracy.• Two counteracting aspects of classification are influenced by the choice of threshold: 1) accuracy of classification, and

2) number of unclassified patterns (see graph below)

0.4 0.5 0.6 0.7 0.8 0.9 10.5

0.6

0.7

0.8

0.9

1

Threshold

% o

f pix

els

abov

e th

resh

old

0.4 0.5 0.6 0.7 0.8 0.9 10.95

0.96

0.97

0.98

0.99

1

Threshold

% o

f cor

rect

ly c

lass

ified

pix

els

Figure 4: Thresholded classification map for Gaussian ARTMAP (accuracy 99%, =0.921)

Results and discussion

• The presented method offers a simple and fast way for assessment of quality of the Gaussian ARTMAP neural network classification.

• The method can be used also with other neural networks employing the voting strategy.

• The method offers a tool for production of maps with arbitrary classification accuracy.

• Because of the way in which this method exploits the computations done during the classification, it is possible that the confidence assigned to some of the classified points is incorrect.

• Often, the complexity of the data is too high, or the size of the data set is too large (in terms of memory or time requirements), for standard versions of classifiers.

• To overcome this problem, hierarchical/modular classification systems are usually applied.

• Here, a hierarchical structure based on ARTMAP networks is proposed, and its properties are analyzed.

• The system is called Parallel ARTMAP, because it consists of ART sub-nets, each trained independently on a single category (i.e., each sub-net learns to detect data from one category).

• In the following simulations, fuzzy ART/ARTMAP system is used in the sub-net modules of the network.

3. Hierarchical classifier

• Parallel processing system• Every sub-net in the

clustering layer is trained on data belonging to a single class, i.e., it learns to detect data fromthat class.

• Conflict resolution module serves to resolve conflicts if two or more clustering nets identify a pattern as “theirs”.

• Two design choices:1) rule for determination of optimal for each clustering

subnet,2) conflict-resolving rule.

ART

Parallel ARTMAP system

Clustering subsystem(class detectors)

Conflict resolution

Class 1 data

Class 2 data

Class 3 data

Class n data

ART

ART

ART

Optimal determination rule

Each sub-net is trainedon patterns from a single class using cross-validation:- 9/10 - estimation set,- 1/10 - validation set.Training is repeated fordifferent values of .

Rule:Choose which corresponds to the the point just before the first dipin the “true positive” graph.

This rule assures that each detector will correctly identify almost allof the patterns belonging to it, while minimizing the false alarm rate.

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

Graph of performance of clustering sub-net for class #7

Parameter

Perc

en

t of

corr

ect

ly c

lass

ified

Other classesTrue “–” rate

My class (#7)True “+” rate

Optimal

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 1

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 2

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 3

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 4

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 5

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 6

PC

C

0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1Class 7

PC

C

Optimal for all sub-nets

Other classesTrue negative rate

My class True positive rate

Optimal

• For the previously defined -determination rule, all the sub-nets will have almost 100% true positive rate.

• But their false alarm rate (false negative rate) will be non-zero. So the nets are biased towards identifying patterns as “theirs”. Moreover, the false alarm rate will be different for each sub-net.

• So the conflict resolving rule can be based on the true negative rate.

Conflict-resolving rule

Rule:

If there is a conflict between two or more sub-nets, assign the patternto the class with the largest true negative rate.

This rule was chosen, because the parameters needed for conflict resolution (true “-” rates) are easily computed during the -determination process.

Actual class

A B C D E F G NONE TOTALA 80.28 0.43 41.44 0.89 0.00 1.44 2.56 4.93B 14.08 99.57 0.00 0.00 0.00 0.00 0.00 37.03C 0.00 0.00 1.10 0.00 0.00 0.00 0.00 0.06D 0.00 0.00 23.76 98.00 0.00 88.91 0.00 42.84E 0.00 0.00 1.66 0.00 99.51 0.00 0.00 6.54F 0.00 0.00 20.99 0.11 0.00 8.62 0.00 2.56G 1.41 0.00 10.50 0.00 0.00 0.41 96.15 5.43

NONE 4.23 0.00 0.55 1.00 0.49 0.62 1.28 0.60TOTAL 2.24 36.87 5.72 28.37 6.48 15.39 4.93 100.00

Results and discussion

Overall classification accuracy is 78.89%.

The decreased performance is caused mainly by the misclassification of the patterns from classes C and F.Despite worse performance, the system can be usefully applied to large data sets. Also the speed of the system can be advantageous. Choice of alternative rules for -determination and conflict resolution can improve the performance.

Pre

dict

ed c

lass

Contingency table for Parallel ARTMAP