Novel Approaches to Optimised Self-configuration in High

Performance Multiple Experts

M.C. Fairhurst and S. HoqueUniversity of Kent

UK

A. F. R. Rahman BCL Technologies Inc.

USA

Basic Problem Statement

• Given a number of experts working on the same problem, is group decision superior to individual decisions?

Is Democracy the answer?

• Infinite Number of Experts• Each Expert Should be Competent

How Does It Relate to Character Recognition?

Each Expert has its:• Strengths and Weaknesses• Peculiarities• Fresh Approach to Feature Extraction• Fresh Approach to Classification• But NOT 100% Correct!

Practical Resource Constraints

Unfortunately, We Have Limited• Number of Experts• Number of Training Samples• Feature Size• Classification Time• Memory Size

Solution

• Clever Algorithms to Exploit Experts– Complimentary Information– Redundancy: Check and Balance– Simultaneous Use of Arbitrary Features and

Classification Routines

How are they Employed?

Expert1 Expert 2 Expert n

Horizontal Systems

How are they Employed?

Vertical Systems

Expert 1

Expert 2

Expert n

How are they Employed?

• Combined System:– A hybrid of Horizontal and

VBertical– More Complicated to

Analyse?– Even more Complicated to

Optimise?

What to Optimise?

• Number of Experts in a configuration• Type of Expert in each Position in the

hierarchy• Optimising Criteria

– Do we want a fast system? Or– Do we want an accurate System?

Proposed Methodology

• Genetic Algorithm: A Generalised Search and Optimisation Method

• Problem Coding:– Chromosome Structure– Fitness Function– Genetic Operators

Methodology • Chromosome Structure: A

Classifier is a Machine Obeying a Set of Production Rules. A Generalised Rule is:<classifier>::=<condition>:<message>– <condition> part is a pattern matching

device– <message> part is a feedback

mechanism

Methodology

• Fitness Function: Fitness = Correct_Patterns/Total_Patterns

• Correct_Patterns corresponds to the number of correctly identified patterns in one cycle

• Total_Patterns corresponds to the number of total patterns being fed to the optimising process

Methodology • Genetic Operators:

– Reproduction: • Weighted Roulette Wheel (Goldberg)• Stochastic Remainder Selection (Booker)• Tournament Selection (Brindle)

– Crossover: Swapping at [1,l-1]– Mutation: Random variation

• Single gene only

Selection of a Specific Problem

Expert 1

Expert 2 Expert 3

Expert 4

Decision Compilation

Selection of a Database• Machine Printed Characters Extracted from

British Envelopes• Collected Off-line• Total 34 Classes (0-9, A-Z, no Distinction

between 0/O and I/1)• Total Samples of Over 10,200 characters• Size Normalised to 16X24

Performance of the Classifiers

Classifiers % Error

BWS 1.76

FWS 1.52

MPC 3.90

MLP 1.66

Performance of the Combination

Classifier Position % Error

BWS 1

FWS 4

MPC 3

MLP 2

1.03

The Optimised Combination

Classifier Position % Error

BWS Unused

FWS 2

MPC 1

MLP 3

0.92

Generality of the Solution: Generation of a Vertical System

Expert 1

Expert 2

Expert 4

Decision Compilation

Input Pattern

Classification Decision

Expert 1

Expert 2

Expert 3

Input Pattern

Classification Decision

Optimization for the Vertical System

Optimized Parameters

BWS Sub-set size

FWS Sub-set size

MPC Sub-set size

MLP Sub-set size

2 10 4 8 1 5 3 2

Combined % Error: 1.01

Generality of the Solution: Generation of a Horizontal System

Expert 2 Expert 3

Expert 4

Decision Compilation

Input Pattern

Classification Decision

Expert 1 Expert 2

Expert 3

Decision Compilation

Input pattern

Decision Combination

Classification Decision

Optimization for the Horizontal System

Optimized Parameter

BWS FWS MPC MLP Error %

Weighting Factor

0.14 0.53 0.11 0.22 0.92

Conclusion• Multiple Expert Solutions can be made more

Robust by optimising these structures• Optimisation is made with GA approach• The adopted multiple expert configuration is

generic: it can produce both vertical and horizontal systems (in addition to the hybrid system)

• The optimization approach is generic: it man optimize both vertical and horizontal systems (in addition to the hybrid system)