Pattern Recognition Systems

Pattern Recognition Systems

• One field is concerned with systems that recognize patterns with a classification scheme that has already been determined.– Example: Character recognition, the

categories (26 letter of alphabet) are already known.

– Referred to as pattern classification.

Fuzzy Systems ToolBox, Mark Beale and Howard Demuth

Pattern Recognition Systems

• Other field is concerned with obtaining a classifier for a set of patterns where the categories are not initially known.– Example: Clustering data concerning

patients to see whether patients can be grouped in a way that helps indicate diagnoses.

– Referred to as Pattern clustering


Pattern Classification

• Attempt to pick the correct category when presented with a pattern.– Solution is very problem specific

• Character recognition will be different from recognizing the tank in the battle field


Design a Fuzzy Pattern Classifier

• Pattern Features– Create fuzzy sets that represent features

in the data that are considered important.• Example: ratio of the character’s height to its

width may be important while the size of a character is not!



• Defining the System– fuzzy sets are used to convert crisp inputs to

grades that are then combined by a pattern classifier to obtain grades for each of the possible classification categories.•Highly problem dependent!



• Testing the System– use known and unknown test sets.– Typically, the system will not perform as well as

desired at first.•One may have to test a fuzzy classifier

– find why it gave a wrong result,– make changes to improve the classifier.


Application: Character Recognition


• Goal: design a system that will pick the correct character when it is given a bit map representation of a printed or written input character.– Example: classifying the ten digits o

through 9.

Pattern Features


• INPUT– bit map

• 25 horizontal X 37 vertical pixels (925 pixel field)

• each pixel have a 0 or 1 associated with it.

Pattern Features


• Creating a rule set dealing with 925 inputs would be extremely difficult and time consuming.

• Fuzzify the input, resulting in a compact summary of the overall state of the bit map.

Fuzzy sets for the Horizontal Position


• Define five triangular-shaped fuzzy sets for the horizontal position.– xS = 0:24;– xL = eqshape(3,xS,5);– xG = shapeval(xL, xS);

Fuzzy sets for the Vertical Position


• Define seven triangular-shaped fuzzy sets for the vertical position.– yS = 0:36;– yL = eqshape(3,yS,7);– yG = shapeval(yL, yS);

Fuzzify the input pixel


• Given any pixel (horizontal x and verticle

y), the membership of that pixel can be calculated for each of 5 horizontal and 7 vertical regions.– G1 = fuzzyval (xS,xG,x);– G2 = fuzzyval (yS,yG,x);

Crate 5 x 7 regions of the 2-D bitmap


• Find all combinations of intersections between the two groups of fuzzy sets– G3 = allcomb(‘min’, G1, g2);

• G3 wil be a 35-element column vector of grades associated with the 35 regions of the 2-D bit map

Multiple Pixels


• If vectors X and Y represent the input pixels, then– G1 = fuzzyval(xS, xG, X);– G2 = fuzzyval(yS,yG, Y); and– G3 = allcomb(‘min’, G1, G2);

• G3 represents a 35 -element column for every pixel crossed by all of the input pixels

Single Grade for each 5 x 7 regions


• Taking the union of all grades associated with each square region:– G = or (G3 ’ ) ’;

• We now have 35 grades, each grade measuring to what extent the user’s drawn character passed through a different region.

• 925-pixel bit map ----> 35 grades!

Defining the System


• Once the good features have been found, one can define a system to transform the feature grades into the grades for each class.

Defining the System


• Zero:• G0 = and([ or( G([2 7])); or( G([3 8]));

or( G([4 9]));• or( G([11 12])); or( G([16 17]));

or( G([21 22]));• or( G([14 15])); or( G([19 20]));

or( G([24 25]));• or( G([27 32])); or( G([28 33]));

or( G([29 34]));• not( G([13 18 23]) ) ]);

Defining the System


• One:

Defining the System


• One:• G1 = or([ and([ G([5 9 18 27 31]);• not(G([1 2 3 6 7 11 25 29 30 33 34 35])) ]);• and([ G([1 7 18 29 35]);• not(G([3 4 5 9 10 15 21 26 27 31 32 33])) ]);• and([ G([1 6 11 16 21 26 31]);• not(G([3 4 5 8 10 13 15 18 20 23 25 ...• 28 30 33 34 35])) ]);• and([ G([2 3 4 5]);• not(G([6 7 10 11 12 15 16 17 20 25 30 35]));• or( G([8 9]));• or( G([13 14]));• or( G([18 19]));• or( G([23 24]));• or( G([28 29]));• or( G([33 34])); ]) ]);

Defining the System


• Two:• G2 = and([ somewhat(G(2));• G([3 32 33 34]);• or( G([6 7])); or( G([30 25]));

or( G([4 5]));• not([ G([22 28]); and(G([10

15])) ]) ]);

Defining the System


• Three:• G3 = and([ G([2 3 19 32 33 34]);• or( G([9 10]) );• or( G([14 15]) );• not( G([5 16 21]) ) ]);

Defining the System


• Four:• G4 = and([ or([ and(G([4 14 19 29])); and(G([5 15 20

30])) • and(G([3 13 18 28])) ]);• or([and(G([12 14 15])); and(G([17 19 20]));• and(G([22 24])) ]);• not([ G([1 2 6 7]); and(G([32 33 34])) ]) ]);

Defining the System


• Once the grades for each class have been found, they can be combined into a single grade vector, cG– cG = [G0 G1 G2 G3 G4 G5 G6 G7 G8

G9];• THEN THE MAXIMUMGRADE CHARACTERS CAN

BE FOUND

– C = cS(find(cG >= max(cG)));

Testinging the System


• Demo– Matlab : numrec

Pattern Clustering

Fuzzy Logic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall

• Unsupervised Clustering– Find interesting patterns or groupings

in a given set of data.• Example: perform the task of

“segmenting” the images (I.e., partitioning pixels on an image into regions that correspond to different objects or different faces of objects in the images).

Pattern Clustering


• Conventional clustering algorithms find a “Hard partition” of a given dataset based on certain criteria that evaluate the goodness of a partition.– “Hard Partition” - each datum belongs to

exactly one cluster of the partition.– In many real-world clustering problems,

some data points partially belong to multiple clusters (eg. A pixel in a MRI may correspond to a mixture of two different types of tissues.

Soft Clustering Algorithms

Fuzzy Logic: Intelligence, Control, and Information, J. Yen and R. Langari, Prentice Hall

• Find a “soft partition” of a given data set based on certain criteria.– A datum can partially belong to

multiple clusters– In practice, most soft clustering

algorithms do generate a soft partition that also forms a fuzzy partition.

Fuzzy C-Means Algorithm


• Best known fuzzy clustering algorithm– produces a constrained soft partition

• I.e. cj(xi) = 1

• Both fuzzy c-means and probabilistic clustering produce a partition of similar properties, the clustering criteria underlying these algorithms are very different!



• Generalizes a hard clustering algorithm called the c-means algorithm– Hard C-Means algorithm (HCM) aims

to identify compact well-separated clusters

• compact cluster has a “ball like” shape• the center of the ball is called the center

or the prototype of the cluster.

Example of two clusters that are not compact and well separated


Example of two clusters that are compact, but not well separated




• Like HCM, fuzzy c-means also tries to find a good partition by searching for prototypes vi that minimize the objective function Jm

• k=1,2,…c• m is a weight that affects how partial

members of a cluster affect the clustering result

• FCM also needs to search for membership function ci

that minimize Jm

Fuzzy C-Means Theorem


• A constrainted fuzzy partition {C1, C2, …, Ck} can be a local minimum of the objective function Jm only if the following conditions are satisfied:

• Based on this theorem, FCM updates the prototypes and the membership function iteratively using Equation A and B until a convergence criterion is reached

Fuzzy C-Means - Example


• Use FCM to partition the data set into two clusters (c=2), suppose we set the parameter m in FCM at 2, and initial prototypes to v1 = (5,5) v2 = (10,10)



• The initial membership functions of the two clusters are calculated using Equation A

Example (Continued)


• Similarly,

Example (Continued)


• FCM algoruthm then updates the prototypes according to Equation B– For V1

Example (Continued)


• FCM algoruthm then updates the prototypes according to Equation B– For V2

Example (Continued)


• The updated prototype v1 and v2 are move closer to the center of the cluster formed by x1,x2, x3 and x4,x5,x6 , respectively

• Show Matlab example

Recognition of Geometric shapes

Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively

• Used in PDAs

• Interactive CAD systems


Multi-stroke shapesManuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively


Uni-stroke shapesManuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively


Recognition algorithm:

1. Use global geometric properties of the shapes

2. Use a set of filters to either to identify shapes or to filter out unwanted outcomes

3. Use fuzzy logic to handle uncertainty



Feature extraction

Use statistical properties of shape’s features!



Circle: Thinness ratio Ach/ P2ch

Ach – area of the convex hullP2

ch – perimeter squared

(line from 10% to 90%, bar from 25% to 75%)


Line: Thinness ratio Her/ Wer

Her – height of enclosing rectangleWer – width of enclosing rectangle



Rectangle: first ratio R1 = Ach/ Aer

Aer – area of enclosing rectangleAch – area of convex hull



Rectangle: second ratio R2 = Alq/ Aer

Alq – area of largest quadrilateralAer – area of enclosing rectangle



Diamond: ratio of Alt/ Alq

Alt – area of largest triangleAlq – area of largest quadrilateral



Triangle/Move: first ratio T1= Alt/ Ach

Alt – area of largest triangleAch – area of convex hull



Triangle/Move: sec. ratio T2= Plt/Pch

Plt – perimeter of largest trianglePch – perimeter of convex hull

How to distinguish between triangle and move?


Another parameter – HOLLOWNESS-Place a triangle with approx. 60% area of the convex hull in the center of mass of a shape-Count shape points inside the triangle



Features used by each shapeManuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to Recognize Geometric Shapes Interactively


Defining fuzzy sets

a – minimum from a feature distribution

b – 10% from a feature distribution

c – 90% from a feature distribution

d – maximum from a feature distribution



There are 4 ambiguous cases for which it is impossible todefine membership functions in the way demonstrated in the previous slide.



The membership functions for the ambiguous cases are defined as shown.



Rules:



Recognition accuracy:

Problems: poor recognition of arrows


Pattern Recognition Systems

Documents

Transcript of Pattern Recognition Systems