Fuzzy Inference Systems

Review Fuzzy Models

If <antecedence> then <consequence>.

Fuzzification DefuzzificationInferencing

Input Output

Basic Configuration of a Fuzzy Logic System

Target

Error =Target -Output

Types of RulesMamdani Assilian Model

R1: If x is A1 and y is B1 then z is C1

R2: If x is A2 and y is B2 then z is C2

Ai , Bi and Ci, are fuzzy sets defined on the universes of x, y, z respectively

Takagi-Sugeno Model

R1: If x is A1 and y is B1 then z =f1(x,y)

R1: If x is A2 and y is B2 then z =f2(x,y)

For example: fi(x,y)=aix+biy+ci

Types of RulesMamdani Assilian Model

Takagi-Sugeno Model

Mamdani Fuzzy Models

The Reasoning SchemeBoth antecedent and consequent are fuzzy

The Reasoning SchemeBoth antecedent and consequent are fuzzy

1: IF FeO is high & SiO2 is low & Granite is prox & Fault is prox, THEN metal is highImplication (Max)

0

1

0

1

=

2: IF FeO is aver & SiO2 is high & Granite is interm & Fault is prox, THEN metal is aver

30% 50% 70%

0

1

40% 55% 70%

0 km 10 km 20km

0 km 5 km 10km

0t 100t 1000t

3: IF FeO is low & SiO2 is high & Granite is dist & Fault is dist, THEN metal is low

FeO = 60% SiO2 = 60% Granite = 5 km Fault = 1 km Metal = ?

0t 100t 1000t

=

=

Defuzzifier• Converts the fuzzy output of the inference

engine to crisp using membership functions analogous to the ones used by the fuzzifier.

• Five commonly used defuzzifying methods:– Centroid of area (COA)– Bisector of area (BOA)– Mean of maximum (MOM)– Smallest of maximum (SOM)– Largest of maximum (LOM)

Since consequent is fuzzy, it has to be defuzzified

Defuzzifier

Rule 1: Rule 2: Rule 3: Aggregate (Max)

+ + =

Defuzzify (Find centroid)

125 tonnes metal

Formula for centroid

n

ii

n

iii

x

xx

0

0

)(

)(

Sugeno Fuzzy Models• Also known as TSK fuzzy model

– Takagi, Sugeno & Kang, 1985

If x is A and y is B then z = f(x, y)

Fuzzy Rules of TSK Model

Fuzzy Sets Crisp Functionf(x, y) is very often a polynomial

function w.r.t. x and y.The order of a Takagi-Sugeno type fuzzy inference system = the order of the polynomial used.

While antecedent is fuzzy, consequent is crisp

The Reasoning Scheme

ExamplesR1: if X is small and Y is small then z = x +y +1

R2: if X is small and Y is large then z = y +3

R3: if X is large and Y is small then z = x +3

R4: if X is large and Y is large then z = x + y + 2

TAKAGI-SUGENO SYSTEM1. IF x is f1x(x) AND y is f1y(y) THEN z1 = p10+p11x+p12y2. IF x is f2x(x) AND y is f1y(y) THEN z2 = p20+p21x+p22y3. IF x is f1x(x) AND y is f2y(y) THEN z3 = p30+p31x+p32y4. IF x is f2x(x) AND y is f2y(y) THEN z4 = p40+p41x+p42y

The firing strength (= output of the IF part) of each rule is:s1 = f1x(x) AND f1y(y)s2 = f2x(x) AND f1y(y)s3 = f1x(x) AND f2y(y)s4 = f2x(x) AND f2y(y)

Output of each rule (= firing strength x consequent function) :5. o1 = s1 ∙ z16. o2 = s2 ∙ z27. o3 = s3 ∙ z38. o4 = s4 ∙ z4

Overall output of the fuzzy inference system is: o1+ o2+ o3+ o4 s1+ s2+ s3+ s4

z =

18

Sugeno systemRule1: IF FeO is high AND SiO2 is low AND Granite is proximal AND Fault is proximal, THEN Gold =p1(FeO%)+q1(SiO2%) +r1(Distance2Granite)+s1(Distance2Fault)+t1

Rule 2: IF FeO is average AND SiO2 is high AND Granite is intermediate AND Fault is proximal, THEN Gold =p2(FeO%)+q2(SiO2%)+r2(Distance2Granite)+s2(Distance2Fault)+t2

Rule 3: IF FeO is low AND SiO2 is high AND Granite is distal AND Fault is distal, THEN Gold =p3(FeO%)+q3(SiO2%)+r3(Distance2Granite)+s3(Distance2Fault)+t3

Gold(R1) =p1(FeO%)+q1(SiO2%) + r1(Distance2Granite) +s1(Distance2Fault)+t1

1: IF FeO is high X SiO2 is low X Granite is prox X Fault is prox, THEN

0

1

0

12: IF FeO is aver X SiO2 is high X Granite is interm X Fault is prox, THEN

30% 50% 70%

0

1

40% 55% 70%

0 km 10 km 20km

0 km 5 km 10km

3: IF FeO is low & SiO2 is high & Granite is dist & Fault is dist, THEN

FeO = 60% SiO2 = 60% Granite = 5 km Fault = 1 km Metal = ?

s1

Gold(R2) =p2(FeO%)+q2(SiO2%) + r2(Distance2Granite) +s2(Distance2Fault)+t2

s2

Gold(R3) =p3(FeO%)+q3(SiO2%) + r3(Distance2Granite) +s3(Distance2Fault)+t3

s3

Sugeno system

Sugeno system: OutputGold(R1) =p1(FeO%)+q1(SiO2%) +

r1(Distance2Granite) +s1(Distance2Fault)+t1

s1

Gold(R2) =p2(FeO%)+q2(SiO2%) + r2(Distance2Granite) +s2(Distance2Fault)+t2

s2

Gold(R3) =p3(FeO%)+q3(SiO2%) + r3(Distance2Granite) +s3(Distance2Fault)+t3

s3

Firing strength

Rule output

321

321 )3(*)2(*)1(*sss

RGoldsRGoldsRGoldsOutput

A neural fuzzy system

Implements FIS in the framework of NNs

Fuzzification Nodes

Antecedent Nodes

Output Nodes

x y

Fuzzification Nodes

Represents the term sets of the features.

If we have two features x and y and two linguistic variables defined on both of it say BIG and SMALL. Then we have 4 fuzzification nodes.

x y

BIGBIG SMALL SMALL

We use Gaussian Membership functions for fuzzification ---

They are differentiable, triangular and trapezoidal membership functions are NOT differentiable.

Fuzzification Nodes (Contd.)

zx

exp

2

2

and are two free parameters of the membership functions which needs to be determined

How to determine and

Two strategies:

1) Fixed and

2) Update and , through any tuning algorithm

Consequent nodes

kqypxz p, q and k are three free parameters of the consequent polynomial function

How to determine p, q, k

Two strategies:

1) Fixed

2) Update through any tuning algorithm

Fuzzification nodes

x y

BIG BIG SMALLSMALL

μx1 μx2 μy1 μy2

Antecedent nodese.g. If x is Small & y is Small

Consequent nodes

w1 w2w3 w4

e.g. z4 = p4x + q4y + k4

z1 z2 z3z4

Output node O = (w1z1+w2z2+w3z3+w4z4)/

(w1+w2+w3+w4

Target (t)

Error = ½(t-o)2

ANFIS Architecture

Squares: Adaptive nodesCircles: Fixed nodes

ANFIS ArchitectureLayer 1 (Adaptive)Contains adaptive nodes, each with a Gaussian membership function:

Number of nodes = number of variables x number of linguistic values

In the previous example there are 4 nodes (2 variable x 2 linguistic values for each)

Two parameters to be estimated per node: mean (centre) and standard deviation (spread)

These are called premise parameters

Number of premise parameters = 2 x number of nodes = 8 in the example

2

2)(exp)(

xcxf

ANFIS ArchitectureLayer 2 (Fixed)

Contains fixed nodes, each with product operator (T-norm operator). Returns the firing strength of each If-Then Rule.

The firing strength can be normalized. In ANFIS, each node returns a normalized firing strength –

Fixed nodes – no parameter to be estimated.

yxyx

yxyx

ffsffs

ffsffs

224213

122111

;

;

4321

1

ssssss

ANFIS ArchitectureLayer 3 (Adaptive)

Each node contains an adaptive polynomial, and returns output for each fuzzy If-Then rule

Number of nodes = number of If-Then Rules.

The parameters ps are called consequent parameters.

y)pxp(ps zy)pxp(ps zy)pxp(ps z

y)pxp(ps z

42414044

32313033

22212022

12111011

ANFIS ArchitectureLayer 4 (Fixed)

Sums up the output of each node in the previous layer:

A single node in this layer.

No parameter to be estimated.

z z z z z 4321

ANFIS Training

z z z z z 4321

y)pxp(ps zy)pxp(ps zy)pxp(ps z

y)pxp(ps z

42414044

32313033

22212022

12111011

Linear in the consequent parameters Pki, if the premise parameters and, therefore, the firing strengths sk of the fuzzy if-then rules are fixed.

ANFIS uses a hybrid learning procedure (Jang and Sun, 1995) for estimation of the premise and consequent parameters.

The hybrid learning procedure estimates the consequent parameters (keeping the premise parameters fixed) in a forward pass and the premise parameters (keeping the consequent parameters fixed) in a backward pass.

Squares: Adaptive nodesCircles: Fixed nodes

The forward pass:Propagate informationforward until Layer 3 Estimate the consequent parameters by the least square estimator.

The backward pass:Propagate the error signals backwards and update the premise parameters by gradient descent.

ANFIS Training

ANFIS Training : Least Square Estimation

1. Data assembled in form of (xn; yn)

2. We assume that there is a linear relation between x and y:y = ax + b

3. Can be extended to n dimensions:y = a1x1 + a2x2 + a3x3 + … + bThe problem: Given the function f, find values of coefficients ais

such that the linear combination best fits the data

ANFIS Training : Least Square Estimation

Given data {(x1; y1 (xN ; yN)}, we may define the error associated to saying y = ax + b by:

This is just N times the variance of data : {y1 - (ax1+b),…., yn - (axN +b)}The goal is to find values of a and b that minimize the error. In other words minimize the partial derivative of the error wrt a and b:

ANFIS Training : Least Square Estimation

Which gives us:

We may rewrite them as:

The values of a and b which minimize the error satisfy the following matrix equation:

Hence a and b are estimated using:

ANFIS Training : Least Square Estimation

For the following data find least square estimator

SNo X Y X2 XY1 2 9 4 182 3 11 9 333 4 13 16 524 6 17 36 1025 8 21 64 1686 1 7 1 77 2 9 4 188 11 27 121 2979 14 33 196 462

TOTAL 51 147 451 1157

yyx

xxx

xxxba

yyx

xxx

ba

22

12

1.1.

1

1

ANFIS Training : Least Square Estimation

LSE. use andSimplify

0)y.()]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(2p

:

0)s.()]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(2p

)]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(

:

)]ypxp(ps)ypxp(ps)ypxp(ps)ypxp(ps[(

xxxxxx

:be data trainingLet they)pxp(psy)pxp(psy)pxp(psy)pxp(ps

zzzz o Output y)pxp(ps z y);pxp(ps z y);pxp(ps z y);pxp(ps z

1142141404132131303122121202112111101142

1142141404132131303122121202112111101110

26426414046326313036226212026126111016

21421414041321313031221212021121111011

666

555

444

333

222

111

4241404323130322212021211101

4321

42414044323130332221202212111011

tE

tE

tE

tE

tytytytytyty

.parameters spread and centre update toused are sexpression above he

Similarly, function. membershipfuzzy theis output, theis where

(spread). and (centre) by drepresente be 1Layer in parameters spread and centre Let the2

Output) Actual-Output (Target E 1layer at Error

valuesy)(x, variable- and aluesstrength v firing -

values,parameter the-:in plug ,parameters consequent all of estimate squareleast After the

2

T

ssO

OO

OEE

fiOc

ssO

OO

OE

cE

sc

i

i

i

i

i

i

ii

i

i

i

i

i

i

ii

ii

ANFIS Training : Gradient descent