Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research...
-
Upload
allen-austin -
Category
Documents
-
view
213 -
download
0
Transcript of Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research...
![Page 1: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/1.jpg)
Adaptive fuzzy Adaptive fuzzy clusteringclustering
Kharkiv National University of Radio Electronics
Control Systems Research Laboratory
![Page 2: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/2.jpg)
Crisp Approach to Clustering
AB
C
1x
2x
![Page 3: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/3.jpg)
Fuzzy Approach to Clustering
1x
2x
AB
C
),( BAw
),( CBw),( CAw
),,( CBAw
![Page 4: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/4.jpg)
Fuzzy Approach to Clustering
Input Data SetNkXxxxxX kN ,...2,1,},,,...,,{ 21
Goal Function of Clustering
N
k
m
jjkjkjjk cxdwcwE
1 1
2,, ),(),(
Batch Approaches
m
jjk Nkw
1, ,...,1,1
N
kjk mjNw
1, ,...,1,0
mNWwW jk dim},{ ,
![Page 5: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/5.jpg)
Batch Fuzzy Clustering Algorithms: Probabilistic ApproachFuzzy clustering as a constrained optimization problem
nN RxxxX }...,{ ,2,1 m Divide a data set into clusers.
N
k
m
jjkkjjk cxdwcwE
1 1
2, ),,(,),(
,,...,1,11
, Nkwm
jjk
.,...,1,01
, mjNwN
kjk
(1)
(2)
(3)
Lagrange function:
N
k
m
j
N
k
m
jjkkjkjkkjjk wcxdwcwL
1 1 1 1,
2,, .1),(),,( (4)
![Page 6: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/6.jpg)
Batch Fuzzy Clustering Algorithms: Probabilistic ApproachDegrees of membership:
,
)),((
))((
1
1
1
2
1
1
2
,
m
llk
jkjk
cxd
cxdw (5)
prototypes:
N
kjk
N
kkjk
j
w
xw
c
1,
1,
(6)
A Bkx 2,kw1,kw
px
2,pw1,pw
1x
2x
5.02,1,2,1, ppkk wwww
![Page 7: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/7.jpg)
Batch Fuzzy Clustering Algorithms: Possibilistic Approach
2, , ,
1 1 1 1
( , ) ( , ) (1 ) .N m m N
k j j k j k j j k jk j j k
E w c w d x c w
0j
.
),(
1,
1
2,
N
kjk
N
kjkjk
j
w
cxdw
Unconstrained optimizationModified objective function:
Scalar parameters determine the distance at which the degrees of membership are equal to 0.5:
(8)
(7)
![Page 8: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/8.jpg)
Batch Fuzzy Clustering Algorithms: Possibilistic Approach
,),(
1
1
1
12
,
j
jkjk
cxdw
.
1,
1,
N
kjk
N
kkjk
j
w
xw
c
Possibilistic algorithmDegrees of membership:
prototypes:
(10)
(9)
![Page 9: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/9.jpg)
Adaptive Fuzzy Probabilistic Algorithms
m
j
m
jjkkjkjkkjjkk wcxdwcwL
1 1,
2,, ).1(),(),,(
,
)),((
)),((
1
1
1
,2
1
1
,2
,
m
llkk
jkkjk
cxd
cxdw
).( ,1,,,1 jkkjkkjkjk cxwcc
One-step Largange function:
Degrees of membership:
prototypes:
(13)
(11)
(12)
Unsurprised fuzzy competitive learning (UFCL) algorithm
![Page 10: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/10.jpg)
Adaptive Fuzzy Probabilistic Algorithms
, 1 ,
1,,
( ), ,
, ,
k j k k k j
k jk j
c x c j lc
c j l
l
1
2
1,
2,
,
m
llkk
jkkjk
cx
cxw
).( ,12,,,1 jkkjkkjkjk cxwcc
UFCL is an extension of the Kohonen competitive learning rule:
where
When , UFCL transforms into the Gradient-based fuzzy c-means (GBFCM) algorithm
prototypes:
(16)
(14)
(15)
is a number of the “winning” prototype.
Degrees of membership:
![Page 11: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/11.jpg)
Adaptive Fuzzy Possibilistic Algorithms
m
j
m
jjkjjkjkjjkk wcxdwcwE
1 1,
2,, .)1(),(),(
,),(
1
1
1
12
,
j
jkjk
cxdw
).( ,1,,,1 jkkjkkjkjk cxwcc
One-step objective function:
Adaptive possibilistic algorithmDegrees of membership:
prototypes:
(19)
(18)
(17)
![Page 12: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/12.jpg)
Adaptive Fuzzy Possibilistic Algorithms
,2
,
,
jkkj
jjk
cxw
).( ,12,,,1 jkkjkkjkjk cxwcc
When β=2Degrees of membership:
prototypes:
(20)
(21)
![Page 13: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/13.jpg)
Adaptive Fuzzy Possibilistic Algorithms
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
wkj
mu=0.1mu=0.2mu=0.3
Possibilistic membership function for 3.01.0,2
![Page 14: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/14.jpg)
Matrix Fuzzy C-means Algorithm
1 2
2,{ ( )} , 1,2,... ,n ni iX x k R k N
1
1( ), ( ) ( ( ) ( )),
NT
k
x x k x k Tr x k x kN
1 1
( , ) ( ) ( ( ) )( ( ) )N m
Tj j j j j
k j
E u c u k Tr x k c x k c
1
1,m
jj
u
1
0 ( ) , 1,..., .N
jk
u k N j m
Input Data Set(22)
.Goal Function of Clustering
.Batch Approaches(25)
. (26)
(23)
(24)
![Page 15: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/15.jpg)
Matrix Fuzzy C-means Algorithm
Karush-Kuhn-Tucker equation system:
1 2
1
1
( ( ), , ( ))( ) ( ( ), ) ( ) 0,
( )
( ( ), , ( ))( ) 1 0,
( )
( ( ), , ( ))2 ( )( ( ) ).
( )
j jj j
j
mj j
jjj
Nj j
j jkj
L u k c ku k D x k c k
u k
L u k c ku k
k
L u k c ku k x k c
c k
O
(27)
![Page 16: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/16.jpg)
Matrix Fuzzy C-means Algorithm
Fuzzy clustering algorithm:
12 1
12 1
1
112 1
1
1
1
( ( ( ), ))( ) ,
( ( ( ), ))
( ) ( ( ), ) ,
( ) ( ).
( )
jj m
ll
m
ll
N
jk
j N
jk
D x k cu k
D x k c
k D x k c
u k x kc
u k
(28)
![Page 17: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/17.jpg)
Matrix Fuzzy C-means Algorithm
J. Bezdek’s FCM-algorithm:
1
1
1
2
1
2
1
( ( ( ) )( ( ) ) )( ) ,
( ( ( ) )( ( ) ) )
( ) ( ).
( )
Tj j
j mT
l ll
N
jk
j N
jk
Tr x k c x k cu k
Tr x k c x k c
u k x kc
u k
. (29)
![Page 18: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/18.jpg)
Experiments
MaxSelector
1x
2x
nx
mw
2w
1w
Class number
Classification system based on fuzzy clustering (the number of the rule nodes is equal to the number of classes)
![Page 19: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/19.jpg)
Experiments
Data Fuzzy c-means
Batch possibilistic
Park-Dagher Recursive possibilistic
Iris 7,4% 7,6% 6,9% 7,8%Wine 3,8% 4,4% 3,9% 4,3%
Data Fuzzy c-means
Batch possibilistic
Park-Dagher Recursive possibilistic
Iris, class 3 33,3% 14,0% 33,3% 15,3%Iris, class 1 38,0% 6,0% 38,0% 6,7%Wine, class 3 29,0% 19,6% 29,0% 19,6%Wine, class 1 35,9% 23,6% 35,9% 23,4%Thyroid, class 3
18,1% 20,9% 18,6% 11,6%
Thyroid, class 1
15,4% 17,2% 15,5% 5.6%
Table 1: Classification with known number of classes (error rate on testing data)
Table 2: Classification with one unknown class (error rate on testing data)
![Page 20: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/20.jpg)
Robust Probabilistic Fuzzy Clustering Algorithms
N
kj
m
jjkjjk
R ckxDkwcwE1 1
,, ),()(,
,,,1,1)(1
,
m
jjk Nkkw
.,,1,)(01
,
N
kjk mjNkw
,1 ,)(,
1
1
pckxcxD
pn
i
pjiijk
, ,k jx jic i )1( njk cx ,
Subject to constraints
(31)
(32)
where are th components of -vectors respectively.
(33)
(30)
![Page 21: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/21.jpg)
Robust Probabilistic Fuzzy Clustering Algorithms
,s
cx
s),sSe(c),cp(x
i
ii
iiiii
2sech
2
1
icis
i
iiiiii
cxcxf
coshln),(
i
iiii
xxxf
tanh)()(
n
i
n
i i
jiiijuiij
R ckxckxfckxD
1 1
)(coshln),(),(
(34)
where and are the parameters that define the center and width of the distribution respectively.
, (35)
(36)
(37)
![Page 22: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/22.jpg)
Robust Probabilistic Fuzzy Clustering Algorithms
.)(
coshln),(),(1 1 11 1
N
k
m
j
n
i i
jiiij
N
k
m
jj
Rjjj
E ckxwckxDwckwE
N
k
m
jj
i
jiiN
k
m
j
n
iijjj kwk
ckxkwkckwL
1 11 1 1
1)()()(
coshln)()(,),(
.0))(,),((
,0)(
))(,),((
,0)(
))(,),((
kckwL
k
kckwL
kw
kckwL
jjc
jj
j
jj
j
Objective function for robust clustering
Lagrange function
The system of Kuhn-Tucker equations
(40)
(39)
(38)
![Page 23: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/23.jpg)
Robust Probabilistic Fuzzy Clustering Algorithms
N
kj
Rcjjjc
m
ll
R
ml l
R
jR
j
ckxDwkckwL
ckxDk
ckxD
ckxDkw
jj1
1
1
1
1
11
1
1
1
0),())(,),((
,),()(
,
),(
),(()(
m
jjj
Rm
jjjjk kwkckxDkwkckwL
11
)1)()((),()())(,),((
Local Lagrange function
(42)
(41)
![Page 24: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/24.jpg)
Robust Probabilistic Fuzzy Clustering Algorithms
,)()(
tanh)()()(
)(,),()()()1(
,
),(
),(()(
11
1
1
1
i
jiijji
ji
jjkjiji
ml l
R
jR
prj
kckxkwkkc
c
kckwLkkckc
ckxD
ckxDkw
jikc , i jk
Arrow-Hurwitz-Uzawa procedure
where is the -th component of the -th prototype vector calculated at the -th step.
(43)
![Page 25: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/25.jpg)
Robust Possibilistic Fuzzy Clustering Algorithms
N
kj
m
jj
N
kj
m
jjjjj kwckxDkwckwE
111 1
)(1),()(),),((
N
kj
Rcjjjjc
Nk j
Nk i
Rj
j
j
jR
posj
ckxDwckwE
kw
ckxDkw
ckxDkw
jj1
1
1
1
1
1
.0),(),),((
,)(
),()(
,),(
1)(
The criterion
The system of Kuhn-Tucker equation
(45)
(44)
![Page 26: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/26.jpg)
Robust Possibilistic Fuzzy Clustering Algorithms
n
i
m
jji
i
jiii
m
jj
m
jjj
m
jj
Rjjjj
Rk
kwckx
kw
kwckxDkwckwE
1 11
11
)(1)(
coshln)(
)(1),()(,),(
,)()(
tanh)()()(,),(
)()()1(
,),(
1)(
1
1
1
i
jijji
ji
jjjkjiji
j
jR
posj
kckxkwkkc
c
ckwEkkckc
ckxDkw
Local Lagrange function
Adaptive possibilistic fuzzy clustering algorithm
(47)
(46)
![Page 27: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/27.jpg)
Robust Possibilistic Fuzzy Clustering Algorithms
12 ))(1()( cxxp ii Experiments (48)
Complete data set (left) and its central part (right)
![Page 28: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/28.jpg)
Robust Possibilistic Fuzzy Clustering Algorithms
Cluster prototypes layout obtained by different algorithms
![Page 29: Adaptive fuzzy clustering Kharkiv National University of Radio Electronics Control Systems Research Laboratory.](https://reader038.fdocuments.us/reader038/viewer/2022110206/56649cfa5503460f949cbef8/html5/thumbnails/29.jpg)
Robust Possibilistic Fuzzy Clustering Algorithms
Table 3: Classification results
AlgorithmClassification error rate
Training Checking
Bezdek’s fuzzy c-means 17.1 % (1229 samples) 16.6 % (299 samples)
Probabilistic robust clustering (41) 15.6 % (1127 samples) 15.6 % (281 samples)
Possibilistic robust clustering (47) 15.2 % (1099 samples) 14.6 % (263 samples)