Temporal Knowledge Acquisition From Multiple Experts
description
Transcript of Temporal Knowledge Acquisition From Multiple Experts
![Page 1: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/1.jpg)
Temporal Knowledge Acquisition From Multiple Experts
Temporal Knowledge Acquisition From Multiple Experts
by Helen Kaikova, Vagan Terziyan
![Page 2: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/2.jpg)
The problem of knowledge acquisition
The problem of knowledge acquisition
T T1 2
Event 1 Event 2
DOMAINTemporal
relation
Most supported opinion
Experts
Experts'
ranking
1. How to derive the most supported knowledge from the multiple experts about the unknown temporal relation between the two events?
2. How to make quality evaluation of the most supported opinion?
3. How to make semantic analysis of the most supported opinion to make necessary corrections?
4. How to make, evaluate, use and refine ranking of all the experts to improve the results?
![Page 3: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/3.jpg)
Basic concepts: Basic relationsBasic concepts: Basic relations
M
Group 1 Group 2
M1 X-< Y
- M4 X-< Y
+
M2 X-> Y
- M5 X-> Y
+
M3 X-= Y
- M6 X-= Y
+
Group 3 Group 4
M7 X+< Y
- M10 X+< Y
+
M8 X+> Y
- M11 X+> Y
+
M9 X+= Y
- M12 X+= Y
+
T1
T2
Relation between endpoints
M - the set of m basic relations between temporal endpoints of two intervals [ X-, X+] and [ Y-, Y+]
![Page 4: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/4.jpg)
Basic concepts: Experts and Ranks Basic concepts: Experts and Ranks
with ranks
S - the set of n knowledge sources or experts
S1 S2 S3 S4
R1 R2 R3 R4
![Page 5: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/5.jpg)
BeforeT
T1
2T T1 2
AfterT
T1
2T T1 2
R 1
R 2
Met-byT
T1
2T T1 2 R 4
MeetsT1
2T T1 2 R 3
T
DuringT
T1
2T T1 2 R 5
T2
T1 IncludesT T1 2 R 6
OverlapsT
T1
2T T1 2
OverlappedT
T1
2 T T1 2
R 7
R 8
Started-byT
T
1
2
T T1 2 R 10
StartsT1
2T T1 2 R 9
T
FinishesT
T1
2T T1 2 R 11
T2
T1 FinishedT T1 2 R 12
by
by
T2
1 T T1 2 R 13EqualsT
Basic concepts: RelationsBasic concepts: Relations
R - the set of r basic relations for temporal intervals
(Allen’s basic temporal relations)
T1
T2
Relation:
T1 Overlaps T2
Relation between endpoints
![Page 6: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/6.jpg)
Basic conceptsBasic concepts
R M - t h e m a t r i x r × m d e f i n e s r e l a t i o n s h i p b e t w e e n t h e
r e l a t i o n s ’ s e t s R a n d M b y t h e f o l l o w i n g w a y :
R R M M R M R Mk q k q k q, ( ( ) ), ;
R M X- < Y
-X
- > Y-
X- = Y
- … X+ = Y
+
R 1 1 0 0 … 0
R 2 0 1 0 … 0
… … … … … …
R 1 3 0 0 1 … 1
![Page 7: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/7.jpg)
Basic concepts Basic concepts
T - the set of t temporal intervals which correspond to the domain events
The Domain
Event 1 Event 2
Event 3
T1
T3
T2
TemporalIntervals
??
?
![Page 8: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/8.jpg)
Basic concepts: Semantic predicateBasic concepts: Semantic predicate
P - the semantic predicate which defines piece of knowledge about temporal relationship by the following relation between the sets T, R and S:
P T R T S
if the knowledge source S
uses R to describe relation
between T and T
otherwise
a k b i
i
k
a b( , , , )
,
;
, .
1
0
![Page 9: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/9.jpg)
Basic Concepts: Basic Concepts:
SMa,b - the matrix n×m with expert knowledge about
two fixed temporal intervals Ta and Tb; it defines relationship between the set of knowledge sources S and set of basic relations M by the following way:
T T T S S M M R
R P T R T S R M SM
a b i q k
a k b i k q i qa b
, , , ,
(( ( , , , )&( )) ).,,
![Page 10: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/10.jpg)
Refinement strategyRefinement strategy
•All experts have the same initial rank, equal to one.
•After each vote the rank of each expert should be recalculated.
•An expert improves his rank after some vote if his opinion has less conflicts with the most supported one than the average number of conflicts among all the experts. Otherwise he loses some part of his rank. The responsibility of each expert for the results of his voting should grow from vote to vote.
•An expert’s rank should not be changed after some vote if expert does not participate it or his opinion has as many conflicts with the most supported one as the average number of conflicts among others.
•An expert’s rank should always be more than zero and less than n (number of experts).
•The value of punishment (or prize) for presence (or absence) of each conflict should be maximal for expert whose rank is equal to n/2.
•The value of punishment (or prize) for presence (or absence) of each conflict should be aspire to zero for expert whose rank is close to zero or to n.
![Page 11: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/11.jpg)
Ranking votes refinement techniqueRanking votes refinement technique
rvi - is the rank of i-th expert before v-th vote
rvi - is the rank of i-th expert before
v-th vote
0 r niv
r i i ni1 1 1 , ( , . . . , )
VOTE a,b - vector which contains results of the current experts’
voting concerning relation between intervals Ta and Tb derived
from the matrix SM a,b as follows:
VOTE abs
a b t q m
qa b
qa b
qa b, , ,( ),
, , , ,
1 1
where qa b
iv
ii SM
nr
i qa b
,
,( ),
,
1
qa b
iv
ii SM
nr
i qa b
,
,( ),
,
0
![Page 12: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/12.jpg)
The most supported opinionThe most supported opinion
MSUP a,b - is the vector which contains most supported opinion
concerning relation between intervals Ta and Tb derived as follows:
( ) ,
, , , ,
, , , qa b
qa b
qa bMSUP
a b t q m
0
1 1
![Page 13: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/13.jpg)
Correction rulesCorrection rules
1. If <all the components from the same group of the most supported opinion are equal to zero>,
then <that one which has the least vote should be changed to one>.
a b t h h
MSUP q Group
s Group VOTE
VOTE MSUP
qa b
h
h sa b
Groupa b
sa b
h
, , , ,
, (( , )&
&( (
min( )))) .
,
,
, ,
1
1 4
![Page 14: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/14.jpg)
Correction rulesCorrection rules
2. If <all the components from the same group of the most supported opinion are equal to zero and more then one of them has minimal vote in the group>,
then <that one which corresponds to the relation of equivalence between temporal points should be changed to one>.
a b t h h Group
q q q MSUP q Group
VOTE VOTE VOTE OR
OR VOTE VOTE
VOTE MSUP
h
qa b
h
qa b
qa b
qa b
qa b
qa b
qa b
qa b
, , , , , ((
, , )&( , )&
&((( ) )
(( )
))) .
,
, , ,
, ,
, ,
1 1 4
1 2 3
1 3 2
2 3
1 3
![Page 15: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/15.jpg)
Correction rulesCorrection rules
3. If <there are more than one components from the same group of the most supported opinion which are equal to one>,
then <that one which has the least vote should be changed to zero>.
a b t h h MSUP
MSUP q s Group VOTE
VOTE MSUP
sa b
qa b
h sa b
qa b
sa b
, , , , , (( &
& , , )&(
)) .
,
, ,
, ,
1 1 4
![Page 16: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/16.jpg)
Correction rulesCorrection rules
4. If <there are more than one equal to one components from the same group of the most supported opinion including the last one and they have the same vote>,
then <those ones which are not correspond to the relation of equivalence between temporal points should be changed to zero>.
a b t h h Group q q q
s Group q MSUP MSUP
VOTE VOTE MSUP
h
h qa b
sa b
qa b
sa b
sa b
, , , , , (( , , )&
&( )&( & &
&( ))) .
, ,
, , ,
1 1 4 1 2 3
33
3
![Page 17: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/17.jpg)
Refinement of Expert ranksRefinement of Expert ranks
r r riv
iv
iv 1
rcon
coniv
iv
iv
viv
convi - is the number of
conflicts between opinion of i-th expert and the most supported opinion calculated through all set M during the v-th vote:
con SM MSUP
a b t i n v
iv
i qa b
qa b
q
m
( ),
, , , , , , ,
,, ,
1 1 1
iv i
vivr n r
n
( )
1
i n
1
where vjv
j
n
ncon
1,
![Page 18: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/18.jpg)
Most supported opinion “quality” evaluationMost supported opinion “quality” evaluation
Q Votes accepted as most supported opinion
All votes
Q
VOTE
m rva b
qa b
q
m
iv
i
n,
,
![Page 19: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/19.jpg)
ProgramProgram
Experts’ votes on 3 problems
Resulting Most supported opinion
Consensus’dynamics
Few tasks canbe maintained
Qualities
Starting
![Page 20: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/20.jpg)
The reverse example (first vote)The reverse example (first vote)
Old ranks New ranks
Most supported on this stage
Endpoints’ relations
![Page 21: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/21.jpg)
Ranks’ Changing During The ExampleRanks’ Changing During The Example
0
0.51
1.5
2
2.53
3.5
4
1 11 21
Experts' votings
Exp
ert
s' ra
nks
0
0.2
0.4
0.6
0.8
1
S 1
S 2
S 3
S 4
Quality
MSUP: 12,1,8
Qu
ali
ty
![Page 22: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/22.jpg)
Ranks’ Changing During The Reverse Example
Ranks’ Changing During The Reverse Example
0
0.51
1.5
2
2.53
3.5
4
1 11 21
Experts' votings
Exp
ert
s' ra
nks
0
0.2
0.4
0.6
0.8
1
S 1
S 2
S 3
S 4
Quality
MSUP: 7,3,12
Qu
ali
ty
![Page 23: Temporal Knowledge Acquisition From Multiple Experts](https://reader035.fdocuments.us/reader035/viewer/2022062423/568149f3550346895db72433/html5/thumbnails/23.jpg)
ConclusionConclusion
The method can also be used to derive the most supported knowledge in domains that have a well-defined set of basic relations (such as the set M) and a set of compound relations (such as the set R). In those cases, experts are allowed to give incomplete or incorrect knowledge about a relation from the set R because it can be handled by deriving the most supported opinion of multiple experts through the set M.
Ranking technique in this research supports experts whose opinion is close to the most supported one. It even may happen that only one expert remains after several votes whose rank becomes more than all other ranks together. In many applications, the most supported knowledge is not the best one. Moreover the talent individuals can easily lose their rank if they are thinking not like others. That is why method should be developed to pick up and classify experts whose opinions are the most different from the most supported opinion, and then try to take them into account.