Brasil 96
-
Upload
astefanovich -
Category
Documents
-
view
220 -
download
0
Transcript of Brasil 96
-
8/12/2019 Brasil 96
1/6
A Methodology for Implementing Hybrid Expert Systems
L.M. Brasil, F.M. de Azevedo, R.Garcia and J.M. Barreto.Dept. of Electrical Engineering,
Federal University of Santa Catarina
Biomedical Engineering Research Group (GPEB)
University Campus, Florianpolis, Brazil, 88040-900
Phone (48)231-9594 Fax(48) 231-9770 E-Mail: [email protected]
Abstract - This paper deals with a new methodology for
developing an Expert System (ES). It has the ability of learning
to extract knowledge from a poor knowledge base using the
learning by example paradigms. The choice of a poor knowledge
base was motivated by the fact that in this case it is easier to
have consistence in putting together the several pieces of
knowledge. So, the problems attached to knowledge elicitation
are simplified. The implementation leads to a Hybrid Expert
System (HES). This system consists of a Neural Network based
Expert System (NNES) and a Rule Based Expert System(RBES). The main idea is that if the knowledge engineer has
conditions to obtain some basic rules, and a set of examples,
from the domain expert then it is possible to define the basic
structure of the NNES using those basic rules. Then the NNES
can be refined using the set of examples. In this stage structural
changes of the network are allowed by the learning algorithm.
Rules can be deduced after this refinement. Then it can be used
to form a RBES and an Explanatory Expert System (EES). The
methodology developed to HES is intended to be used in
implementing Decision Support Systems in the Medical Area.
I -INTRODUCTION
At present, one of the most known products of the
Artificial Intelligence (AI) is ES. It has proved its efficiency
independently of the implementation paradigm adopted:
symbolic manipulation or connectionist.
There are several problems in building an ES. One of them
is the process of Knowledge Acquisition (KA), which
comprehend knowledge extraction from a domain expert and
the choice of the model for the knowledge representation to
code human reasoning [1]. Knowledge elicitation stage
consumes time mainly because normally human beings, even
knowing how to solve a problem, have difficulty in
explaining how they reached the solution of a specific
problem by themself. Moreover, there is the problem of
knowledge representation of a domain expert. There are
many forms to represent it. When symbolic manipulation is
used, production rules are one of the most common
knowledge representation schemes used. It is simple and
direct, but it relies on a rich knowledge base. Moreover, it is
necessary to update it constantly. In case of connecionist
paradigm, the problems have been basically the choice of
data to be fed to the input of neural network (NN), the
number of neurons in the hidden layer and the kind of
topology of the NN used. Finally, only in some special cases,
there are difficulties in obtaining the explanation on how the
network arrived to a conclusion.
So, a HES is proposed to deal with the problems
mentioned above. This system consists of a NNES, a RBES
and, an EES [2].
The paper presents the architecture and operation of the
HES, the system methodology, the learning algorithm, and
discusses the proposed approach.
II- PROPOSED SYSTEM
The proposed architecture for ES includes a NNES, a
RBES, and an EES.
NNES has been used to implement ES as an alternative
manner to RBES. Artificial Neural Networks (ANN) are
made through a big number of units. These units own some
properties like natural neurons. Therefore, every unit presents
several inputs, some excitatories and others inhibitories.
Moreover, these units take values of each input and generate
an output that is function of the inputs. So, a network ischaracterized by the units (neurons) and by the way the
neurons are connected (topology). Moreover, algorithms are
used to change weights of the connections (learning rules).
Thus, these three aspects constitute the connectionist
paradigm of the artificial intelligence [3]. The
implementation of ES this way are called NNES. These
systems are generally developed using a static network with
feedforward topology trained by a backpropagation-like
learning algorithm [4]. So, while the basic network represents
relations among concepts and connections by way of
inferring something through it, the set of examples will refine
NNES. In this last process, the algorithm provides
modifications not only in the weights of connections, but alsoin the network structure. It uses this topology and it generates
and/or eliminates connections that had not been in basic
rules. Moreover, it can also occasionally generate more
concepts that were not in the basic rules. Therefore, the
system translates as rules the basic rules that the expert was
not able to extract. The basic rules after extraction suffers a
treatment due to the kind of variables applied as input of the
network, where they represent different types of concepts, as
quantitative, linguistic, boolean or a combination of them
-
8/12/2019 Brasil 96
2/6
[2,4,5]. Moreover, a structural modification of the ANN
consists in determining the number of neurons of the hidden
layer using a Genetic Algorithm (GA) [6].
RBES broadly depends on formal logic as a way of
explicit knowledge representation. Our system has two kinds
of data: basic rules and set of examples. The basic rules are
used to create the initial NNES which is refined through the
examples. The refined NNES is then translated in a RBES
from which explanations can be obtained. The model
proposed is through fuzzy logic. The theory of fuzzy logic
provides a good mathematical framework to represent
imprecise knowledge as in our case [2,5,7].
Finally, the EES of the HES is derived from RBES. It
compares the answers given by the NNES and by the RBES.
If the two answers are equals the EES is triggered and it will
give an explanation. Otherwise, it will state the impossibility
of reaching the goal and it will suggest how to obtain a
suitable solution [2].
III -METHODOLOGY
One of the difficulties in eliciting the knowledge of domain
experts is when one wishes to obtain an adequate set of rules.
Firstly, in many fields experts are not capable of realizing or
articulating which knowledge they use in solving their
problems. Secondly, different experts have often different
explanations to their decisions and sometimes even their
decisions disagree. [3].
On the other hand, a NNES needs only a set of examples to
represent the problem considered. However, as the
knowledge is often distributed in the network connections,
explaining how the NNES reached a conclusion is very
difficulty except if the knowledge representation is localized.Or, explaining the ES reasoning is of capital importance,
mainly when the user disagrees with the ES suggestion and
also during the tuning phase. This is extremely important in
our case, where physicians are the potential users. So, the
proposed methodology deals with a hybrid scheme, which
explores the intrinsic suitable characteristics of each
approach (Fig.1).
KNOWLEDGE ACQUISITION
INTEGRATED SYSTEM
EXPLANATORY SYSTEM
RBES
REFINED NNES
NNES
SET OF
EXAMPLES
BASIC
RULES
Fig.1 - Block diagram of the general system.
A.Knowledge Acquisition
The KA task consists on extracting knowledge of the
domain expert (i.e., a physician expert). In our case, the main
goal is to minimize the intrinsic difficulties of the KA
process. To do so, we try to obtain all possible rules from the
domain expert in a short time and also a set of examples.
The basic rules can be improved capturing the
uncertainties associated with the human cognitive processes.
The model proposed uses fuzzy logic [5].
Basic rules are translated by AND/OR graphs, so that they
define the NN basic structure of the NNES. In other words,
an AND/OR graph, which represent concepts and
connections, indicates the neurons number in the input and
output layers. The graph also shows the existence of
intermediate concepts and their connections which they
translate in the intermediate layer of the NN, according to
Fig.2.
R6
R6: If Q Then X.
R5: If P Then X.
R4: If G And H Then Q.
R3: If E And F Then Q.
R2: If C And D Then P.
R1: If A And B Then P.
Basic Rules:
R4R3
R5
R2R1
X
QP
HGFEDCBA
Fig.2- Organization of the rules base as a network.
Basic rules consist of an if-part, that indicates the
antecedent and it expresses, in our case, the symptoms of a
disease, while a then-part deals with the consequent and it
expresses the possible diagnostics [8]. Moreover, each of
these rules presents a membership degree. It corresponds tothe value that will be put in the inputs of the basic NN after
the treatment of the semantic variables.
B.Neural Model
A NNES structure that has conditions to receive several
kinds of semantic variables, i.e., boolean, linguistic and,
quantitative inputs, is considered (Fig.3). These inputs have
been treated as a unique kind of variable of fuzzy type. It is
believed that this sort of structure deals with modeling a
structure by a possible simpler way and also consuming less
learning time than a traditional structure.
Quantitative
Boolean
Linguistic
STATEMENTS
Neural
Inputs
(Neural
Outputs)
DecisionsNeural
Network
Learning
Algorithm
Interface
to Fuzzy
Fig.3 - State Variable Kinds
-
8/12/2019 Brasil 96
3/6
The mathematics model of the neuron is given by,
X(t)= n-dimensional input vector
X(t) = [x1(t), x2(t),..., xi(t),...,xn(t)]T n (1)
y(t)= scalar output of each neuron
y(t) 1
N: nonlinear mapping function, X Y, x(t) |y(t)
where: X:
+
n
and Y:
+
(2)
This mapping can be noted as N and so:
y(t) = N [X(t) n] 1 (3)
Mathematically, the neural nonlinear mapping function N
can be divided into two parts: a function called confluence
[5] and a nonlinear activation operation. The confluence
function is the name given to a general function having as
arguments the synaptic weights and inputs. A particular case
widely use is the inner product.
This mapping yields a scalar output u(t) which is a
measure of the similarity between the neural input vector X(t)
and the knowledge stored in the synaptic weight vector W(t).So, u(t) and W(t) are given by,
W(t) = [w0(t), w1(t),..., wi(t),..., wn(t)]T n+1 (4)
u(t) 1
Redefining X(t) to include the bias x0(t) we have:
u(t) = X(t) W(t) (5)
The nonlinear activation function maps the confluence
value u(t) [-,] to a bounded neural output. Then, the nonlinear
activation operator transforms the signal u(t) into a bounded
neural output y(t), that is,
y(t) = [u(t)] (6)
y(t) = [W(t) X(t)] 1 (7)
Applying the equations (1), (4), (5), and (7) to a multilayer
NN (e.g.,three layers), we have:
Y(t) = N3[N2[N1[X(t)n]]] m (8)
Y(t) = 3[W3(t) 2[W2(t) 1[W1(t) X(t)]]] (9)
Where i is non-linear activation operator, is the
confluence operator, and W1(t), W2(t), and W3(t) are the
synaptic weight vectors for the input, hidden and output
layers, respectively.
If we express the neural input signals in terms of their
membership functions each over the interval [0,1], rather
than in their absolute amplitudes, then we can perform
mathematical operation on these signals using logical
operations such as AND/OR, according to [5].
Let us express the inputs x1 and x2 over [0,1]. Then we
define the generalized AND (T-norm) as a T mapping
function and generalized OR (T-conorm) as a S mapping
function [7]:
T: [0,1] x [0,1] [0,1]
S: [0,1] x [0,1] [0,1], given by:
y1= [x1AND x2] [x1T x2] = T[x1, x2] (10)
y2= [x1OR x2] [x1S x2] = S[x1, x2] (11)
Then, in (5), by replacing the -operation by the T-
operation, and the -operation by the S-operation, we get
u(t) = Si
n
=0[wi(t) T xi(t)] [0,1] (12)
and
y(t) = [u(t)] [0, 1] (13)
C. Other Stages of the HES
After the basic NNES is obtained, the set of examples
serves to validate the neural network structure. In worst case,
the network does not represent the knowledge of the
problem. Therefore, it becomes evident that the basic rulesextracted from the expert are not sufficient, as expected. So,
these same examples are used by the learning algorithm to
refine the NN. This algorithm can change, generate and/or
eliminate connections, or it can still generate or eliminate
neurons in the hidden layer. After the refinement of the
network, a new discussion is made with the domain expert to
validate the modifications in the basic structure of the
network. Thus, a new set of examples is obtained to test
again the network. In the case that it has well performed, it is
supposed the network represents the proposed goal.
After the NNES is refined, a reverse process is followed
toward the inferring of the if-then rules together with their
membership degrees. So, a RBES is implemented. Following,
the RBES serves as basis for developing other system, the
EES, that is supposed able to explain why the NNES reached
a conclusion.
IV- LEARNING ALGORITHM
The learning algorithm developed for NNES is inspired on
the traditional backpropagation algorithm. Nevertheless, it
presents some differences:
- Optimization of the hidden layer is supported by GA.
- Incorporation of logic operators AND/OR in place of theweighted sum.
Therefore, the conventional backpropagation learning
algorithm is altered in function of these modifications. Some
works that integrate these topics are [2,4,5,6,8,9,10].
A.Learning Algorithm Stages
Summarizing, the first stage in the implementation of the
learning algorithm is considered as the assembly of the NN
basic structure. It is based in function of a AND/OR graph, in
-
8/12/2019 Brasil 96
4/6
other words, the hidden layer consists of AND nodes and the
output layer of OR nodes. So, the logic operators AND/ OR
are incorporated in place of the weighted sum in the network
neurons [5].
In next stage, the learning algorithm uses examples, which
are extracted from the domain expert, such as modifying, not
only the weight of each connection, but also the network
structure. In this last case, connections are included or
excluded among neurons, as well as neurons can be included
or excluded of the hidden layer by the application of the GA
[6].
B. Genetic Algorithm
GA are based on the work of Holland [11] where he was
inspired by the evolution of a population subject to
reproduction, mutations and crossover in a selective
environment. So, following this idea, we choose GA to
optimize [6] the size of the hidden layer and determining
weights to be set to zero. This can be justified by the
following main facts: it allows to avoid local optimum andprovides near-global optimization solutions and they are easy
to implement. Nevertheless, when it is applied with this goal,
in hidden layer of a NN, we must take care of respecting a
maximum and a minimum number of neurons of this layer. In
fact, too many neurons generally have as effect a decrease of
the generalization capabilities of the network and implies a
long learning phase. On the other hand, too few neurons can
be unable to learn, with the desired precision, the task. So,
there is an intermediate number of neurons that must be put
in the hidden layer, to avoid the problems mentioned above.
Then, the network must be sufficiently rich to solve a
problem and as it must also be adequately simple to solve a
problem well as it must not consume longer time of training.The parameter called as momentum, which it has the
objective to give higher speed of training to network, can still
be added in this same algorithm [12].
V- RESULTS AND DISCUSSIONS
An HES, including a RBES and a NNES, is discussed
under the aspects of KA, where the treatment of imprecision
is a possibility of explaining the reasoning to attain a
conclusion. The KA phase is based on learning by example
paradigms and the intrinsic capabilities of the NNES are
exploited. Soon afterwards, that knowledge can be
transferred to the RBES that uses fuzzy logic to deal with
imprecision. This methodology has proved, in the
preliminary studies performed, very promising by leading to
an easier KA phase than expected if the KA was performed
using symbolic techniques alone. The hybridism, on the other
hand, allows to complement the NNES with explanation of
reasoning facilities, that in most cases are difficult to obtain
with a NNES.
In future we intend to validate the approach with a
concrete example. This example will be probably the
construction of a Medical Decision Support System to
classify epileptic crises. This system will have about 15 to 30
rules combined with their membership degrees. These data
will be elicited by physician experts, mainly at the University
Hospital of Federal University of Santa Catarina.
ACKNOWLEDGMENT
The first author acknowledges the CAPES (Coordination
Foundation of High Level Personnel Improving) and the last
author the CNPq (National Counsil for Scientific and
Technological Development) - RHAE Program - for the
material support in the development of this work.
REFERENCES
[1] L.M. Brasil, "Aquisio de Conhecimento Aplicada aoDiagnstico de Epilepsia", M.Sc.Thesis in Electrical
Engineering, Federal University of Santa Catarina,
Florianpolis, UFSC, 1994.
[2] L. M. Brasil, F.M. de Azevedo, R. Garcia and J.M. Barreto
Cooperation of Symbolic and Connectionist Expert System
Techniques to Overcome Difficulties. In: Proceedings of 2nd
Neural Networks Brazilian Congress, Brazil: Curitiba,
pp.177-182, 1995.
[3] F.M. de Azevedo, "Contribution to the Study of Neural
Networks in Dynamical Expert Systems", Ph.D. Thesis,
Institut d'Informatique, FUNDP, Belgium, 1993.
[4] S. Mitra and S.K. Pal, "Logical operation based fuzzy MLP
for classification and rule generation", Neural Networks,vol.7, no.2, pp.353-373, 1994.
[5] M.M. Gupta and D. H. Rao, "On the principles of fuzzy neural
networks", Fuzzy Sets and Systems, vol.61, no.1, pp.1-18,
1994.
[6] R. G. Ojeda, F.M. Azevedo and J.M. Barreto, "Genetic
algorithms in the optimal choice of neural networks for signal
processing", in 38th. Midwest Symposium on Circuits and
Systems, Rio de Janeiro, Brazil. (In Press), 1995.
[7] H.J. Zimmermann, Fuzzy Set Theory - and Its Applications,
Massachusetts, USA: Kluwer Academic Publishes, 1991.
[8] L.M. Fu, "Knowledge-based connectionism for revising domain
theories", IEEE Transactions on Systems, Man, and
Cybernetics, vol.23, no.1, pp.173-182, 1993.[9] H. Ishibuchi, R. Fujioka and H. Tanaka, "Neural networks that
learn from fuzzy if-then rules", IEEE Transactions on Fuzzy
Systems, vol.1, no.2, pp.85-97, 1993.
[10] R. Sun, "Rule-based structure identification in an adaptive-
network-based fuzzy inference system", IEEE Transactions on
Fuzzy Systems, vol.2, no.1, pp.64-73, 1994.
[11] J.H. Holland, Adaptation in Natural and Artificial Systems,
Cambridge, Massachusetts: MIT Press, 1992.
[12] J.M. Zurada, Introduction to Artificial Neural Systems, St.
Paul, MN: West Publishing Company, 1992.
-
8/12/2019 Brasil 96
5/6
-
8/12/2019 Brasil 96
6/6