Knowledge-based systems Rozália Lakner University of Veszprém Department of Computer Science.

Knowledge-based systems

Rozália Lakner

University of Veszprém

Department of Computer Science

Engineering Application of AI - PhD Course - 2/57

An overview

Knowledge-based systems, expert systems structure, characteristics main components advantages, disadvantages

Base techniques of knowledge-based systems rule-based techniques inductive techniques hybrid techniques symbol-manipulation techniques case-based techniques (qualitative techniques, model-based techniques, temporal

reasoning techniques, neural networks)


Structure and characteristics 1

KBSs are computer systems contain stored knowledge solve problems like humans would

KBSs are AI programs with program structure of new type knowledge-base (rules, facts, meta-knowledge) inference engine (reasoning and search strategy for solution, other

services) characteristics of KBSs:

intelligent information processing systems representation of domain of interest symbolic representation problem solving by symbol-manipulation symbolic programs



Explanationsubsystem

Inferenceengine

Knowledgeacquisitionsubsystem

Case specificdatabase

Knowledgebase

Userinterface

Developer'sinterface

User

Knowledgeengineer


Main components 1

knowledge-base (KB) knowledge about the field of interest (in natural language-like formalism) symbolically described system-specification KNOWLEDGE-REPRESENTATION METHOD!

inference engine „engine” of problem solving (general problem solving knowledge) supporting the operation of the other components PROBLEM SOLVING METHOD!

case-specific database auxiliary component specific information (information from outside, initial data of the concrete

problem) information obtained during reasoning


Main components 2

explanation subsystem

explanation of system’ actions in case of user’ request

typical explanation facilities: explanation during problem solving:

WHY... (explanative reasoning, intelligent help, tracing information about the actual reasoning steps)

WHAT IF... (hypothetical reasoning, conditional assignment and its consequences, can be withdrawn)

WHAT IS ... (gleaning in knowledge-base and case-specific database) explanation after problem solving:

HOW ... (explanative reasoning, information about the way the result has been found)

WHY NOT ... (explanative reasoning, finding counter-examples) WHAT IS ... (gleaning in knowledge-base and case-specific database)


Main components 3

knowledge acquisition subsystem main tasks:

checking the syntax of knowledge elements checking the consistency of KB (verification, validation) knowledge extraction, building KB automatic logging and book-keeping of the changes of KB tracing facilities (handling breakpoints, automatic monitoring and reporting

the values of knowledge elements)

user interface ( user) dialogue on natural language (consultation/ suggestion)

specially intefaces database and other connections

developer interface ( knowledge engineer, human expert)


Main components 4

the main tasks of the knowledge engineer: knowledge acquisition and design of KBS: determination,

classification, refinement and formalization of methods, thumb-rules and procedures

selection of knowledge representation method and reasoning strategy

implementation of knowledge-based system verification and validation of KB KB maintenance


Expert Systems



expert systems knowledge-based systems employ expert’ knowledge applied in a narrow specific field solve difficult problems (must be demand on special knowledge) specialized human experts are needed experts must be agreed on the fundamental questions of

professional field learning examples and raw data are needed

expectations from an ES (like a human expert): make intelligent decision: offer intelligent advice and explanations question/ answer (“treated as an equal conversation partner”) explanation of questions acceptable advice even in case of uncertain situation



AI programs:intelligent problem solving tools

KBSs

AI programs with special program

structure separated knowledge base ESs

KBSs applied in a specific narrow field

AI programs


Expert systems


Expert system shells 1

„empty” ESs, contain all the active elements of an ES empty KB, powerful knowledge acquicition subsystem contain services for construction and operation of ES

independently of the field of interest support the development of rapid prototype and the

incremental construction examples: CLIPS, GoldWorks, G2, Level5


Expert system shells 2

Explanationsubsystem

Inferenceengine

Knowledgeacquisitionsubsystem

Case specificdatabase

Knowledgebase

Userinterface

Developer'sinterface

User

Knowledgeengineer


Advantages of KBSs and ESs

make up for shortage of experts, spread expert’ knowledge on available price (TROPICAID)

field of interest’ changes are well-tracked (R1) increase expert’ ability and efficiency preserve know-how can be developed systems unrealizabled with tradicional

technology (Buck Rogers) self-consistents in advising, equable in performance are available permanently able to work even with partial, non-complete data able to give expanation


Disadvantages of KBSs and ESs

their knowledge is from a narrow field, don’t know the limits

the answers are not always correct (advices have to be analysed!)

don’t have common sence (greatest restriction) all of the self-evident checking have to be defined (many exceptions increase the size of KB and the running time)


Base techniques of KBSs


Techniques of KBSs

based on the knowledge-representation methods and reasoning strategies applied in the implementation

rule-based techniques inductive techniques hybrid techniques symbol-manipulation techniques case-based techniques (qualitative techniques, model-based techniques, temporal

reasoning techniques, neural networks)


Rule-based techniques(a short review)


Reasoning with rules 1

knowledge-representation form: rule rule-base can be according to the structure of KB

simple/unstructured structured (contexts)

reasoning strategies: according to the control direction

data-driven/forward chaining goal-driven/backward chaining


Reasoning with rules 2

aim: proving a goal statement or achieving a goal state the reasoning algorithm:

pattern matching finding applicable rules (watching condition/conclusion part of rules) fireable rules conflict set (match condition/conclusion part of rules)

conflict resolution selecting the most appropriate rule from conflict set conflict resolution strategies

firing executing the selected rule new knowledge (new facts or new

subgoals to be proved) watching termination conditions restart of the cycle


Inductive techniques


Inductive reasoning

a type of machine learning technics inferring from individual cases to general information

given a collection of training examples (x, f(x)) return a function h that approximates f h is called hypothese

aim: finding the hypothese fits well on the training examples h is used for prediction the values of the unseen examples

x

f(x)

h y p o t h e s e s


Decision tree 1

one of the most known methods of inductive learning: learning decision trees

decision tree: simple representation for classifying examples

elements of the decision tree: nonleaf (internal) nodes are labelled with attributes (A) arcs out of a node are labelled with possible attribute values of A leaf nodes are labelled with classifications (Boolean values –

yes/no - in the simplest case)


Decision tree 2

nobluediesel3-6Japan3.

yesreddiesel6-10Japan2.

yeswhitediesel3-6Germany1.

Easy to sellColourEngineAgeCountry We want to classify new examples on property Easy to sell based on the examples’ Country, Age, Engine and Colour.

Country

Colouryes

yes no

Germany Japan

red blue


Decision tree 3

a decision tree under construction contains: nodes labelled with attributes nodes labelled with classifications (yes/no values) unlabelled nodes arcs labelled with attribute values outlet only form nodes labelled

with attributes every unlabelled nodes possess:

a subset of training examples eligible attributes


Decision tree 4

some questions about decision tree: Given some data (set of training examples and attributes), which

decision tree should be generated? A decision tree can represent any discrete function of the inputs.

Which trees are the best predictors of unseen data? You need a bias (preference for one hypothesis over another).

Example, prefer the smallest tree. Least depth? Fewest nodes?

How should you go about building a decision tree? The space of decision trees is too big for systematic search for the smallest decision tree.


Learning decision trees 1

learning decision tree ID3 algorithm:1. initially decision tree contains an unlabelled node with all of the

training examples and attributes

2. selecting an unlabelled node (n) with non-empty set of training examples (T) and non-empty set of attributes (A) if T is homogen class n leaf node, label with the classification otherwise

choosing the „best” attribute (B) from A extension of the tree with all of the possible attribute values of B (devide into

subclasses) classification of T to the children nodes according to the attribute values

(assign the elements of T to subclasses) continue with step 2.

building the tree top-down


Learning decision trees 2

how to choose the „best” attribute? attribute divides the examples into homogen classes otherwise attribute makes the most progress towards this

hill-climbing search on the space of decision trees searching for the smallest tree heuristics (maximum information gain)

information gain of an attribute test measures the difference between the original information

requirement and the new requirement (after the attribute test) information gain (G) it is based on information contents (entropy, E)

where: S: set of classified examples, A: attribute

S1, … , Sn: subsets of S according to A

E: entropy

)()(),(1

i

n

i

i SES

SSEASG

S

S

S

S

S

S

S

SSE

22 loglog)(


Learning decision trees 3Author Thread Length Reads

1 known new short true

2 unknown new long true

3 unknown old short false

4 known old short true

5 known new long true

6 known old long true

7 unknown old long false



10 unknown old long false


12 known old long true

13 known new long true


Using decision trees 1

major problem with using decision tree: overfitting occurs when there is a distinction in the tree that appears in the

training examples, but it doesn’t appear in the unseen examples handling overfitting:

restricting the splitting, so that you split only when the split is useful

allowing unrestricted splitting and pruning the resulting tree where it makes unwarranted distinctions: examples are devided into two sets: training set and test set constructing a decision tree with the training set examining all of the nodes with the test set: whether the subtree

under the node is replaceable with a leaf node


better: each one path (root leaf) on the decision tree is a rule

Using decision trees 2

supporting knowledge acquisition/ fast prototype-making (rule-based/ hybrid systems with inductive services)

each one row in the matrix of training examples is a rule

Author Thread Length Reads



…

IF (Author = known) and (Thread = new) and (Length = short)THEN (Reads = true)

IF (Author = unknown) and (Thread = new) and (Length = long)THEN (Reads = true)

…

IF (Author = known)THEN (Reads = true)

IF (Author = unknown) and (Thread = new) THEN (Reads = true)

IF (Author = unknown) and (Thread = old) THEN (Reads = false)


Main components of inductive systems

Knowledge representation:The matrix of training examples:

attributes, values

Reasoning and control:Algorithm, which constructs a

decision tree using the matrix oftraining examples and operates

the generated system.


Main steps of inductive systems

problem definition (knowledge representation): attributes (head of the matrix, generate coloumns, define object

classes) training examples (fill the raws of the matrix, define instances)

reasoning (generating a hypothese) checking the contradiction freeness of the training examples learning optimal decision tree (DT) knowledge base

control (operating the system) classification of user’ (unknown) examples (traversing DT) analysis of user’ examples (with the help of DT)


Hybrid techniques


Characteristics of hybrid systems

supporting various programming techniques: frame-based techniques rule-based techniques

data-driven reasoning goal-driven reasoning

inductive techniques realization:

using of object-oriented tools


Frames

knowledge-representation unit developed on epistemology foundations

formal tool using for description of structured objects or events or notions

characteristics of frames: a frame contains:

the name of the object/event its important properties (attributes) stored in slots (slot identifier,

type, value – it can be another frame) classes, subclasses, instances hierarchical structure (is_a, instance_of relations) inheritance (classes - subclasses, classes - instances) procedures controlled by events: daemons


Formalization of frames 1

directed graph

Person

Student

Peter

f_namel_name

is_a

instance_of

subjectsstatussubjects

Peter

Kis

f_name

l_name

subjects

Subject namepreconditions

ES

Expert_systems

AI

name

preconditions

instance_of

Teacher

is_a

Rozália

f_namel_namestatussubjects

instance_of


Formalization of frames 2 description in frame-based environment

frame person frame student frame subject

is_a class is_a person is_a class

f_name: subjects: collection_of subject name:

l_name: end precond: collection_of

end subject

end

frame Peter frame ES

instance_of student isnstance_of subject

f_name: Peter name: Expert_systems

l_name: Kis precond: AI

subjects: ES end

end


Formalization of frames 3 object-attribute-value triplets

<Peter, f_name, Peter>

<Peter, l_name, Kis>

<Peter, subjects, [ES]>

<ES, name, Expert_systems>

<ES, preconditions, [AI]>


Daemons 1

active elements of a frame system standard built-in procedures assigned to the attributes of the classes and instances automatically invoked in case of predefined changing in

the value of the slot usual daemons are as follows:

when-needed: describes the steps to be performed when the value of slot is read

when-changed: is invoked when the value of the slot is changed when-added: contains the actions to be performed when the slot

gets its first value when deleted: is executed when the value of the slot is deleted


Daemons 2

the executable part of the daemons is determined by the user or it may even be empty

execution is controlled by events daemons can invoke (call) each other via changing slot

values spread over and over the operation of a frame system is described in an

indirect way (embedded in the daemons) daemons can be used for restricted data-driven

reasoning


Daemons versus rules

Daemons Rules

Faster and more independent than rules. „Reason/action” is connected to the changes in values and the system’ responses. They act in autonomous way.

A rule is invoked by another rule or in case of presence of a certain data. The execution depends on the situation and cannot be seen in advance.

Less readable than rules.(daemons are defined on the implementation language of the given tool)

Easy to read.(symbolic formalism, natural-language like)

They handle the pre-defined changes of the given attribute-values.

The built-in knowledge of the rules steams freely to all of the rules.

The range of a deamon is bounded statically in advance. (more or less flexible)

The range of a rule is stand out dynamically in run-time. (flexible, creative problem solving)


Hybrid techniques

rules: used for description of heuristic knowledge frames: contains both descriptive and procedural

knowledge of the given objects/ events/ notions (altogether in one place! easy to read and modify, the effects of modifications can be held easily)

inference engine of hybrid techniques can contain: mechanisms insuring inheritance and handling of daemons mechanisms insuring message changing (object-oriented) data-driven and/or goal-driven reasoning mechanism can support the organization of rules and/or frames into

hierarchical modules can support making and using of meta-rules


Symbol-manipulation techniques


Programming languages of AI

high-level symbol-manipulation languages are used to support the implementation of AI methods LISP (LISt Processing)

based on the notion and operations of lists all of the problems can be described in the form of function calls

PROLOG (PROgramming in LOGic) high-level declarative language

define relationships between various entities with the help of logic special type of clause (A B1 … Bn): fact, rule, question

reasoning environment with a built-in inference engine answer to a question with the help of logical reasoning goal-driven (backward) reasoning


Comparison of symbol-manipulation and traditional techniques

Traditional programming languages LISP PROLOG

numeric calculus symbol-manipulation symbol-manipulation

Neumann-principle languagesconsist of sequence of commands executed in a predefined order

functional approachsequence of evaluation of function-expressions (-calculus)

relation approachbased on mathematical logic (predicate-calculus)

main elements: commands main elements: functions (procedures) main elements: predicates (relations among objects)

procedural (executing in a predefined order)

procedural declarative (defining only the description of the problem)

executing mechanism have to be defined by the programmer

executing mechanism have to be defined by the programmer

built-in executing mechanism (goal-driven reasoning with backtracking search strategy)

the structure of program and data is different

the sructure of program and data is the same (can produce, execute other programs, can modify themselves)

the sructure of program and data is the same (can produce, execute other programs, can modify themselves)

readability: LISP-like hard to read easy to read


Case-based techniques


Case-based reasoning (CBR) 1

basic assumption: like was the past like will be the future the „really” observation can be describe hard with the help of

classical rules it consists of interconnected relationships of more or less

generalized events idea:

solving problems based on solutions for similar problems solved in the past

requires storing, retrieving and adapting past solutions to similar problems


Case-based reasoning 2

solve a new problem by making an analogy to an old one and adapting its solution to the current situation

retrieving a case starts with a problem description and ends when a best matching case has been found

all case-based reasoning methods have in common the following process: identifying a set of relevant problem descriptors retrieve the most similar case (or cases) comparing the case to

the library of past cases reuse the retrieved case to try to solve the current problem revise and adapt the proposed solution if necessary retain the final solution as part of a new case


Case

a case represents specific knowledge in a particular context there are three major parts in any case:

a description of the problem/situation

the state of the world when the case is available solution

the chain of operators that were used to solve the problem (solving path)

outcome/consequence

the state of the world after the supervention of the case (description of the effect on the world)

in addition to specific cases, one also has to consider the case memory organisation


Case - indexing

the most important problem in CBR how do we remember when to retrieve what?

essentially, the indexing problem requires assigning labels to cases to designate the situations in which they are likely to be useful

indexing of cases - issues indexing should anticipate the vocabulary a retriever might use indexing has to be by concepts normally used to describe the

items being indexed indexing has to anticipate the circumstances in which a retriever

is likely to want to retrieve something


Main components of case-based systems 1

case-base (library of cases) tools for determining of key-elements of actual case and

for retrieving of most-similar cases for speeding of data-retrieval indexing for finding suitable cases pattern, similarity-estimation

tools for the solution’ adaptation according to the specialities of the new case finding the deviations, implementation of alterations in the

suggested solution (ex. null-adaptation, parameter adjustment) supervision (solution after the adaptation is suitable or not) learning (finding the reason of failure or enclosing the case to the

case-base)


Main components of case-based systems 2

case-base

retrieving(case-matching)

selecting

adapting

checking

new problem

solution

indexing

learning

similar cases

proposed solution


Advantages and disadvantages

advantages: case-base is more objective and formal than the expert’s

interpretation (knowledge of expert’s) knowledge are represented in an explicit way case can be defined for incomplete or badly-defined notions CBR is suitable for domains for which a proper, theoretical

foundations do not exist CBR is applicable in default of algorithmic method easy knowledge acquisition (get well during usage)

disadvantages: CBR solves only the problems covered by cases CBR might use a past case blindly without validating it in the new

situation solution is time-demanding (also in case of proper indexing)


Rule-based systems versus case-based systems

Rule-based systems Case-based systemsRule: symbolic pattern Case: collection of data, constants

Rule: individual unit, independent of the other rules, consistent piece of field of interest

Case: depends on the other cases (often overlap each other), individual unit of the field of interest

Retrieving rule: exact matching Retrieving case: partial matching

Using of rules: general iterativ cycle Using of cases: several steps (approximate retrieval, adaptation, refinement)

The model of the problem have to be developed (sometimes it is hard or impossible)

The model of the problem needn’t be developed

The knowledge-acquisition of field of interst is hard and time-demanding

The knowledge-acquisition of field of interst is limited to collecting and analysing the past cases

Development time is long Development time is short

Slow, handling of many data is difficult Many data is treatable with the useing of database –handling techniques

Enlargement is hard (the validation have to be repeated after enlargement)

Enlargement and development is easy.

Learning is not supported It is able to learn (preserving new cases)


Summary

K. M. Hangos, R. Lakner and M. Gerzson: Intelligent Control Systems. An Introduction with Examples. Kluwer Academic Publishers, 2001. Chapter 5.

D. Poole, A. Mackworth, R. Goebel: Computational Intelligence. A logical Approach. Oxford University Press, 1998. Chapter 6.

I. Futó (Ed.): Mesterséges intelligencia. Aula Kiadó, 1999. Chapter 12. (in hungarian)

References

Knowledge-based systems, expert systems Base techniques of knowledge-based systems

rule-based techniques inductive techniques hybrid techniques symbol-manipulation techniques case-based techniques

Knowledge-based systems Rozália Lakner University of Veszprém Department of Computer Science.

Documents

Transcript of Knowledge-based systems Rozália Lakner University of Veszprém Department of Computer Science.