1

An Evolutional Network Architecture for Developmental Knowledge Bases

Ashraf Saad* and A. R. M. Zaghloul** *Associate Professor, ** Professor

Computer Engineering Georgia Institute of Technology

{ashraf.saad,ar.zaghloul}@gtrep.gatech.edu

Abstract

We present a network architecture that supports de-velopmental knowledge bases. Knowledge representation in our architecture is based on knowledge quanta. The network architecture described herein is evolutional in nature and utilizes spreading activation algorithms for network formation. In doing so, our research is seeking answers to fundamental questions pertinent to the way people learn as well.

1. Introduction and Motivation

Mental development of individuals is a dynamic life-long process. In order to emulate this process in engi-neered systems, one must first understand the innately dynamic and adaptive structure of the human brain.

1.1. Neural Connectivity of the Brain

Neuroscience studies of the human brain have revealed that information is stored and processed by intricate net-works of nerve cells: the neurons, interconnected via biological organs: the synapses [1]. It is estimated that the number of neurons in the human brain is on the order of 1011 interconnected by synapses on the order of 1015 [1-3]. Synapses have been shown to be excitatory or inhibitory in nature [2]. Neuroscientists have also shown that the formation of synapses, or “wiring” of the brain, is shaped by experience and learning during the developmental years of the brain – which can be up to 8-10 years from birth for some parts of the brain [3].

1.2. Neural Modeling

Information processing in the brain is carried out by the collective activity of neurons and the propagation of signals over the synapses connecting them. Computing researchers have capitalized on these findings by creating computational networks that attempt to mimic the infor-mation processing and cognitive abilities of the

brain. Two main such computational networks have been proposed to-date, neural and semantic networks:

a. Neural networks, also known as Artificial Neural

Networks (ANN). To-date, three main models dominate the neural network research: the state transfer models, the signal transfer models, and competitive learning [2]. Neural networks appli-cations are based on all three models, they include: pattern recognition, image and signal processing, classification, and control, just to name a few.

b. Semantic networks were developed to represent

the organization of someone's ideas or the under-lying organization of ideas in a content domain [4]. Semantic nets are spatial representations of ideas and their interrelationships. Semantic net-works have been used at the basis of concept maps. A concept map is composed of nodes, that represent any concept or idea, that are intercon-nected via links, that represent any statements of relationships among the nodes [5].

1.3. The Science of Learning and Constructivism

The focus of the new science of learning is on the processes of knowing that lead to understanding [3]. Hu-mans are viewed as goal-directed cognitive agents who actively seek information. A constructivist approach to learning assumes that all knowledge is constructed from previous knowledge. Such an approach can be successful in the knowledge formation when previous knowledge of the individual is used as the basis or starting point for new instruction [3].

These findings are at the basis of our evolutional net-

work architecture for knowledge bases that can support autonomous mental development. This architecture and the research issues it poses are described next.

Proceedings of the 2nd International Conference on Development and Learning (ICDL�02) 0-7695-1459-6/02 $17.00 © 2002 IEEE

2

2. Evolutional Network of Knowledge Quanta

Knowledge representation plays a central role in men-tal development. In order to do so, knowledge representa-tion must support: a. a constructivist approach to knowledge formation, b. dynamic additions and deletions of knowledge, c. the ability to reuse knowledge in multiple contexts, d. the ability to represent abstractions, and e. the ability to represent expertise.

Neither a neural nor a semantic network is able to sup-

port these requirements. In order to create a manageable knowledge base, from a computational standpoint, one has to specify some structure for the knowledge represen-tation and formation. The proposed structure, however, must have the structural flexibility needed to support the aforementioned requirements.

We have therefore adopted a network structure for the

representation of knowledge that is comprised of two basic elements, nodes and links: i. Nodes: a node should represent a knowledge quan-

tum (KQ), or a certain abstraction, that is fundamen-tally derived to represent a collection of knowledge quanta nodes. In order to better visualize abstraction in our network model, we use a layered approach, where: a. nodes contained in the bottom-most layer are those corresponding to the fundamental knowledge quanta of an individual’s knowledge base at any given point in time, and b. nodes contained in the subsequent higher layers represent higher levels of abstraction. Each node in a higher layer is an aggregation, based on a certain metric, of two or more nodes from a layer below it.

ii. Links: we specify two types of links in our network model: a. Intra-layer Link: is a link that connects two nodes within any given layer. Such a link represents the causality relationship between the two nodes from the knowledge acquisition standpoint; i.e.: which node(s) must be learned prior to others. Intra-layer links can be one of three types: one-to-one, many-to-one, and one-to-many. b. Inter-layer Link: is a link that connects two or more nodes from one layer to a single abstraction node in a layer above the layer where they reside. An inter-link layer can therefore only be a many-to-one link (from many nodes in one layer to a single node in the layer above it).

An example of such layered network structure – lim-

ited to the first three layers– is illustrated in Figure 2.

A fundamental principle underlying our network archi-tecture is that of local connectionism: each node is only aware of its rules of interactions with possible neighbor-ing nodes, whether these nodes lie within the same layer or in other layers. These rules are at the basis of the crea-tion of inter- and intra-layer links for each node. The structure of the proposed network therefore differs from semantic networks and neural networks in two fundamen-tal ways: i. Network formation is evolutional and based on a

constructivist approach that involves the learner. In Figure 2, the nodes and interconnections within layer 1 must represent the knowledge quanta and their interconnections at a fundamental level from the user perspective – keeping in mind that more fundamental layers of knowledge may still exist below it.

ii. Neural networks model are yet to develop further in the hierarchical direction before they can cope with matching conditions that relate to many levels of abstraction [2]. In our work we intend to specify the conditions needed for the automatic generation of such levels of abstraction: Knowledge Quanta Abstraction, or KQA; e.g., KQ1-4 and KQ4-5 in Figure 2. Such vertical aggregation may be achievable, for instance, using hierarchical, clustering/chunking or relational structures.

At any given time, the nodes and links comprising the

network represent the current state of the knowledge base. The evolutional nature of the network, leading to the creation of new nodes and links, or pruning of existing ones, is realized via learning algorithms, as described next.

3. Learning Algorithms

The creation of inter- and intra-layer links leading to formation of the knowledge base must be achieved using learning algorithms. The algorithms needed must lend themselves to be implemented using a local connectionism approach, that is: an algorithm that resides locally at the node level and enables the dynamic creation and maintenance of the network. A viable approach to develop such an algorithm is by using spreading activation.

3.1. Spreading Activation

Spreading Activation is a well-studied technique by cognitive scientists seeking to understand the learning processes that take place to form learning networks. The origin of these studies lies in the study of how [biological] learning networks are actually formed [7]. The origins of spreading activation can be traced back to early work on associative memory at the end of the nineteenth century.

Proceedings of the 2nd International Conference on Development and Learning (ICDL�02) 0-7695-1459-6/02 $17.00 © 2002 IEEE

3

Researchers in the field of Artificial Intelligence have emulated this process in a variety of ways and applied spreading activation over a semantic network to solve several important problems in Artificial Intelligence. Such an approach has also been applied for machine learning in engineered systems, as in the field of robotics [8]. Some of the main characteristics of spreading activation that are important to this work include: a. Spreading activation subsumes both marker passing

and local connectionism. b. Since spreading activation is a distributed process,

information must be localized at eachnode in the network.

c. Spreading activation is therefore the passing of messages – of arbitrary complexity – between concurrent objects – of arbitrary complexity.

The successful application of spreading activation to

robotic task planning [3] is of particular relevance and application to our work. This novel approach to task planning utilizes a decision-theoretic framework where action selection is driven by a spreading activation mechanism over a probabilistic network that encodes the domain knowledge. The following are the main characteristics of this approach that are important to our network architecture: a. It utilizes a semantic network that comprises both

proposition and action nodes that are interconnected via probabilistic links; i.e. the weight associated with each link defines the correlation between the success of an action and its preconditions.

b. Spreading activation for action selection utilizes both forward propagation from the current state as well as backward propagation of the goal utility. Such a propagation mechanism is inspired by neural network models.

c. Since spreading activation is done over a probabilistic network, action selection therefore leads to decision-theoretic planning: the utility received by an action represents the product of the probabilities of success of the subsequent actions in the path leading to the goal proposition.

d. The tradeoff between the reliability of an action versus its cost is also explicitly addressed by incorporating the cost (expressed in terms of any resource of interest) of the action as well as its probability of success.

3.5.2 Spreading Activation over a Network Model

of Knowledge Quanta

We propose using a spreading activation-based algo-rithm to automatically generate the inter- and intra-layer links in our network of knowledge quanta. In order to do so, we have to identify the mechanism needed to match

the conditions of activation of a knowledge quantum with the conditions being set by other knowledge quanta, as illustrated in Figure 2. Some of the main characteristics of this mechanism include: a. Forward propagation proceeds from the state of the

knowledge network at any given point in time. b. Backward propagation proceeds from the desired

state of knowledge. c. Spreading activation persists until the knowledge

state equals the desired knowledge state in terms of both quanta and links in the network.

d. The weights of the links between knowledge quanta and propositions can be learned and adaptively fine-tuned based on the success of the network in fulfilling the desired learning objectives for each learner.

4. Fundamental Research Issues

The proposed network structure of knowledge quanta raises is unique in two fundamental ways: i. The knowledge network is dynamically created as a

result of interaction with the environment and continuously evolving to account for three fundamental phenomena: 1. The acquisition of new knowledge. This results

in the creation of new knowledge quanta nodes and their interconnection to existing nodes.

2. The hierarchical representation of knowledge abstraction via nodes in higher network layers (ref. Figure 2).

3. The development of expertise as a result of knowledge quanta creation, layered representation of knowledge abstraction, and the resulting interconnections.

ii. At any given point in time, the state of the network model reflects what the current knowledge base. This state constitutes the basis for subsequent evolution through the addition of new knowledge quanta nodes, or deletions of old ones, and their interconnection to existing nodes via new links.

Some of the fundamental issues that are being

addressed by our research fall under the following categories:

4.1. Nodes

a. What is a knowledge quantum? However, the answer that we seek in our work is one that is fundamental to the representation of knowledge.

b. In addition to being a fundamental representation of knowledge, a knowledge quantum should be endowed with the ability to be interconnected to other quanta in multiple ways.

Proceedings of the 2nd International Conference on Development and Learning (ICDL�02) 0-7695-1459-6/02 $17.00 © 2002 IEEE

4

c. The validation of a knowledge quantum must be based on a rigorous mathematical-logical foundation.

d. What kind of metadata is necessary for a knowledge quantum to allow it to be comprised in a knowledge quanta network model that utilizes the spreading activation mechanism?

4.2. Links

a. The links connecting knowledge quanta are probabilistic in nature and weighted – not deterministic. They are to be weighted using normalized weights/probabilities to reflect the relative strength/weakness of any given link – relative to other links in the same layer.

b. The weight/probability of an inter- and intra-layer link is envisaged as a measure of its utility/relevance for knowledge formation.

c. Inter- and intra-layer links are envisaged to capture the evolution of knowledge abstraction.

d. The validation of a link must be based on a rigorous mathematical-logical foundation.

e. What information should be endowed in an inter- or intra-layer link to allow it to be comprised in a knowledge quanta network model that utilizes a spreading activation mechanism?

4.3. Layers

Fundamental questions pertinent to the nature of layers include: a. What are the metrics that will govern binding

knowledge quanta within a single layer? b. What are the rules that will trigger the creation of an

inter-layer versus an intra-layer link?

4.4. The Network

The evolutional nature of the network poses some quite interesting challenges, including: a. How to begin the network construction? b. What rules govern the evolutional behavior of the

network? c. What measures of optimization are to be used, and if

so, relative to which measures of performance? d. Would the evolutional network lend itself to self-

organization? e. What kind of information-theoretic foundation can be

developed? f. How to detect preconceptions in knowledge

formation and remedy them?

4.5. Knowledge Abstraction

Fundamental questions pertinent to Knowledge abstraction include:

a. How will horizontal and vertical aggregation of knowledge quanta be developed?

b. Will that require hierarchical or clustering/chunking techniques, or a combination thereof?

4.6. Development of Expertise

Fundamental questions pertinent to the development of expertise include: a. How will the network account for the development of

expertise? b. What role does the development of multi-layered

knowledge abstraction play in the development of expertise?

c. Will the development of expertise lead to any pruning of knowledge quanta and the links connecting them?

5. Concluding Remarks

These are just but a few of the issues addressed by our research. While applicable to computational mental de-velopment, these issues are also fundamental to the way people learn. They cannot be addressed without undertak-ing the work presented in this paper and related efforts. However, once knowledge is rooted on the fundamental premise presented herein, it can be readily applied to multiple learning and disciplinary contexts, and to realiz-ing the goal of computational autonomous mental devel-opment [9,10].

References

[1] Braitenberg, V., "Brain size and number of neurons: an exercise in synthetic neuroanatomy", Journal of Computa-tional Neuroscience, vol.10, no.1, p.71-79, 2001.

[2] Teuvo Kohonen, “Self-Organizing Maps”, Third Edition, Springer Series in Information Sciences, Springer-Verlag Publishers, 2001.

[3] National Research Council Committee on Developments in the Science of Learning, “How People Learn: Brain, Mind, Experience, and School: Expanded Edition”, John D. Bran-ford, Ann L. Brown, and Rodney R. Cocking, Editors, Na-tional Academy Press, 2000.

[4] Jonassen, D.H., Beissner, K., & Yacci, M.A., “Structural knowledge: Techniques for conveying, assessing, and ac-quiring structural knowledge”, Lawrence Erlbaum Associ-ates, 1993.

[5] Lawson, M. J., “Concept Mapping”, in T. Husén & T. N. Postlethwaite (Eds.), The international encyclopedia of education, 2nd ed., Vol. 2, pp. 1026-1031, Elsevier Science, 1994.

[6] A. Saad, “A Multi-Agent Spreading Activation Network Model for Online Learning Objects,” MultiAgent-Based Learning Environments Workshop, 10th International Con-ference on Artificial Intelligence in Education, San Anto-nio, TX, May 19, 2001, pp. 55-59 – workshop proceedings.

Proceedings of the 2nd International Conference on Development and Learning (ICDL�02) 0-7695-1459-6/02 $17.00 © 2002 IEEE

5

[7] Fomin, T.; Szepesvari, C.; Lorincz, A. , “Self-organizing neurocontrol”, 1994 IEEE International Conference on Neural Networks, IEEE World Congress on Computational Intelligence, vol. 5, pp.: 2777-2780, 1994.

[8] S. Bagchi, G. Biswas, and K. Kawamura, “Task Planning under Uncertainty using a Spreading Activation Network”, IEEE Transactions on Systems, Man and Cybernetics, Vol. 30, No. 6, November 2000, pp. 639-650.

[9] J. Weng, J. McClelland, A. Pentland, O. Sporns, I. Stock-man, M. Sur and E. Thelen, “Computational Autonomous Mental Development”, Science Magazine, vol. 291, no. 5504, January 2001, pp. 599-600.

[10] Proceedings of Workshop on Development and Learning, 5-7 April 2000, Michigan State University, East Lansing, MI, www.cse.msu.edu/dl/.

6. Biographical Sketches

Ashraf Saad is associate professor of computer engi-neering at the Georgia Institute of Technology. He ob-tained his PhD degree in Electrical and Computer Engi-neering in 1996 from Vanderbilt University, where he was a member of the Intelligent Robotics Laboratory (1991-1996). Between 1996 and 2000, he was assistant then associate professor at the University of Cincinnati. He is currently co-Principal Investigator (2001-02) on the NSF DUE ATE project supported by grant no. 9950029 (PI 1999-2000), and co-PI on the NSF DUE CCLI project supported by grant no 0196015. His research interests include adaptive decentralized systems, mobile comput-ing, and novel learning environments. He is a member of the IEEE, ACM, ASEE and AACE professional societies.

A. R. M. Zaghloul is a professor of electrical and computer engineering at the Georgia Institute of Technol-ogy. In addition to teaching and research, he served as an electrical engineering consultant, computer consultant, educational consultant, and expert witness. He received his PhD from the University of Nebraska-Lincoln in 1975. In 1982 he received a Higher Diploma in Educational Technology from Dundee College of Technology, Scot-land. His research interests include semiconductor materi-als characterization and novel cognitive-enabling educa-tional environments. He is a senior member of the IEEE, a member of IEE, OSA, and ASEE professional societies. He is a chartered engineer in the UK and a European Engineer. He is biographically listed in “Outstanding Intellectuals of the 20th Century,” “Outstanding People of the 20th Century,” “Dictionary of International Biography, Landmark Millennium Edition,” “Who’s Who in Science and Engineering,” “Who’s Who in Finance and Industry,” “Who’s Who in The World,” and “International Who’s Who in Optical Sciences and Engineering.”

Figure 1. Layered Network Model for Knowledge Representation

KQ = Knowledge Quantum C = [Activation] Condition

Figure 2. Knowledge Quanta Network [Single Layer]

KQ

KQ

KQ

C

C

KQ

KQ

KQ1

KQ2

KQ3

KQ5

KQ6

KQA1-4

KQA4-5

KQ4

Layer 2: Abstraction Level

Layer 1: Knowledge Quanta

Layer 3: Abstraction Level

Proceedings of the 2nd International Conference on Development and Learning (ICDL�02) 0-7695-1459-6/02 $17.00 © 2002 IEEE