Design of Web Agents Inspired by Brain Research

Maya Dimitrova, Hiroaki Wagatsuma * and Yoko Yamaguchi*

Institute of Control and System Research – Bulgarian Academy of Sciences

* Laboratory for Dynamics of Emergent Intelligence, RIKEN Brain Science Institute, Japan

30/09/2007 ADBIS'07 Varna 2

Content1. Introduction2. Information retrieval studies based on cognitive

heuristics (problem definition)3. Theta phase coding in the brain and episodic

memory 3.1. The Theta phase model3.2. Relevance to episodic/autobiographical memory4. Autobiographical memory in Web context: An

example5. Conclusions and future work

30/09/2007 ADBIS'07 Varna 3

Introduction

Modeling agents’ own experience – their reconstructed past - is essential to optimize agent behavior (i.e. agents for Web knowledge search, retrieval and presentation)

We propose to base this optimization on recent studies and models of the brain – focusing on the generality of processing in the animal and human, not on their difference

We have focused on “episodic” memory and the underlying hippocampal mechanisms of processing as closest to the “story-telling” approach of agent design

30/09/2007 ADBIS'07 Varna 4

Information retrieval studies based on cognitive heuristics

Problem statement:- Current Web page classifiers perform excessive amount of

processing to reach simple classificatory results- Whenever the classificatory task is multi-dimensional, agent

performance deteriorates dramatically- There is the database or storage problem – where models have

always been least analogous to human memory- Human memory processing is fast, heuristical and much more

efficient in the multi-dimensional case, than in the sequential processing case (contrary to the machine) etc. (recent finding)

- There is always a point in looking into recent brain studies – for their fast and accelerated development as empirically-based science

30/09/2007 ADBIS'07 Varna 5

Information retrieval studies based on cognitive heuristics (2)

The current study:- Presents the main ideas of the Theta phase precession

theory as proposing a memory model, different from the mainstream rate-coding models

- Operationalizes “episodic memory” for the purpose of design of memory-optimized Web search agent

- Provides an example to illustrate the described process in a human Web search process and some relevant conclusions for agent future development

30/09/2007 ADBIS'07 Varna 6

The agent (1)

In earlier work we designed a Java-servlet (a wrapper to a search engine) to diagnose the level of expertise and detail of a corpus of on-line available Web pages

Weak point – what to store in the database and in what form

Example of classified documents on one criterion

Example of classified documents on two criteria

Legend

- expert level

- detail ∆ - illustration Ω - explanation

..

.

Ω

..

.

∆

∆

∆

Ω

Results of the online classification on a specific criterion

Types (sample, test set) of documents

Document of type

Document of type

Document of type ∆

Document of type Ω

..

.

Classifier

(heuristics and methods for

classification)

Process of comparison for validity of the results

Adjustment of the parameters of the classifier

Lexical Database

30/09/2007 ADBIS'07 Varna 7

The agent (2)

It employed pattern classification based on syntactic analysis for diagnosing the amount of detail

And cognitive heuristics based on memory research for the expert level of the text

Example of classified documents on one criterion

Example of classified documents on two criteria

Legend

- expert level

- detail ∆ - illustration Ω - explanation

..

.

Ω

..

.

∆

∆

∆

Ω

Results of the online classification on a specific criterion

Types (sample, test set) of documents

Document of type

Document of type

Document of type ∆

Document of type Ω

..

.

Classifier

(heuristics and methods for

classification)

Process of comparison for validity of the results

Adjustment of the parameters of the classifier

Lexical Database

30/09/2007 ADBIS'07 Varna 8

The agent (3)

In following history of interaction, optimization is needed much the way natural cognitive systems perform - learn, abstract, remember and forget

We use the ‘autobiographical memory’ metaphor to apply to such kind of Web agents and the Theta phase precession theory to model it

Example of classified documents on one criterion

Example of classified documents on two criteria

Legend

- expert level

- detail ∆ - illustration Ω - explanation

..

.

Ω

..

.

∆

∆

∆

Ω

Results of the online classification on a specific criterion

Types (sample, test set) of documents

Document of type

Document of type

Document of type ∆

Document of type Ω

..

.

Classifier

(heuristics and methods for

classification)

Process of comparison for validity of the results

Adjustment of the parameters of the classifier

Lexical Database

30/09/2007 ADBIS'07 Varna 9

Current MATLAB GUI versionof the agent interface

The difference of the MatLab version from the Java version is that the Java-servlet processed also the hyperlink structure of the Web page

The current version is simplified for algorithmic and modeling reasons

30/09/2007 ADBIS'07 Varna 10

Assumptions of the current study:

Assumptions:- User interaction with the present-day Web is an

autobiographical experience (remembering sequences of events like personal-life episodes)

- Memorizing events in a human-like memory system undergoes constant synthesis as any other learning process and therefore can be implemented in autonomous agent on the Web

30/09/2007 ADBIS'07 Varna 11

Operational definitions in the current study:Definition of Episodic Memory: - The system of processing episodes of Web search and

transforming them into autobiographical events of successful/unsuccessful Web navigation to reach the needed goal

- Every Web page viewed by the user is a complex perceptual and semantic stimulus, i.e. an episode of Web search

- What is stored in episodic memory from the rapid viewing of a succession of pages is a set of fragments of the search episode rather than the entire pages and their exact order of viewing

30/09/2007 ADBIS'07 Varna 12

Hypotheses of the current study:

- The theoretical hypothesis is that episodic memory extracts meaningful cues from the fragments of the page-retrieval episodes, rather than random perceptual or semantic cues

- “Meaningful” we define as having anticipatory value (i.e. useful in the future)

- The tested hypothesis is that the mechanisms of event synthesis from fragments of page-retrieval episodes are the same in personally-significant and neutral situations, that is – universal for learning per se

- These effects are reproduced by both the human and the agent in the current study

30/09/2007 ADBIS'07 Varna 13

Theta phase coding in the brain and episodic memory

Theta phase theory models rhythmic patterns of activity of assemblies of neurons in the brain and their role in complex cognitive tasks (theta rhythm 4-10 Hz)

Theta phase modulation is associated with learning and memory in rats and humans A mechanism of pattern compression has been observed in hippocampal neuron assemblies, which is called theta phase precession

Modelling hippocampal nerve cirquit by the Theta phase precession theory (Yamaguchi, Yamaguchi et al.) reinstates complex learning effects observed in human episodic memory like object-context integration, single-trial learning and time-space contextualization

30/09/2007 ADBIS'07 Varna 14

Neuro-realistic model of episodic encoding and retrieval

(a) Main modules of the hippocampus – ECII, ECIII, CA3, CA1, DG

(b) Entorhinal input, emerging from the sensory system and representing the vector time series produced by behavior

(c) when the rat runs to the right, the phase shift in firing within each theta rhythm cycle occurs in place cells 1 to 4, which are activated sequentially

30/09/2007 ADBIS'07 Varna 15

Properties of the phase model

(The recurrent connections in the CA3 layer of the hippocampus form asymmetric connections so that units with prior activations are connected to those with subsequent activations

These activations are a consequence of the coincidence between their phase differences and the time window of the asymmetric synaptic plasticity

30/09/2007 ADBIS'07 Varna 16

Model of hippocampal nerve circuit

(a) mechanism of theta phase coding (b) phase shift in firing within each theta rhythm cycle

30/09/2007 ADBIS'07 Varna 17

Relevance to modeling memory: Partial cuing triggers pattern completion in the CA3 and its

transmission to the CA1 and EC. Thus a stored pattern in the hippocampus can be sent to the cortical area to replay the input pattern

The temporal difference between encoding and retrieval in the network parallels behavioral tests

The retrieval is ten times faster than the behavioral input in the encoding stage and the retrieval time scale is similar to that of the compressed pattern of theta phase precession

The retrieved activity can cause further synaptic plasticity in other cortical areas for memory consolidation or memory transfer from the hippocampus to the cortex

30/09/2007 ADBIS'07 Varna 18

Importance of theta phase precession theory to the present study:

The emergence of rhythmic patterns in the absence of memory to cause its formation is a plausible candidate for a neural correlate of the formation of complex and durable cognitive effects governing future behavior i.e. autobiographical memories

We have aimed to give a relevant example to illustrate these effects

30/09/2007 ADBIS'07 Varna 19

How to find an Art Gallery on the Web with incomplete initial information?

Participants:- Motivated- NeutralSearch goal:- Example of new concept learning, similar to

artificial concept learning: “Kurenai Kai”Search task:- Google search from incomplete initial information

(The name of the gallery “Kurenai Kai” not known apriory)

30/09/2007 ADBIS'07 Varna 20

Results and discussion:

User Searches Time Random Meaningful Approximate

Motivated 23 3-5 30 min Neutral 19 3-5 30 min

Table 1. User performance on Web search with incomplete initial information about the search goal

Our study of Web search has replicated the mechanism of performance of the theta phase precession model :- The Web paths resemble random search until a meaningful path is composed-We observe that meaningful paths are composed, rather than encountered

30/09/2007 ADBIS'07 Varna 21

Web search: “Kurenai Kai”

Left path represents the nature of the search with incomplete initial information Right path represents the shortest path with well-remembered concept term

30/09/2007 ADBIS'07 Varna 22

Remembering meaningful fragments of search episodes

Left path represents episodic memory guided retrieval based on a set of meaningful cues and “insight”The pages to the right represent the mappings to the respective initial cues (none of them are identical)

30/09/2007 ADBIS'07 Varna 23

Conclusions from the Web search study:

Users base subsequent retrieval of the search path on remembering the ‘events’ of their search, which are the meaningful fragments of knowledge inside the Web pages, not the exact ‘page-retrieval’ episodes

We propose that the autonomous Web agent behaves in analogy with the event-based (autobiographical) strategy that Web users are applying for retrieval of useful information, which is some state of knowledge with anticipatory value (useful in the future)

Next will be tests of the agent for optimized storage of fragments of knowledge contained in the Web page, rather than of snapshots of sequences of Web pages

30/09/2007 ADBIS'07 Varna 24

Fragment extraction and recombination

of textual elements as a

process of encoding

search events

Agent reinstatement of the search task

30/09/2007 ADBIS'07 Varna 25

Temporal evolution of episodic events by the neuro-cognitive agent

30/09/2007 ADBIS'07 Varna 26

Conclusions and Future WorkWe presented one approach towards the design of

Web agents based on recent knowledge from brain research

We have tried to demonstrate how the theoretical framework of theta phase coding endows real-time process of memory formation of Web search experiences and the retrieval process of effective routes to the targets through the experienced sites

The proposed approach focuses on bridging the gap between neuroscience and Web technologies

It is currently being applied to retrieval of cardiological knowledge from the Web

30/09/2007 ADBIS'07 Varna 27

AcknowledgementThis work is supported by:

NSRF of the Bulgarian Ministry of Education and Science as part of the Research Project

MI - 1509/2005

”Multimodal User and Sensor Interface in a Computer-Aided System for Cardiological

Diagnosis and Intervention”

30/09/2007 ADBIS'07 Varna 28

Thank you for your attention