Scene Based Reasoning Cognitive Architecture Frank Bergmann, [email protected] Brian Fenton,...

Scene Based ReasoningCognitive Architecture

Frank Bergmann, [email protected] Fenton, [email protected]://tinycog.sourceforge.net/

Reasoning

Scenes

Planner

Pla

nR

ecog

nitio

n

Pla

n

Exe

cutio

n

Persistent Plans

3D

Rec

onst

ruct

ion

Persistent Plans

Attention Subsystem

Persistent Plans

Episodic Memory

mailto:[email protected]

mailto:[email protected]

http://tinycog.sourceforge.net/



(cc) Frank Bergmann, Brian Fenton, http://tinycog.sourceforge.net/ 2

Problems Addressed We propose an integrated architecture for

implementing a number of “self-models”. We provide a model for talking about

modalities (auxiliary verbs) without the use of higher-order or modal logics.

Approach Use Thomas Metzinger’s “Self-Model

Theory of Subjectivity” as a kind of requirement statement.

Extend/generalize existing knowledge representation and reasoning formalisms to accommodate self-models.

Implementation Status Plays “Towers of Hanoi” Working on a “Stone age prey/hunter” sandbox http://tinycog.sourceforge.net/

Problems Addressed & Approach

Reasoning

Scenes

Planner

Pla

nR

ecog

nitio

n

Pla

n E

xecu

tion

Persistent Plans

3D

Rec

onst

ruct

ion

Persistent Plans

Attention Subsystem

Persistent Plans

Episodic Memory


Physical Volume Self-Model “This is the room I’m in, this is my leg.“Identify the 3D volume of the robot representing SBR and model its capabilities

Capabilities Self-Model „I am good at this.“Performance statistics of task decompositions

Planning Self-Model „I usually hit the goal in 30% of all cases.“Performance statistics of plans performed

Intention Self-Model ”I currently try to do this.“Introspection into the goals currently pursued

Goal Self-Model “I would like to be to do this.“Introspection into active “Persistent Goals”

Social Self-Model “Other group members respect me.“Role of self in group activities.

Behavioral Self-Model “I usually react like this.“Episodic Memory recordings of past SBR actions

Emotional Self-Model “I am happy to hear the news.”Introspection into current emotions and historic model of emotions

Historical Self-Model “I used to do a lot of this.”Episodic Memory recorded SBR past actions.

Terminological Self-Model “I know that a penguin is not a bird.”Export the Description Logic TBox into a 2d scene diagram.

Flow of Mental Events Self-Model “I just had an idea how to solve plan X”Short-term memory of “mental events”

Different Levels of Self-Models

Physical

Planning

Social

Internal


Scene Based Reasoning “Core” Architecture

Reasoning

Scenes

Planner

Pla

nR

ecog

nitio

n

Pla

n

Exe

cutio

n

Persistent Plans

3D

Rec

onst

ruct

ion

Persistent Plans

Attention Subsystem

Persistent Plans

Episodic Memory

1

2

3

20+ subsystems Explained at http://tinycog.sourceforge.net


SBR „Core“

Reasoning

„Scenes“

Planner

„Scene“

„Plan“ We use “Scenes” as a unified representation of internal and world states and as “situations” for the planner.– 3D Scene Graph level – suitable for 3D

reconstruction and low-level planning. Reasoning using physics simulation.Reasoning using “diagram reasoning”.

– Semantic Network level – “1st Mind”, provides “concepts” and “roles” as an abstraction from object and their attributes.Reasoning using Description Logic reasoning

– 2D Graph level – suitable to represent 2D maps, meta-representations of internal data-structure.

– “Situation” level – Scenes act as world states and internal states to the planner, adding “dynamics”

Eat DinnerStart Goal

. . .

. . .

. . .

Get pizzafrom Fridge

CleanUpEat

ObtainFood

Table1 Human1

. . .

. . .

Type: TableColor: greenPosition: ...Size: 1 x 1...

Type: HumanGender: malePosition: ......

in-front-of

Task/Action

Scene

1

2

3

1



SBR „Core“

Reasoning

„Scenes“

Planner

„Scene“

„Plan“ We use “Scenes” as a unified representation of internal and world states and as “situations” for the planner.

Reasoning is performed by several specialized subsystems. – We use Description Logic as a “First

Mind” convenient “knowledge assembler“ to classify objects and their attributes into concepts and roles, but not for “higher-level” reasoning.

– 2D graph reasoning is a special type of planning.

– “Two Minds”: Planning together with the attention, motivational and “persistent goal” subsystems forms a 2nd reasoning system in addition to DL


. . .

. . .

. . .


CleanUpEat

ObtainFood

Table1 Human1

. . .

. . .



in-front-of

Task/Action

Scene

1

3

1


2

2


SBR „Core“

Reasoning

„Scenes“

Planner

„Scene“

„Plan“ We use “Scenes” as a unified

representation of internal and world states and as “situations” for the planner.

Reasoning is performed by several specialized subsystems.

The „Planner“ uses Scenes as situations and goals.

– “Task decompositions” (HTNs) provide a middle ground between STRIPS style planning and procedural execution and are easy to learn.

– Actions may have multiple deterministic outcomes – statistics about actions are collected in the episodic memory.

– Physics simulations cover a range of planning capabilities that are difficult to handle using FOL and derivates

– A “simulation subsystem” allows for what-if analysis of plans


. . .

. . .

. . .


CleanUpEat

ObtainFood

Table1 Human1

. . .

. . .



in-front-of

Task/Action

Scene

2

1

3

1

2

3



Attention Subsystem controls sensor focus Persistent Plans for setting plan priorities Episodic Memory stores plans indexed by content Plan Recognition allows for 1-shot learning 20+ subsystems defined at http://tinycog.sourceforge.net

Reasoning

Scenes

Planner

Pla

nR

ecog

nitio

n

Pla

n

Exe

cutio

n

Persistent Plans

3D

Rec

onst

ruct

ion

Persistent Plans

Attention Subsystem

Persistent Goal Hierarchy

Episodic Memory

Architecture Summary and “Two Minds”


Physical Volume Self-Model “This is my leg.“Identify the 3D volume of the robot representing SBR and model its capabilities










Flow of “Thoughts” Self-Model “I was surprised by this event”Short-term memory (“time-line”) of “mental events”

Self-Models and Self-Referentiality


Examples of Scenes representing Self-Models: Plan & DL-TBox

Modalities examples:– „My goal is ...“– „I believe that...“

Explicit representation of data-structures

No need to use modal or higher order logic

Apply the same reasoning engine as to object-level data

DinnerStart Goal

. . .

. . .

. . .


CleanUpEat

ObtainFood

. . .

. . .

Bird

Canary

Penguin

Animal

Fish

has skin eats breathes moves

!can fly can swim

can sing is yellow

has wings can fly has

feathers has fins can swim has gills


Physical Volume Self-Model “This is my leg.“Identify the 3D volume of the robot representing SBR and model its capabilities










Flow of “Thoughts” Self-Model “I was surprised by this event”Short-term memory (“time-line”) of “mental events”

Self-Models and Self-Referentiality

Physical

Planning

Social

Internal

Scene Based Reasoning Cognitive Architecture Frank Bergmann, [email protected] Brian Fenton,...

Documents

Transcript of Scene Based Reasoning Cognitive Architecture Frank Bergmann, [email protected] Brian Fenton,...