Erich Rome Robot Control Architectures Department

EU Project MACSMulti-sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

Erich RomeRobot Control Architectures Department

Cognitive Systems Kick-Off Meeting, Bled, Oct 28–30, 2004

Erich Rome Cognitive Systems Kick-Off Meeting Oct 28–30, 2004Bled, Slovenia

MACS Project Overview

Sections

a. MACS Facts

b. Affordance-based Robot Control

c. The Vision & the Key Objectives

d. Steps to Achieve the Objectives

e. The Key Milestones

f. Next Steps: Some Details


a. MACS: Facts

Project type: STReP

Grant no.: FP6-004381

Project start: September 1, 2004

Duration: 3 years

Kick-off meeting: Sankt Augustin, September 9–10, 2004

Web site: www.macs-eu.org

Consortium: 5 participants


a. MACS: Facts

Participants and Competences

1 FhG/AIS (Coord.) Fraunhofer Institute for Autonomous Intelligent Systems, Sankt Augustin, D Erich Rome Robot control architectures, robot & sensors & autonomous systems design, biologically inspired robot vision

2 JR_DIB JOANNEUM RESEARCH Forschungsgesellschaft mbH, Graz, A Lucas Paletta Computer vision, ANN-based sensorimotor learning, mobile mapping

3 LiU-IDA Linköpings Universitet, AI & Integrated Computer Systems Division, S Patrick Doherty Autonomous systems, knowledge representation, AI planning

4 METU-KOVAN Middle East Technical University, Ankara, TR Erol Sahin Evolutionary & swarm robotics, physics-based modelling & simulation, distributed computing environments

5 OFAI Österreichische Studiengesellschaft für Kybernetik, Vienna, A Georg Dorffner Cognitive modelling & neuroscience, symbol grounding, AI learning


b. MACS: Affordance-based Robot Control

Some Objectives of the Cognitive Systems Call:

• To construct embodied systems that can perceive,

understand, and interact with their environment

while performing goal-directed tasks.

• Methodologies for the construction of robust and

adaptive cognitive systems integrating perception,

reasoning, representation and learning.



The MACS Claim:

The use of affordances in control architectures may linkperception, action, learning and reasoning in a new way.

Perceiving the world in terms of affordances will provide a paradigmatic change in the architecture of embedded cognitive systems by helping to structure perception and reasoning in both an action-oriented and goal-directed way.



Affordances:

Notion created by cognitive psychologist J. J. Gibson (1979):

„An affordance is a resource or support that the environment offers an agent for action.

The agent must possess the capabilities to perceive and act upon it.“



Examples of Affordances:

throw

use as tool

hide,climb

Place to look for prey



Affordances (1):

• Affordances can be put in terms of abstract properties:throwable -> fist-sized dense object of certain weight rangesittable upon -> knee-high flat stable horizontal surface of certain minimum size

• Affordances depend particularly on the agent’s properties like body size, weight, form, and its perception and action capabilities



Affordances (2):

• Object identity is just another property, not necessarily the most important one – Its importance is goal dependent

• The abilities to perceive and act upon affordances may be acquired by learning – by experimentation and observation

• Acting upon affordances may require episodic knowledge:The sequence of actions required to act upon the affordance of a cup of coffee to drink from



Affordances (3):

• Selection of affordances depends on high-level goals

• Goals influence perception of affordances

• We do not get flooded by thousands of affordances

• Affordances comprise a functional view of environment

• Affordances are suited to structure the perceptual input for action and reasoning

• Link for perception, action, reasoning and learning



Some implications for adaptivity:

• Affordances would allow greater flexibility in manipulation tasks: If a searched object for a manipulation is not available, therobot may look for another one with the same affordancesand act upon the substitute instead.

• Such an ability would be especially helpful in complex environments with significant dynamics.


c. MACS: The Vision

The Vision:

Affordance-based control as a new

paradigm

to better link perception, action, reasoning

and learning, suited to advance the further

development of embodied cognitive

systems.


c. MACS: The Key Objectives

Main objective of MACS:

Explore and exploit the concept of affordances for the

design and implementation of autonomous mobile

robots

• Develop affordance-based control as a method for robotics

• Provide a new way for reasoning and learning to connect with reactive robot control



5 scientific & technological objectivesplus

1 dissemination objective



Scientific and technological objectives:

1. A radically new robot control architecture,

implementing affordance-based control

• Affordance-based control changes deeply the flow of information

as well as the required processes

• Use of affordances in control architectures is no emergent phenomenon,

• cannot be added on top of an existing control architecture,

• needs to be considered in the basic design.

• An affordance-based architecture will be proposed, tested and evaluated.

• Affordances will be integrated into perception, action and learning.




2. Grounded and goal-directed perception of affordances

• Affordances spring off perception on a low level, associating salient

perceptual features to the representation of what an object affords.

• What can or should be perceived and used as an affordance depends

on the sensors and actuators that the robot has.

• Filtering mechanism to prevent the robot from drowning in affordances

needs to be in effect deep down in the process of affordance perception.

• It has to be influenced from high-level goal-orientation or attention modes

preventing currently irrelevant affordances from distracting the controller.




3. Explicit affordance representations for different granularity levels

• Using affordances for reasoning and symbolic learning

requires an explicit representation

• Representation includes perception and action side of an affordance

plus

• episodic knowledge and

• expectations about feed-back from the environment when acting upon

an affordance.




4. Learning affordances by experimentation or by observation

• Affordances are individual on sensoric, physical, and experience

level

• A natural way of getting at affordances is learning

• Learning by individual experimentation or by imitation.

• Suitable learning methods will be developed

• Extreme option of teaching: programming

• Will be used for higher-level, complex or abstract affordances




5. Integrated demonstrator on an autonomous mobile robot

• Results will be demonstrated in integrated form

• Mobile robot: able to navigate and do simple manipulation tasks

• Wide range of perceptions through multi-modal sensor

configuration including vision and a 3D Laser scanner

• Robot control: plan-based to provide goal-directed behaviour



Dissemination objective:

Make the different involved scientific

communities,

possible appliers of the results, and the

interested

general public aware of the respective MACS

achievements


d. MACS: Steps to Achieve the Objectives

Workpackages:

• WP0 – Management (13+6 PM)

• WP1 – Infrastructure (39+4 PM)

• WP2 – Affordance-based Control Architecture (57+6 PM)

• WP3 – Perception of Affordances (76+2 PM)

• WP4 – Representation of Affordances (42+4 PM)

• WP5 – Learning of Affordances (47+4 PM)

• WP6 – Proof of Concept & Dissemination (38+4 PM)


e. MACS: The Key Milestones

month

Surv

eys

4M1

Spec

ifica

tions

7M2

Refer

ence

Impl

emen

tatio

ns

12M3

Mod

ule

Prot

otyp

es

18M4

Inte

grat

ion

&

Sim

ulat

ion

24M5

Afforda

nce-

base

d

Robot

Con

trol

30M6

Demon

stra

tor

33M7


f. MACS: Next Steps

Physical demonstrator

KURT2 (with 3D Laser scanner)

Commercial platform for research & education

Developed initially at FhG/AIS,Produced & distributed by KTO

3D Scanner developed and assembledat FhG/AIS,

Simple gripper under construction


f. MACS: Next Steps

Next steps:

• Publish evaluation of state of the art in affordance-related research

• Specify a demonstrator scenario

• Design missions and tasks that the affordance-based robot should accomplish in the demonstrator scenario

• Specify requirements for perception, representation and learning of affordances as well as for reasoning about and acting upon affordances

• Later:Publish benchmark problems suited to demonstrate the power and limitations of the approach


f. MACS: Next Steps

Next step towards dissemination:

• Application for a Dagstuhl seminar “Towards Affordance-based Robot Control”

• International conference and research center for computer science

• Schloss Dagstuhl in Wadern (close to Saarbrücken, Germany)

• Dagstuhl-Seminars: gatherings of 35–45 scientists working for a week on a specific computer science related topic with interdisciplinary aspects

• International participants: established researchers and promising young scientists

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.


f. MACS: Next Steps

Dagstuhl seminar details:

• Organizers: Erich Rome, Patrick Doherty, Georg Dorffner, Joachim HertzbergDeadline: Nov 15, 2004Seminar: 5 days in 1st half of 2006

• Preliminary list of potential participants; may be updated and extended when application succeeds

• Participation interests can be expressed via email ([email protected])(preferably non-German female young researchers)

• Participation fee (150 €) also covers accommodation and foodTravel expenses are not covered

• Young researchers may apply for grants covering travel expensesInformation available at: http://www.dagstuhl.de/HLSC/


Thank you for your attention

Erich Rome Robot Control Architectures Department

Documents

Transcript of Erich Rome Robot Control Architectures Department