Behavior Based SystemsBehavior Based Systems

Lezione 5

Key aspects of the behavior-based Key aspects of the behavior-based methodology:methodology:

Situatedness:Situatedness: The robot is an entity situated and surrounded by The robot is an entity situated and surrounded by

the real world (or real virtual world); Brooks: the real world (or real virtual world); Brooks: "World is its best model""World is its best model"

Embodiment:Embodiment: A robot has a physical presence (a body with A robot has a physical presence (a body with

limitations, non-holonomic robot), or virtual limitations, non-holonomic robot), or virtual presence (a software body with limitations)presence (a software body with limitations)

Emergence:Emergence: Intelligence arises from the interactions of the Intelligence arises from the interactions of the

robotic agent with its environment. It is not the robotic agent with its environment. It is not the property of either the agent or the environment property of either the agent or the environment in isolation but is rather a result of the interplay in isolation but is rather a result of the interplay between thembetween them

Issues in Behavior-based ParadigmIssues in Behavior-based Paradigm

Grounding in realityGrounding in reality Symbol grounding problem; Brooks: "The Symbol grounding problem; Brooks: "The

world is its own best model" Problem of world is its own best model" Problem of using simulatorsusing simulators

Ecological dynamicsEcological dynamics An agent is immersed in a highly dynamic An agent is immersed in a highly dynamic

environment Evolutionary processes shape environment Evolutionary processes shape agents to fit their ecological nicheagents to fit their ecological niche

ScalabilityScalability

DefinitionsDefinitions

An individual behavior:An individual behavior: A stimulus-response pair for a given A stimulus-response pair for a given

environmental setting that is modulated by environmental setting that is modulated by attention and determined by intentionattention and determined by intention

Attention:Attention: prioritizes tasks and focuses sensory prioritizes tasks and focuses sensory

resources and is determined by the current resources and is determined by the current environmental contextenvironmental context

Intention:Intention: Determines what set of behaviors should Determines what set of behaviors should

be active based on the robotic agent's be active based on the robotic agent's internal goals and objectivesinternal goals and objectives

Definitions (cont.)Definitions (cont.)

Emergent behavior:Emergent behavior: The global observed behavior; a The global observed behavior; a

consequence of the interaction of the consequence of the interaction of the active individual behaviorsactive individual behaviors

Reflexive behavior (purely reactive behavior)Reflexive behavior (purely reactive behavior) Behavior that is generated by hardwired Behavior that is generated by hardwired

reactive behaviors with tight sensor-reactive behaviors with tight sensor-effector coupling.effector coupling.

Animal BehaviorAnimal Behavior IdeasIdeas

Animal behavior defines intelligenceAnimal behavior defines intelligence Proof that intelligent behavior is possibleProof that intelligent behavior is possible Can provide models that can be used to create Can provide models that can be used to create

intelligent machinesintelligent machines Biological studies are not necessary viewed as Biological studies are not necessary viewed as

constraining for robots;constraining for robots; ProblemsProblems

Biological hardware is differentBiological hardware is different Our knowledge of the functioning of the Our knowledge of the functioning of the

biological hardware is often inadequatebiological hardware is often inadequate

Animal behavior StudiesAnimal behavior Studies

NeuroscienceNeuroscience The study of the nervous system's anatomy, The study of the nervous system's anatomy,

physiology, biochemistry, and molecular biologyphysiology, biochemistry, and molecular biology PsychologyPsychology

The study of mind and behaviorThe study of mind and behavior EthologyEthology

The study of animal behavior in natural conditionsThe study of animal behavior in natural conditions

Evidence from NeuroscienceEvidence from Neuroscience

Many specialized small systems in animal Many specialized small systems in animal kingdom have been analyzed:kingdom have been analyzed: bat sonarbat sonar wiping reflex in frogswiping reflex in frogs cockroach locomotioncockroach locomotion

Evidence exists that vector calculation is used in Evidence exists that vector calculation is used in some areas in the brainsome areas in the brain

Evidence exists that the "planning" in the central Evidence exists that the "planning" in the central nervous system translate into establishing nervous system translate into establishing equilibrium points that implicitly specify the equilibrium points that implicitly specify the desired motiondesired motion

Force fields in frog spinal cordForce fields in frog spinal cord

Some neuroscience theoriesSome neuroscience theories

Schema theorySchema theory Philosophical model; Immanuel KantPhilosophical model; Immanuel Kant Neurological schema theory (C.1910)Neurological schema theory (C.1910) A schema is the basic unit of behavior from which A schema is the basic unit of behavior from which

complex actions can be constructed.complex actions can be constructed. Neural networks theoryNeural networks theory

Basic model (1943) (McCulloch&Pitts)Basic model (1943) (McCulloch&Pitts) Perceptron (1959) (Minsky&Papert)Perceptron (1959) (Minsky&Papert) Backpropagation (1985) (Rummelhart, Hinton & Backpropagation (1985) (Rummelhart, Hinton &

WilliamsWilliams

PsychologyPsychology

Robotics is currently using ideas and theories of Robotics is currently using ideas and theories of human psychology that researchers in psychology human psychology that researchers in psychology no longer accept as valid.no longer accept as valid.

i.e.recycling of old ideas of psychology in the i.e.recycling of old ideas of psychology in the field of robotics;field of robotics;

controversial ideas and theorems can be used in controversial ideas and theorems can be used in roboticsrobotics

Some paradigms in psychologySome paradigms in psychology

Behaviorism, (1910)Behaviorism, (1910) Main idea: Everything is "Stimulus and Response"Main idea: Everything is "Stimulus and Response" B.F.SkinnerB.F.Skinner

Gestalt psychology, (1947)Gestalt psychology, (1947) Main idea: “Pure behaviorism is limited; there exists Main idea: “Pure behaviorism is limited; there exists

levels of organization above the sensation“levels of organization above the sensation“ Ecological psychology, (1979)Ecological psychology, (1979)

Main idea: "Things are perceived in terms of the Main idea: "Things are perceived in terms of the opportunities they afford." Concept of affordances. opportunities they afford." Concept of affordances.

The observer and the environment complement each The observer and the environment complement each other.other.

Some paradigms in psychologySome paradigms in psychology

Cognitive psychology, (1975)Cognitive psychology, (1975) Cognition: the activity of knowing: the acquisition, Cognition: the activity of knowing: the acquisition,

organization, and use of knowledge;organization, and use of knowledge; Main idea:Main idea:

““classical behaviorism explains only animal behaviorclassical behaviorism explains only animal behavior A series of subsystems processes the environmental A series of subsystems processes the environmental

information:information: stimulus attention perception thought processes ⇒ ⇒ ⇒stimulus attention perception thought processes ⇒ ⇒ ⇒

decision response⇒ ⇒ decision response⇒ ⇒

EthologyEthology

Animal behavior can be categorized into three Animal behavior can be categorized into three major classes:major classes: ReflexesReflexes

rapid, automatic, involuntary responses to a stimulirapid, automatic, involuntary responses to a stimuli TaxesTaxes

behavioral responses that orient the animal toward (positive behavioral responses that orient the animal toward (positive tropism) or away (negative tropism) from a stimulus, tropism) or away (negative tropism) from a stimulus, examples:examples:

chemotaxis (positive/negative tropism towards a certain chemotaxis (positive/negative tropism towards a certain chemical),chemical),

phototaxis (positive/negative tropism towards light)phototaxis (positive/negative tropism towards light) Fixed-action patternsFixed-action patterns

Time-extended response patterns triggered by a stimulus Time-extended response patterns triggered by a stimulus but persisting for longer than the stimulus itselfbut persisting for longer than the stimulus itself

Ecological nicheEcological niche

The status of an animal in its community, in The status of an animal in its community, in terms of its relations to food and enemies, is terms of its relations to food and enemies, is generally called its nichegenerally called its niche

Evolution has molded animals to fit their niche.Evolution has molded animals to fit their niche. To be self sufficient, an agent must exhibit To be self sufficient, an agent must exhibit

behavioral stability and market viability.behavioral stability and market viability. Behavioral stability implies that the agent does not Behavioral stability implies that the agent does not

succumb to irrecoverable debt of any vital resourcesuccumb to irrecoverable debt of any vital resource Market viability amounts to pleasing the robot's Market viability amounts to pleasing the robot's

employeremployer

Ecological niche of robotsEcological niche of robots

If the roboticist intends to build a system that is If the roboticist intends to build a system that is autonomous and can successfully compete with autonomous and can successfully compete with other environmental inhabitants, that system must other environmental inhabitants, that system must find a stable niche or it (as an application) will be find a stable niche or it (as an application) will be unsuccessful.unsuccessful.

For robots to be commonplace, they must find the For robots to be commonplace, they must find the ecological niches that allow them to survive ecological niches that allow them to survive and/or dominate their competitors, whether they and/or dominate their competitors, whether they be mechanical or biological.be mechanical or biological.

Robot behavior in Reactive systemsRobot behavior in Reactive systems

In reactive controlIn reactive control perception and action are tightly coupled typically in perception and action are tightly coupled typically in

the context of motor behaviors,the context of motor behaviors, to produce timely robotic response in dynamic and to produce timely robotic response in dynamic and

unstructured worldsunstructured worlds without the use of intervening abstract representation without the use of intervening abstract representation

or time history.or time history.

Purely reactive robotic systemPurely reactive robotic system

Behaviors serve as the basic building blocks for Behaviors serve as the basic building blocks for robotic actionsrobotic actions

Use of explicit abstract representational Use of explicit abstract representational knowledge is avoided in the generation of a knowledge is avoided in the generation of a responseresponse

Animal models of behavior often serve as a basis Animal models of behavior often serve as a basis for these systemsfor these systems

These systems are inherently modular from a These systems are inherently modular from a software design perspectivesoftware design perspective

Example:Example: task of going from one classroom to anothertask of going from one classroom to another IssuesIssues

route (shortest route)route (shortest route) safety (avoiding obstacles, avoiding teacher X)safety (avoiding obstacles, avoiding teacher X) speed (time optimal, or if with person Y, as slow as speed (time optimal, or if with person Y, as slow as

possible)possible) means of going (walking, elevators/lifts, conveyor means of going (walking, elevators/lifts, conveyor

belts etc)belts etc) opportunism (cafeteria/pub effect, event Z)opportunism (cafeteria/pub effect, event Z) cultural and social conventions (meeting a very cultural and social conventions (meeting a very

important person)important person) Observation:Observation: a simple task is actually a very a simple task is actually a very

complex one!complex one!

Behavior-based RoboticsBehavior-based Robotics

Behavior-based robotics grew out of the Behavior-based robotics grew out of the recognition that planning, no matter how recognition that planning, no matter how intentioned, is often waste of time.intentioned, is often waste of time.

Behavior-based robotic systems provide a means Behavior-based robotic systems provide a means for a robot to navigate in an uncertain and for a robot to navigate in an uncertain and unpredictable world without planning, by unpredictable world without planning, by endowing the robot with behaviors that deal with endowing the robot with behaviors that deal with specific goals independently and coordinating specific goals independently and coordinating them in a purposeful waythem in a purposeful way

BBSBBS

Behaviors are the underlying module of the Behaviors are the underlying module of the systemsystem

Behavioral decompositionBehavioral decomposition Systems consist of sequential modules achieving Systems consist of sequential modules achieving

independent functionsindependent functions

Robotic BehaviorRobotic Behavior

Generate a motor response from a given Generate a motor response from a given perceptual stimulusperceptual stimulus

Basis in biological studiesBasis in biological studies Serves as inspiration for designServes as inspiration for design

Behavior vs ActionBehavior vs Action BehaviorBehavior

Based on dynamic Based on dynamic processprocess

Operating in parallelOperating in parallel Lack of a central controlLack of a central control Fast couplings between Fast couplings between

sensors and motorssensors and motors Exploiting emergenceExploiting emergence

Side-effects from Side-effects from combined processescombined processes

Using properties of Using properties of environmentenvironment

ReactiveReactive

ActionAction Discrete in timeDiscrete in time

Well-defined start and Well-defined start and end pointsend points

Allows pre- and Allows pre- and postconditionspostconditions

Avoidance of side Avoidance of side effectseffects

Only one action or few Only one action or few actions at a timeactions at a time

Conflicts are undesired Conflicts are undesired and avoidedand avoided

DeliberativeDeliberative

ReactivityReactivity

Behaviors serve as building blocks for actionsBehaviors serve as building blocks for actions Abstract representation avoidedAbstract representation avoided Often modeled after animal behaviorsOften modeled after animal behaviors Inherently modularInherently modular

StimuliStimuli

Presence of stimulus is necessary but not Presence of stimulus is necessary but not sufficient in behavior-based robotsufficient in behavior-based robot

Stimulus must reach threshold value before Stimulus must reach threshold value before response is generatedresponse is generated

RepresentationRepresentation

Behaviors can be represented/stored in a network, Behaviors can be represented/stored in a network, with relationships between themwith relationships between them

Strength multiplier, or gain, can turn off Strength multiplier, or gain, can turn off behaviors or increase responsebehaviors or increase response

Properties of BehaviorsProperties of Behaviors Behaviors are feedback controllersBehaviors are feedback controllers Behaviors achieve specific tasks/goals (e.g., avoid-others, find-Behaviors achieve specific tasks/goals (e.g., avoid-others, find-

friend, go-home)friend, go-home) Behaviors are typically executed in parallel/concurrentlyBehaviors are typically executed in parallel/concurrently Behaviors can store state and be used to construct world Behaviors can store state and be used to construct world

models/representationmodels/representation Behaviors can directly connect sensors and effectors (i.e., take inputs Behaviors can directly connect sensors and effectors (i.e., take inputs

from sensors and send outputs to effectors) from sensors and send outputs to effectors) Behaviors can also take inputs from other behaviors and send outputs Behaviors can also take inputs from other behaviors and send outputs

to other behaviors (this allows for building networks)to other behaviors (this allows for building networks) Behaviors are typically higher-level than actions (go-home rather Behaviors are typically higher-level than actions (go-home rather

than turn-left-by-37.5-degrees)than turn-left-by-37.5-degrees) Behaviors are typically closed-loop but extended in timeBehaviors are typically closed-loop but extended in time When assembled into distributed representations, behaviors can be When assembled into distributed representations, behaviors can be

used to look ahead but at a time-scale comparable with the rest of the used to look ahead but at a time-scale comparable with the rest of the behavior-based systembehavior-based system