Lecture 3: Behavior Selection Gal A. Kaminka [email protected] Introduction to Robots and...
-
date post
21-Dec-2015 -
Category
Documents
-
view
225 -
download
0
Transcript of Lecture 3: Behavior Selection Gal A. Kaminka [email protected] Introduction to Robots and...
![Page 1: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/1.jpg)
Lecture 3: Behavior Selection
Gal A. Kaminka
Introduction to Robots and Multi-Robot Systems
Agents in Physical and Virtual Environments
![Page 2: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/2.jpg)
2 © Gal Kaminka
Previously, on Robots …Multiple levels of control: Behaviors
Avoid Object
Wander
Explore
Map
Monitor Change
Identify objects
Plan changes
![Page 3: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/3.jpg)
3 © Gal Kaminka
Subsuming Layers How to make sure overall output is coherent?
e.g., avoid object is in conflict with explore Subsumption hierarchy: Higher levels modify lower
Avoid Object
Wander
Explore
Map
![Page 4: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/4.jpg)
4 © Gal Kaminka
This week, on Robots ….
Behavior Selection/Arbitration Activation-based selection
winner-take-all selection argmax selection (priority, utility, success likelihood, … )
Behavior networks Goal-oriented behavior-based control Takes a direct aim at key weaknesses of reactive approach
Behavior hierarchies
![Page 5: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/5.jpg)
5 © Gal Kaminka
Behavior Selection (Arbitration) One behavior takes over completely
All sensors, actions controlled by the behavior Behaviors compete for control
Key questions: How do we select the correct behavior? When do we terminate the selected behavior?
![Page 6: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/6.jpg)
6 © Gal Kaminka
Maes’ Actions Selection Mechanism (MASM)
Some key highlights: Merges some planning with behavior-based control
Goal-oriented, allows predictions Responsive, allows reactivity “Speed vs. thought” trade-off
Lots of number-hacking A later article addressed this issue with learning However, complex environment may suffer from this
![Page 7: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/7.jpg)
7 © Gal Kaminka
Overall Structure
Behaviors: preconditions, delete/add lists, activation Activation links spread positive and negative activation
Sensor
Sensor
Sensor
Goal
Goal
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
Behavior
![Page 8: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/8.jpg)
8 © Gal Kaminka
Behaviors
Similar to a fully-instantiated planning operator No variables (i.e, pick-up-A, not pick-up(A)
Preconditions (what must be true to be executable) Add/delete list (what changes once behavior
executes)
Behavior
![Page 9: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/9.jpg)
9 © Gal Kaminka
Connecting Behaviors
Activation: Sensors to behaviors with matching preconditions
Sensor
Behavior
![Page 10: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/10.jpg)
10 © Gal Kaminka
Connecting Behaviors
Activation: Sensors to behaviors with matching preconditions Add lists to behaviors with matching preconditions
Sensor
Behavior
Behavior
Behavior
![Page 11: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/11.jpg)
11 © Gal Kaminka
Connecting Behaviors (Backward)
Activation: Goals to behaviors with matching add lists Behaviors to behaviors with matching add lists
Sensor
Behavior
Behavior
Behavior
Goal
![Page 12: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/12.jpg)
12 © Gal Kaminka
Connecting Behaviors (Backward)
Advantages: Goal-orientedness (goal drives behaviors) Reactivity (sensors drive behaviors) Parameterized!
Sensor
Behavior
Behavior
Behavior
Goal
![Page 13: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/13.jpg)
13 © Gal Kaminka
Handling Conflicts
Conflicting behaviors inhibit each other This is a winner-take-all configuration
Sensor
Behavior
Behavior
Behavior
Goal
Sensor
Sensor
GoalBehavior
Behavior
Behavior
![Page 14: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/14.jpg)
14 © Gal Kaminka
Winner Take All
A very basic structure in neural networks Relies on recurrence
Key idea: Nodes compete by inhibiting each other After some cycles, winner emerges This is useful in many neural models of behavior
![Page 15: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/15.jpg)
15 © Gal Kaminka
Basic Structure
Each node excited by incoming information Each node’s activation inhibits its competitors
1+
+
+
-
-
-
-
+
+
+
2
3
![Page 16: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/16.jpg)
16 © Gal Kaminka
First activation
Darker == more activation
(2 is most active, 1 least)
+
+
+
-
-
-
-
+
+
+
1
2
3
![Page 17: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/17.jpg)
17 © Gal Kaminka
After a few cycles
3 and 2 stronger than 1, so 1 quickly deactivates 2 slightly stronger than 3, so 3 slowly deactivates
+
+
+
-
-
-
-
+
+
+
1
2
3
![Page 18: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/18.jpg)
18 © Gal Kaminka
After a few more cycles
Once 1 is out of picture, only 2 and 3 compete 2 becomes stronger: a weaker 3 inhibits 2 less
+
+
+
-
-
-
-
+
+
+
1
2
3
![Page 19: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/19.jpg)
19 © Gal Kaminka
Until finally….
Only output from 2 remains
+
+
+
-
-
-
-
+
+
+
1
2
3
![Page 20: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/20.jpg)
20 © Gal Kaminka
Winner Take All
Output from winning node ends up being used Typically, if over a threshold
Once node becomes active, never lets in any other A basic problem. Standard solutions: reset after some time, decay, …
This mechanism can be used to solve competition Activation is key feature/requirement
![Page 21: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/21.jpg)
21 © Gal Kaminka
Running a behavior network
Let activation spread for a while, wait for threshold Once behavior over threshold, execute it
Reset activation after it’s done
Sensor
Behavior
Behavior
Behavior
Goal
Sensor
Sensor
GoalBehavior
Behavior
Behavior
![Page 22: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/22.jpg)
22 © Gal Kaminka
Advantages
We’ve discussed planned vs. reactive behavior Threshold value changes “speed vs. thought”
Larger threshold, more behaviors involved before selection Small threshold, less likely to find optimal chain
This is not hybrid architecture—really something new!
![Page 23: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/23.jpg)
23 © Gal Kaminka
Criticisms
Where will this fail? Succeed?
What needs improvement? What does not?
What tasks is it good for?
As scientists,
you must always ask yourself these questions
![Page 24: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/24.jpg)
24 © Gal Kaminka
Protected Goals Sussman Anomaly:
Given: A on B, B on table, C on table Do: A on B, B on C, C on table
No way to do this without undoing a subgoal If one is not careful, might go into thrashing
Take off A, put A back, Take off A, …. Maes added mechanism for protected goals
Not clear where protection comes from
![Page 25: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/25.jpg)
25 © Gal Kaminka
Other problems with MASM No variables Blow up in the number of behaviors Thrashing: Behavior resets, then re-selected Bug in activation algorithm:
Activation from goals is divided by number of goals Thus a behavior satisfying more goals is not preferred Additional minor issues like this found, corrected later Tyrell 1993,1994, Dorer 1999, Blumberg 1994, …
![Page 26: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/26.jpg)
26 © Gal Kaminka
Reminder We are talking about behavior selection Multiple behaviors exist Question is which one to choose
Behaviors compete for control of robot
Behavior networks have activation: Goal priority “meets” sensor data (preconditions, effects) Winner-take-all selection
![Page 27: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/27.jpg)
27 © Gal Kaminka
Activation-based selection For each behavior, build an activation function
How useful it is (utility, value) How urgent it is (priority) How likely it is to succeed (likelihood of success) How much it matches current state (applicability) ….
Can of course combine these (e.g., utility X priority) Select behavior with top activation Let it run Re-evaluate all activations
![Page 28: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/28.jpg)
28 © Gal Kaminka
Formal behavior selection Behaviors are arranged in a DAG <B,E>
DAG: Directed Acyclic Graph B set of behaviors (vertices) E set of edges (a,b), where a, b in B.
The graph is structured hierarchically: Single root behavior is most general leaf behaviors correspond to primitive actions A path from every behavior to at least one primitive behavior children(b) = { all behaviors a, such that (b,a) is in E }
![Page 29: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/29.jpg)
29 © Gal Kaminka
Hierarchical behaviors The root behavior is always active An active behavior with no active child must select one An active behavior can decide to deactivate itself
WinGame
Play Interrupt
Attack-Center Zone Defense
Move Kick Pass Clear Turn
Attack Pincer
![Page 30: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/30.jpg)
30 © Gal Kaminka
argmax selection At any given time, select behavior whose
priority value likelihood of success applicability
is greatest No sequence of behaviors known in advance
Many instances of behaviors can co-exist, compete
![Page 31: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/31.jpg)
31 © Gal Kaminka
Formally …. f(b) be a function which gives the behavior’s activation Then the arbitration result is:
argmaxc (f(c)), where c in children(b) For instance, to choose by value,
argmaxc (value(c)) Or, to choose by priority,
argmaxc (priority(c) Or decision-theoretic choice,
argmaxc (probability(c) * value(c))
![Page 32: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/32.jpg)
32 © Gal Kaminka
Subsumption as argmax selection Subsumption level of behavior b, given by level(b) Applicability of behavior b, given by app(b) --- 0 or 1 Subsumption arbitration: argmaxb (app(b) * level(b))
Avoid Object
Wander
Explore
Map
![Page 33: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/33.jpg)
33 © Gal Kaminka
Case Study: HandleBall Arbitrator (ChaMeleons’01)
HandleBall behavior triggered when player has ball Must select between multiply-instantiated children:
shoot on goal, pass for shot, pass forward, dribble to goal dribble forward, clear, pass to closer, ….
We defined a complex arbitrator combining: priority, and likelyhood of success
![Page 34: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/34.jpg)
34 © Gal Kaminka
HandleBall Example
![Page 35: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/35.jpg)
35 © Gal Kaminka
“Number-hacking”: Thrashing
de-selection and re-selection of behaviors the time
sensor value around threshold
sensor value around threshold
sensor value around threshold
sensor value around threshold
![Page 36: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/36.jpg)
36 © Gal Kaminka
“Number-hacking”: Sensitivity Sensitivity to specific values, ranges
Manually adjusting values by 0.1 to get a wanted result… Where do the numbers come from? Learning?
e.g., programmer forgot a range of values? e.g., programmer needs to extend range
![Page 37: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/37.jpg)
37 © Gal Kaminka
State-Based Selection
State-based selection Look at world and internal state to make selection
Behaviors as operators? Almost. Pre-conditions, termination-conditions Selection control rules (non-numeric preferences, priorities)
Finite state machines and hierarchical machines
![Page 38: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/38.jpg)
38 © Gal Kaminka
State-Based Behavior Selection
Elements from reactive control, but with internal state Quick response to sensor readings
Sensor-driven operation Behaviors maintain internal state
e.g., previously-executed behaviors e.g., previous sensor readings …
![Page 39: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/39.jpg)
39 © Gal Kaminka
Behaviors as operators Conditions:
Preconditions: When is it applicable? Termination conditions: When is it done?
Conditions test sensors, internal state Must maintain World Model
Can be simple (e.g., vector of sensor readings) Or complex (e.g., internal variables, previous readings)
![Page 40: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/40.jpg)
40 © Gal Kaminka
State-Based Selection: Architecture
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
![Page 41: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/41.jpg)
41 © Gal Kaminka
State-Based Selection: Architecture
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
![Page 42: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/42.jpg)
42 © Gal Kaminka
State-Based Selection: Architecture
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
![Page 43: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/43.jpg)
43 © Gal Kaminka
Conflicting Behaviors
What if more than one behavior matches?
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
![Page 44: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/44.jpg)
44 © Gal Kaminka
Preference Rules Prefer one behavior over another Provide “local guidance”
Do not consider all possible cases, nor global ranking Test world model (which also records behaviors)
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
PreferenceRules
![Page 45: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/45.jpg)
45 © Gal Kaminka
שאלות?
![Page 46: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/46.jpg)
46 © Gal Kaminka
What’s in a world model?
World Model(beliefs)
Behavior
Behavior
Behavior
Behavior
CommandScheduling
PreferenceRules
![Page 47: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/47.jpg)
47 © Gal Kaminka
What’s in a world model?
A vector of sensor readings Distance front = 250 Light Left = Detected Battery = Medium level
A vector of virtual sensors Distance front < 90 AND light front Average front distances = 149.4
Complex
Simple
![Page 48: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/48.jpg)
48 © Gal Kaminka
What’s in a world model?
A vector processed data Estimated X, Y from detected landmarks Seen purple blob at pixel 2,5 Communication from teammate
A vector of world models Position of opponent 2 seconds ago My position 10 seconds ago
Complex
Simple
![Page 49: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/49.jpg)
49 © Gal Kaminka
Hierarchical Behaviors
Hierarchies allow designer to build reusable behaviors
At any given moment, a path is selected All behaviors in the path are active
May issue action commands Monitor sensors
This is different from a function call stack What happens when behavior terminates?
![Page 50: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/50.jpg)
50 © Gal Kaminka
Case Study: ModSAF Preference rules manage high-priority interrupts Preconditions dictate ordering
Execute Mission
Fly Flight Plan Wait-at-Point
Fly Route Land
NOE Low Contour Unmask Shoot
Find Position
Halt Join Scout Engage
![Page 51: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/51.jpg)
51 © Gal Kaminka
State-based selection
Preconditions and termination conditions Effective, allow flexible re-use
Very complex behavior generated Thrashing still very much a problem
![Page 52: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/52.jpg)
52 © Gal Kaminka
Finite State Machines: Avoid Thrashing by Sequencing
Every state represents a behavior Transitions are triggered by sensor readings
Start
A
A
B C
AC
A D
C
![Page 53: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/53.jpg)
53 © Gal Kaminka
Example: Foraging
Pick UpClose to Puck
Go HomeAcquire
Have Puck
Drop At Home
![Page 54: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/54.jpg)
54 © Gal Kaminka
Example: Foraging
Pick UpClose to Puck
Go HomeAcquire
Lost Puck
Have Puck
Drop At Home
![Page 55: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/55.jpg)
55 © Gal Kaminka
Hierarchical Finite State Machines
A behaviors can be decomposed into others Decomposition selected based on sensors, memory
Start
A
A
B CA C
A D
C
![Page 56: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/56.jpg)
56 © Gal Kaminka
BITE: Bar Ilan Teamwork Engine
Combining FSAs and state-based selection Multiple opportunities for arbitration
Temporal (what comes next) Hierarchical (which child should be selected)
Prevention of cycling, thrashing e.g., by keeping record of which child was recently selected
![Page 57: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/57.jpg)
57 © Gal Kaminka
שאלות?
![Page 58: Lecture 3: Behavior Selection Gal A. Kaminka galk@cs.biu.ac.il Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.](https://reader031.fdocuments.us/reader031/viewer/2022020921/56649d575503460f94a36788/html5/thumbnails/58.jpg)
58 © Gal Kaminka
Homework #2
1. Propose algorithms for detecting (a) thrashing, (b) cycling. The algorithms must be appropriate for execution on robots.
2. One of the advantages of the state-based and activation-based approaches (non-FSA) is that they allow opportunism. Using FSAs limits this opportunism, since behaviors are executed in pre-determined sequences. Propose a method to allow opportunism in FSAs.
3. Propose a technique for resolving thrashing and cycling once detected.