Spatio-Temporal Case-Based Reasoning
for Behavioral Selection
Maxim Likhachev and Ronald ArkinMobile Robot Laboratory
Georgia Tech
Maxim Likhachev and Ronald Arkin
Broad Picture of the Work
• Part of Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems project at Georgia Tech
• Sponsored by the DARPA MARS program
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Motivation• Constant parameterization of robotic behavior
results in inefficient robot performance• Manual selection of “right” parameters is difficult
and tedious work
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Motivation (cont’d)• Use of Case-Based Reasoning methodology for an
automatic selection of optimal parameters in run-time
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Evaluated on:• Simulations
• Real robot– ATRV-JR in outdoor environment
– Nomad 150 in indoor environment
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Related Work• ACBARR, SINS and KINS systems
– use of case-based reasoning and reinforcement learning for the optimization of behavioral parameters
– contribute to some ideas behind the present algorithm
• Automatic optimization of parameters – genetic programming
– reinforcement learning
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Behavioral Control and CBR Module
CBR Module controls: Weights for each behavior BiasMove Vector
Noise Persistence Obstacle Sphere
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Input Features for Case Selection• Vector of spatial characteristics of environment
– D - distance to the goal– <σ, r> - degree of obstruction and distance to the most obstructing cluster of obstacles for each of K angular regions around the robot
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Input Features for Case Selection• Vector of temporal characteristics of environment
– Rs - short term robot movement
– Rl - long term robot movement
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Computation of Traversability Vector F
F: – represents traversability of each region
– approximates obstacle density function around the robot
– independent of goal distance
– smoothed over time:
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Input Features: ExampleSpatio-Temporal Case-Based Reasoning for Behavioral Selection
f0=0.92
f1=0.58
f2=1.0
f3=0.68
f0=0.02
f1=0.22
f2=0.63
f3=0.02
Vtemporal
ShortTerm: Rs=1.0LongTerm: Rl=0.7
Vtemporal
ShortTerm: Rs=0.01LongTerm: Rl=1.0
Maxim Likhachev and Ronald Arkin
High Level Structure of CBR Module Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Currentenvironment
FeatureIdentification
Spatial Features &Temporal Features
vectors
Spatial Features Vector Matching
(1st stage of Case Selection)
Temporal Features Vector Matching
(2nd stage of Case Selection)
Set ofSpatiallyMatching
cases
Set of Spatially and Temporally
Matching cases
Case switching
Decision tree
CaseAdaptation
Case Library
All the casesin the library
Best Matching orcurrently used case
CaseApplication
Case ready for application
Case Output Parameters(Behavioral Assemblage
Parameters)
Random Selection Process
(3rd stage of Case Selection)
Best Matchingcase
Maxim Likhachev and Ronald Arkin
Case Example I Spatio-Temporal Case-Based Reasoning for Behavioral Selection
CLEARGOALSpatial Vector:D (goal distance) = 5 density distance Region 0: σ0 = 0.00; r0 = 0.00Region 1: σ1 = 0.00; r1 = 0.00Region 2: σ2 = 0.00; r2 = 0.00Region 3: σ3 = 0.00; r3 = 0.00Temporal Vector:(0 - min, 1 - max) ShortTerm_Motion Rs = 1.000 LongTerm_Motion Rl = 0.700Case Output Parameters:MoveToGoal_Gain = 2.00Noise_Gain = 0.00Noise_Persistence = 10Obstacle_Gain = 2.00Obstacle_Sphere = 0.50Bias_Vector_X = 0.00Bias_Vector_Y = 0.00Bias_Vector_Gain = 0.00CaseTime = 3.0
Maxim Likhachev and Ronald Arkin
Case Example II Spatio-Temporal Case-Based Reasoning for Behavioral Selection
FRONTOBSTRUCTED_SHORTTERMSpatial Vector:D (goal distance) = 5 density distance Region 0: σ0 = 1.00; r0 = 1.00Region 1: σ1 = 0.80; r1 = 1.00Region 2: σ2 = 0.00; r2 = 1.00Region 3: σ3 = 0.80; r3 = 1.00Temporal Vector:(0 - min, 1 - max) ShortTerm_Motion Rs = 0.000 LongTerm_Motion Rl = 0.600Case Output Parameters:MoveToGoal_Gain = 0.10Noise_Gain = 0.02Noise_Persistence = 10Obstacle_Gain = 0.80Obstacle_Sphere = 1.50Bias_Vector_X = -1.00Bias_Vector_Y = 0.70Bias_Vector_Gain = 0.70CaseTime = 2.0
Maxim Likhachev and Ronald Arkin
Results Spatio-Temporal Case-Based Reasoning for Behavioral Selection
0
100
200
300
400
500
600
700
800
Heterogeneous Homogenous,15% density
Homogeneous,20% density
Environment type
Nonadaptive system CBR system
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Heterogeneous Homogenous,15% density
Homogeneous,20% density
Environment type
Nonadaptive system CBR system
Average travel distance Mission success rate
Simulations:
ATRV-JR: 12% average performance improvement in time steps ( based on 10 runs for each system in outdoor environment)
Maxim Likhachev and Ronald Arkin
Simulations & real robot experiments: Performance improvement as a function of
obstacle density
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Performance improvement using CBR over nonadaptive system
0.00
5.00
10.00
15.00
20.00
25.00
0.00 5.00 10.00 15.00 20.00 25.00
Obstacle density
Per
cent
impr
ovem
ent
Traveled Distance Improvement Time Steps Improvement
Simulations Nomad 150
Based on 10 runs for each systemin indoor environment
Maxim Likhachev and Ronald Arkin
Real Robot Run with CBRSpatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Real Robot Run without CBRSpatio-Temporal Case-Based Reasoning for Behavioral Selection
Maxim Likhachev and Ronald Arkin
Trajectories of the robotSpatio-Temporal Case-Based Reasoning for Behavioral Selection
Robot with CBR module Robot without CBR module
11% less travel distance
Maxim Likhachev and Ronald Arkin
Conclusions• Automatic selection of optimal behavioral
parameters results in robot performance improvement (based on simulations and real robot experiments)
• Careful manual selection of behavioral parameters is no longer required from a user
• Future Work– Automatic learning of cases:
• identifying when to create a new case• applying reinforcement learning techniques in finding optimal
parameters for existing cases
– Integration with other adaptation & learning methods (e.g., Learning Momentum)
Spatio-Temporal Case-Based Reasoning for Behavioral Selection
Top Related