1
Algoritmos Genéticos aplicados em Machine Learning
Controle de um Robo(em inglês)
2
Robot Control using Genetic Algorithms
3
Summary
• Introduction– Robot Control– Khepera Simulator
• Genetic Model for Path Planning– Chromosome Representation– Evaluation Function– Case Studies
• Conclusions
4
The Robot Controller Problem
• Given a robot and a description of an environment, provide commands (motor speeds) to the robot, in order to achieve a path between two specified locations, which is collision-free and satisfies certain optimisation criteria.
(xi, yi)
(xf, yf)
5
Optimisation Criteria
• Robot should:– attempt near-optimal paths– avoid obstacles– perform straight motion
• Controller should be independent of:– the robot’s environment– target location
6
The Khepera Simulator
• Freeware mobile robot simulator (designed by Olivier Michel, University of Nice Sophia-Antipolis)
• User designed worlds
• Control algorithms can be written in C/C++• Robot’s position and angle reading
• 8 sensors (S0-S7): [0, 1023]• 2 motors (M1, M2): [-10, +10]
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
7
Simulator Readings:sensors, position and angle
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
S0-S7: [0, 1023]S0-S7: [0, 1023]
1000
1000
X
Y
Robot’s World
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
angle of the robot with the world
αα : [-: [-ππ , , ππ ]]
xx
yy
0
obstacle not obstacle very
detected closed
8
Control Mode
• To evolve the robot’s attitudes as it interacts with the environment
• Each robot action determines:– how well the controller performs with
respect a given task;– the next input stimuli to the controller.
• The controller should learnlearn as the robot interacts with the environment
9
Controller Model
Genetic Genetic AlgorithmAlgorithm
evolves robot’s evolves robot’s attitudesattitudes
Sensors
Position
Robot’s Angle
Goal Location
Motor 2 Motor 1
KheperaKheperaSimulatorSimulator
10
Proposed Modelbased on human behavior
IF Obstacle detectedObstacle detected
THEN
Avoid collision, forget targetAvoid collision, forget target
ELSE
SStraight to the target according to the traight to the target according to the
target directiontarget direction
END
11
Sensors Reading Simplification
Sleft Sright
Sback
S0
S1S2 S3
S4S5
S7 S6
Sleft = ( S0 + S1 + S2 ) / 3
Sright = ( S3 + S4 + S5 ) / 3
Sback = ( S6 + S7 ) / 2
12
Determining the Target Direction
(x,y)α = π/2
arrival position: (xf ,yf)
β
β = tan-1
[(yf - y)/(xf - x)]
β - απ/2
π
π/4
0
−π/4
−π/2
3π/4
−3π/4
Goal point toright side
Goal point infront
Goal point toleft side
Goal pointbehind
Direction =Direction =
13
Model
IF ((((SSleftleft > L > L) or () or (SSrightright > L > L) or () or (SSbackback > L > L))))
THENObstacle detected, avoid collision, forget target
Proximity-sensor = highest value (SProximity-sensor = highest value (Sleftleft, S, Srightright, S, Sbackback))
ELSEObstacle not detected (collision-free), straight to the target
Target direction = Target direction = ββ - - αα
END
L=collision threshold=900
14
Genetic Algorithm Modelling
• Problem
• Chromosome Representation
• Evaluation Function• Genetic Operators
• Techniques
• Parameters
15
Chromosome Representation
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2
TargetDirection(collision free) Obstacle detected
Left Right BackFront Left Right Back
Attitudes (Genes)
1 2 3 4 5 6 7
Which speed should be imposed to each motor in Which speed should be imposed to each motor in each situation the robot is?each situation the robot is?
16
Evaluation Function
( )
10;10;10
**7
1
≤≤≤≤≤≤
= ∑=
AiDiVi
AiDiViFi
• Main objectives:– (V) speed: as high as possible– (D) straight motion: same motor speed for M1 e M2– (A) action: reach a target and avoid obstacles
• Calculated based on the contribution for each gene [1,7], at each step.
17
Speed
max)*2/(|)2||1(| MMMVi +=
• Normalised sum of the absolute value of the motors speeds;
• Vi increases as both speeds increase• Whatever the robot does, it does quickly.
18
Straight Motion
=++=
=]7,6,5,1[,2))max)*2()21((1(
]4,3,2[,1
iMMM
iDi
• It favours high positive speeds to both motors
• When the robot is not oriented to the target (2,3,4), D=1 avoids contradictory learning
19
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
20
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
Rates the distance variation to the target between
two consecutive steps, and the maximum distance in one step,
for collision free/front
21
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
Rates the angle variation between two consecutive steps, and
the maximum angle in one step, for collision freeleft, right, back
22
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i Increases as the distance
to the proximity-sensor increasesin the step
23
Improving the Target Direction Model
Â2 - Â1 3π/4 π/4
Target at LEFT
Target Behind Target in FRONT
Target at RIGHT
-3π/4 - π/4
First Target Direction Model
4 possible target directions
0
π/4
π/2
3π/4
π/2
-3π/4 -π/4
-π/2
8 possible target directions
24
Chromosome Representations
Cada estado corresponde a uma única atitude (par de velocidadesM1 e M2), e cada atitude corresponde a um gene do cromossoma. Target Direction (collision free) Obstacle detected
M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 Front Left Right Back Left Right Back
Chromosome with 7 attitude Genes
Target Direction (collision free) Obstacle detected
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 Front Front Left Left Right Right Back Back Left Right Back left right front back front back left right
Chromosome with 11 attitude Genes
25
Genetic Algorithm
• Integer chromosome• Population Size =100• Generations = 50• Crossover Rate = 80 %
• Mutation Rate = 4%• Roulette Wheel Reproduction• Elitism• Linear scaling of fitness• 300 Evaluation Steps for each chromosome
• Average of 25 Experiments
26
Genetic Algorithm Performance7 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1 ,5
2
2,5
3
3,5
4
4,5
Number of Generations
FIT
NES
S
27
Genetic Algorithm Performance7 Genes Chromosome
Average of Best Chromosomes in 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
1 3 5 7 9
11 13 15 17
19
21
23 25 27 29
31
33
35 37 39
41
43
45 47 49
Number of Generations
FIT
NE
SS
28
Genetic Algorithm Performance11 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
Number of Generations
FIT
NE
SS
29
Genetic Algorithm Performance 11 Genes Chromosome
Average of Best Chromosomesin 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
Number of Generations
FIT
NE
SS
30
Paths Achieved in World 1Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
31
Paths Achieved in World 1Case Study 2SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
32
Speed Comparison
0
1000
2000
3000
4000
5000
6000
1 2 3 4 5
Case Studies
Nu
mb
er
of
Ste
ps
11 Genes Chromosome
7 Genes Chromosome
33
Paths Achieved in World 2Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
34
Paths Achieved in World 2Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
350
500
1000
1500
2000
2500
3000
3500
4000
4500
1 2 3
Case Studies
Nu
mb
er
of
Ste
ps
Speed Comparison
11 Genes Chromosome
7 Genes Chromosome
36
84,62%
40,00%
214,29%
Speed Comparison (%)
Case Study 1Case Study 1
Case Study 2Case Study 2
Case Study 3Case Study 3
37
Paths Achieved in World 3Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
38
Paths Achieved in World 3Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
39
Paths Achieved in World 3Case Study 3SITUAÇÃO 3
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
40
Paths Achieved in World 3Case Study 4SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
41
Speed Comparison
Speed Comparison – World 37 Genes Cromosome 11 Genes Cromosome
Case Study 1 target not reached 729Case Study 2 1124 618Case Study 3 target not reached target not reachedCase Study 4 target not reached 1655
42
Conclusions
• A simple GA was able to gradually evolve the robot control
• The robot achieved near optimal path towards the goal,
avoiding obstacles
• Retraining is not necessary when the environment changes
• Controller improved performance with the 11 genes model
• The robot has no memory about previous unsuccessful paths
and may get lost
• Other tasks can be included in the model (e.g. energy supply)
• Chromosome codification is limited for few robot’s situations
Top Related