Master by Research Thesis - QUT · Master by Research Thesis By Hui Miao Student No.: 06478689...
Transcript of Master by Research Thesis - QUT · Master by Research Thesis By Hui Miao Student No.: 06478689...
Master by Research Thesis
By
Hui Miao
Student No.: 06478689
submitted to the
Faculty of Science and Technology Queensland University of Technology
Project Title:
Robot Path Planning in Dynamic Environments using a Simulated Annealing Based Approach
March 2009
Supervisor: Associate Professor Yu-Chu Tian Associate Supervisor: Associate Professor Yanming Feng
II
III
Abstract
Mobile robots are widely used in many industrial fields. Research on path planning for
mobile robots is one of the most important aspects in mobile robots research. Path planning
for a mobile robot is to find a collision-free route, through the robot’s environment with
obstacles, from a specified start location to a desired goal destination while satisfying certain
optimization criteria. Most of the existing path planning methods, such as the visibility graph,
the cell decomposition, and the potential field are designed with the focus on static
environments, in which there are only stationary obstacles. However, in practical systems
such as Marine Science Research, Robots in Mining Industry, and RoboCup games, robots
usually face dynamic environments, in which both moving and stationary obstacles exist.
Because of the complexity of the dynamic environments, research on path planning in the
environments with dynamic obstacles is limited. Limited numbers of papers have been
published in this area in comparison with hundreds of reports on path planning in stationary
environments in the open literature.
Recently, a genetic algorithm based approach has been introduced to plan the optimal path
for a mobile robot in a dynamic environment with moving obstacles. However, with the
increase of the number of the obstacles in the environment, and the changes of the moving
speed and direction of the robot and obstacles, the size of the problem to be solved increases
sharply. Consequently, the performance of the genetic algorithm based approach deteriorates
significantly. This motivates the research of this work.
This research develops and implements a simulated annealing algorithm based approach to
find the optimal path for a mobile robot in a dynamic environment with moving obstacles.
The simulated annealing algorithm is an optimization algorithm similar to the genetic
algorithm in principle. However, our investigation and simulations have indicated that the
simulated annealing algorithm based approach is simpler and easier to implement. Its
performance is also shown to be superior to that of the genetic algorithm based approach in
both online and offline processing times as well as in obtaining the optimal solution for path
IV
planning of the robot in the dynamic environment.
The first step of many path planning methods is to search an initial feasible path for the robot.
A commonly used method for searching the initial path is to randomly pick up some vertices
of the obstacles in the search space. This is time consuming in both static and dynamic path
planning, and has an important impact on the efficiency of the dynamic path planning. This
research proposes a heuristic method to search the feasible initial path efficiently. Then, the
heuristic method is incorporated into the proposed simulated annealing algorithm based
approach for dynamic robot path planning. Simulation experiments have shown that with the
incorporation of the heuristic method, the developed simulated annealing algorithm based
approach requires much shorter processing time to get the optimal solutions in the dynamic
path planning problem. Furthermore, the quality of the solution, as characterized by the
length of the planned path, is also improved with the incorporated heuristic method in the
simulated annealing based approach for both online and offline path planning.
V
List of Tables
Table 1: Four testing environments ----------------------------------------------------------------- 56
Table 2: Control parameters for the simulated annealing algorithm----------------------------- 56
Table 3: Results for environment one --------------------------------------------------------------- 58
Table 4: Results for environment two -------------------------------------------------------------- 59
Table 5: Results for environment three ------------------------------------------------------------- 61
Table 6: Results for environment four -------------------------------------------------------------- 63
Table 7: Processing time of online planning on each case --------------------------------------- 64
Table 8: Three environments for performance evaluation --------------------------------------- 68
Table 9: Comparison of execution time base on same vertices number ------------------------ 68
Table 10: Comparison of the length of the final path --------------------------------------------- 68
Table 11: Results of comparison in case one ------------------------------------------------------- 79
Table 12: Results of comparison in case two ------------------------------------------------------ 80
Table 13: Results of comparison in case three ----------------------------------------------------- 81
Table 14: Results of comparison in case four ------------------------------------------------------ 82
Table 15: Conclusion of the improvements -------------------------------------------------------- 84
VI
VII
List of Figures
1. Classifications of the robot path planning methods -------------------------------------------- 12
2. Visibility graph -------------------------------------------------------------------------------------- 14
3. Complete visibility graph -------------------------------------------------------------------------- 14
4. Quadtree decomposition --------------------------------------------------------------------------- 15
5. Cell decomposition --------------------------------------------------------------------------------- 16
6. Potential field --------------------------------------------------------------------------------------- 18
7. Obstacle is out of the vision zone ---------------------------------------------------------------- 19
8. Obstacle partially occupies the vision zone ----------------------------------------------------- 20
9. Obstacle impedes the main line between robot and target ------------------------------------ 20
10. Obstacle fully occupies the vision zone -------------------------------------------------------- 20
11. Path planning rule 2 ------------------------------------------------------------------------------- 21
12. Path Planning Rule 3 ----------------------------------------------------------------------------- 22
13. The representation of the chromosome in GA planner --------------------------------------- 25
14. Genetic planners ----------------------------------------------------------------------------------- 26
15. Regular grids representation --------------------------------------------------------------------- 30
16. The D* algorithm --------------------------------------------------------------------------------- 31
17. The Focussed D* algorithm --------------------------------------------------------------------- 31
18. Framed-Quadatree Structure --------------------------------------------------------------------- 33
19. Procedure of the simulated annealing algorithm ---------------------------------------------- 36
20. Structure of Player -------------------------------------------------------------------------------- 38
21. Player/Stage architecture ------------------------------------------------------------------------- 39
22. Scenario one of moving obstacle ---------------------------------------------------------------- 43
23. Scenario two of moving obstacle --------------------------------------------------------------- 44
24. Online path planning ------------------------------------------------------------------------------ 44
25. State transformation diagram of the simulated annealing algorithm ----------------------- 44
26. Enlarged obstacles -------------------------------------------------------------------------------- 45
27. Procedure of simulated annealing approach for path planning ----------------------------- 49
28. The initial path selection process --------------------------------------------------------------- 51
VIII
29. Deleting operator --------------------------------------------------------------------------------- 52
30. Offline path planning in case one --------------------------------------------------------------- 57
31. Online path planning in case one --------------------------------------------------------------- 57
32. Offline path planning in case two --------------------------------------------------------------- 58
33. Online path planning in case two 1 ------------------------------------------------------------- 58
34. Online path planning in case two 2 ------------------------------------------------------------- 59
35. Offline path planning in case three ------------------------------------------------------------- 60
36. Obstacles appears on different times ----------------------------------------------------------- 60
37. Online path planning in case three -------------------------------------------------------------- 60
38. Offline path planning in case four -------------------------------------------------------------- 62
39. Obstacles appears on different times ----------------------------------------------------------- 62
40. Online path planning in case four 1 ------------------------------------------------------------- 62
41. Online path planning in case four 2 ------------------------------------------------------------- 62
42. Convergence of the SA approach --------------------------------------------------------------- 63
43. Representation of the chromosome in GA approach ----------------------------------------- 65
44. The repair operator -------------------------------------------------------------------------------- 66
45. Comparison of execution time base on same vertices number ------------------------------ 69
46. Comparison of final path length base on same vertices number ---------------------------- 69
47. Convergence comparison in environment one ------------------------------------------------ 70
48. Convergence comparison in environment two ------------------------------------------------ 70
49. Convergence comparison in environment three ----------------------------------------------- 71
50. Environment one ---------------------------------------------------------------------------------- 72
51. Environment one from [17] ---------------------------------------------------------------------- 72
52. Process of the heuristic selection method ------------------------------------------------------ 76
53. Random picking method ------------------------------------------------------------------------- 77
54. Heuristic picking method ------------------------------------------------------------------------ 77
55. Random picking method in case one ----------------------------------------------------------- 78
56. Heuristic picking method in case one ---------------------------------------------------------- 78
57. Random picking method in case two ----------------------------------------------------------- 79
58. Heuristic picking method in case two ---------------------------------------------------------- 79
59. Random picking method in case three --------------------------------------------------------- 80
IX
60. Heuristic picking method in case three --------------------------------------------------------- 80
61. Random picking method in case four ---------------------------------------------------------- 81
62. Heuristic picking method in case four ---------------------------------------------------------- 81
63. Path solution comparison -------------------------------------------------------------------------84
64. Offline processing time comparison ------------------------------------------------------------ 84
65. Online processing time comparison ------------------------------------------------------------ 84
66. The comparison of the three methods in processing time ----------------------------------- 85
67. The comparison of the three methods in finding path length -------------------------------- 86
X
XI
Contents
Chapter 1 Introduction 1
1.1 Background ------------------- ------------------------------------------------------------ 1
1.2 Research Gap Statements and Motivation -------------------------------------------- 3
1.3 Aim of the Project ------------------------------------------------------------------------ 5
1.4 Significance of the Research ------------------------------------------------------------ 6
1.5 Contributions of the Thesis ------------------------------------------------------------- 7
1.6 Structure of the Thesis ------------------------------------------------------------------- 8
1.7 Related Publication ---------------------------------------------------------------------- 9
Chapter 2 Background and Literature Review 11
2.1 The Classification of the Path Planning Methods ---------------------------------- 11
2.2 Path Planning in a Static and Known Environment -------------------------------- 13
2.2.1 Visibility Graph -------------------------------------------------------------------- 13
2.2.2 Cell Decomposition Method ----------------------------------------------------- 15
2.2.3 Artificial Potential Field -----------------------------------------------------------16
2.3 Path Planning in a Static and Unknown Environment ------------------------------18
2.4 Path Planning in a Dynamic and Known Environment ----------------------------23
2.5 Path Planning in a Dynamic and Unknown Environment -------------------------27
2.6 The Simulated Annealing Algorithm and SA Based Approach -------------------35
2.6.1 The Simulated Annealing Algorithm --------------------------------------------35
2.6.2 Simulated Annealing Based Approach ------------------------------------------36
2.7 Robot and Environment Simulators -------------------------------------------------- 38
Chapter 3 The Simulated Annealing Algorithm Based Approach 41
3.1 Structure and Assumptions of the method ------------------------------------------ 42
3.2 Mathematic Modelling ----------------------------------------------------------------- 45
3.2.1 Environment Modelling ---------------------------------------------------------- 45
3.2.2 Algorithm Structure and Expressions --------------------------------------------46
XII
3.2.3 Initial Path Selection Process -----------------------------------------------------50
3.2.4 Random Path Planning ------------------------------------------------------------52
3.2.5 Online Path Planning --------------------------------------------------------------53
3.3 Platform for the Research --------------------------------------------------------------54
Chapter 4 Simulation Results and Performance Evaluation 55
4.1 Simulation Environments and Algorithm Parameters ------------------------------55
4.2 Simulation Results ----------------------------------------------------------------------57
4.2.1 Case One ----------------------------------------------------------------------------57
4.2.2 Case Two ----------------------------------------------------------------------------58
4.2.3 Case Three -------------------------------------------------------------------------- 60
4.2.4 Case Four --------------------------------------------------------------------------- 61
4.3 Performance Evaluation --------------------------------------------------------------- 63
4.3.1 The Genetic Based Approach ---------------------------------------------------- 64
4.3.2 Results Comparison ----------------------------------------------------------------67
Chapter 5 Heuristic Search Method for the Simulated Annealing Approach 73
5.1 The Structure and Implementation of the Heuristic Selecting Method ----------73
5.2 Performance Evaluation of the Heuristic Method ----------------------------------78
5.2.1 Environment One ----------------------------------------------------------------- 78
5.2.2 Environment Two ----------------------------------------------------------------- 79
5.2.3 Environment Three --------------------------------------------------------------- 80
5.2.4 Environment Four ---------------------------------------------------------------- 81
5.3 Discussion of the Evaluation Results ----------------------------------------------- 82
5.3.1 Offline Planning ------------------------------------------------------------------ 82
5.3.2 Online Planning ------------------------------------------------------------------- 83
5.3.3 The Length of the Path ----------------------------------------------------------- 83
5.3.4 Comparison with the Genetic Algorithm Based Approach ------------------ 85
Chapter 6 Conclusion 87
6.1 Summary --------------------------------------------------------------------------------- 87
XIII
6.2 Research Limitations ------------------------------------------------------------------- 88
6.3 Future Work ----------------------------------------------------------------------------- 89
Acknowledgement 91
References
XIV
XV
Statement of Original Authorship The work contained in this thesis has not been previously submitted to meet
requirements for an award at this or any other higher education institution.
To the best of my knowledge and belief, the thesis contains no material
previously published or written by another person except where due
reference is made.
Signature:
Date:
XVI
XVII
Acknowledgement
I would like to express my gratitude to my supervisor Associate Professor Yu-Chu Tian and
my associate supervisor Associate Professor Yanming Feng, who have given me excellent
supervision and guidance. I would also like to thank them for helping me to gain experience
on research study, which will be very helpful for my future study. I also wish to thank my
friends who helped me through all the difficult times.
XVIII
1
Chapter 1 Introduction This thesis develops and implements a simulated annealing algorithm based approach with a
heuristic search method for path planning for a mobile robot in dynamic environments. The
approach uses the vertices of the static and dynamic obstacles as search space. Using the
heuristic search method, it searches an initial feasible path for the robot in the environment.
Then, the approach applies the simulated annealing algorithm to find the optimal path for the
robot. When a moving obstacle is detected with possible collision with the robot, this
two-step planning process is activated for dynamic path planning. This chapter introduces
some background information about robot path planning, the motivation of the project,
research gaps, and research significance. It also highlights the contributions of this thesis.
1.1 Background
At present, research on various algorithms for mobile robot path planning is a hot topic.
Mobile robots are widely used in many hazardous industrial fields where there may be
dangers for people, such as aerospace research, the nuclear industry, and the mining industry.
2
To find a safe path in a dangerous environment for the mobile robot is an essential
requirement for the success of any mobile robotic systems. Therefore, research on path
planning algorithms to make the robot move from the start point to the termination point
without collision with obstacles is a fundamental requirement for the mobile robot safety in
such environments. Moreover, to reduce the processing time, communication delay and
energy consumption, the planned path is naturally required to be optimal with the shortest
length.
At the initial stage of the robot industry, a robot was simple constituted by mechanical arms
controlled by motor engines. Path planning for the robot was often in stationary obstacle
environment. As an example of the robot, path planning in static environments were
discussed in [1]. However, with the development of the robot technology, robots have been
used in many industrial fields such as aerospace research, marine research, and mining
industry, to just mention a few. A lobster-like underwater walking robot [2] is one of these
new types of robots. Recently, Australian researchers have developed an unmanned
underwater vehicle robot for reef surveying [3]. The robot is equipped with sonar and vision
systems, and works at the platform of the sea. Thus, how to response quickly to the changing
environment to avoid the stationary rocks in the seabed and big moving fish is a primary
issue in the design and operation of the robot.
Recently, the genetic algorithm based approach has been developed for robot path planning
in dynamic environments, but the efficiency is not sufficient for large-scale path planning
problems. Therefore, more efficient path planning methods in dynamic obstacle
environments need to be developed for adapting the development of robots research.
The simulated annealing algorithm is similar to the genetic algorithm in solving general
optimization problems. It has been implemented for robot path planning in static
environments. What’s the performance of the simulated annealing based approach in
dynamic path planning? This is a question yet to be answered and this research gives a
positive answer to the question.
3
1.2 Research Gap Statements and Motivation
Because the entire information of a dynamic environment will change along with the
movement of obstacles, the complexity and uncertainty of the path planning problem
increase greatly in dynamic environments. Therefore, traditional path planning methods,
such as Visibility Graph [4], Voronoi diagrams [5] and Grids [6], are not suitable for planning
path in dynamic environments. Recently, Wang, Sillitoe and Davide [14] introduced a genetic
algorithm based navigation method to deal with the path planning problem in an environment
with moving obstacles. However, this method still has drawbacks: local minima results may
occur in some cases [14] and the calculating time for finding the first feasible path increases
greatly while the number of the obstacles increases [14]. The result (feasible path)
convergence rate is high using genetic algorithms, making the method lose the mechanism
for exploration. If the method has a low convergence rate, it could raise the mutation
probability and reduce the crossover probability [22]. Nevertheless, research in [9] shows
that large mutation rates improve the quality of the algorithm. Therefore, an improved
method for path planning in dynamic environment needs to be designed.
The A* algorithm [10] is another path planning method to help the robot to find the optimal
path in grid decomposed static maps. The environment with free space and obstacles is
presented by a set of uniformed regular grids. The A* algorithm uses heuristic based Dijkstra
algorithm to obtain the optimal result for the robot. The drawback is that the A* uses
uniformed grids representation which must allocate large amounts of memory for regions
that may never be traversed or may not contain any obstacles. This drawback may affect the
efficiency of the method. Quadtree representation can be used to enhance the efficiency of
the method, but it sacrificed the optimality for the efficiency. The above drawbacks also
happen in the D* method [11] which is the dynamic version of the A* method. The D* helps
robot to find the optimal path in uncertain environments. Besides, in dynamic environments
with moving obstacles, the environment changes the planning problem from fixed obstacles
to moving ones; changes the problem from a geometric one to a dynamic one; equally the
environment changes the problem from a deterministic problem to a stochastic problem [12].
4
Therefore, a vertices based stochastic optimisation method can be applied to the dynamic
environments for robots.
The simulated annealing algorithm is a generic probabilistic meta-algorithm for the global
optimization problem, namely locating a good approximation to the global optimum of a
given function in a large search space. It was independently invented by Kirkpatrick, Gelatt
and Vecchi in 1983, and by Cerny in 1985 [13].
The reasons of choosing the simulated annealing algorithm to plan the shortest path for
robots are listed below:
1) Recently, the genetic algorithm was implemented to find the optimal path for a robot in a
dynamic environment [14]. According to [9], the simulated annealing algorithm is an
optimization algorithm similar to the genetic algorithm, and can provide similar results in
optimization searches. The simulated annealing algorithm has been already used in
searching the optimal path in stationary path planning methods [15], [16] and [17].
Therefore, it is expected to work successfully in planning path for a robot in dynamic
environments.
2) The simulated annealing algorithm based methods use vertices as search space. The
reason to use the vertices as search space is the solution constituted by the vertices of
obstacles can be the optimal result. If the search space is represented by quadtree grids,
suboptimal result will be obtained. The robot can obtain the optimal path in a short time
in a dynamic environment, if the developed simulated annealing based method is
efficient.
3) According to [18], the performance of the genetic algorithm deteriorates greatly as the
problem size increases. However, the simulated annealing algorithm performs better than
the genetic algorithm in this case [18, 19]. Thus, it is worth researching this issue if the
simulated annealing approach could perform better in searching the optimal path in an
environment with large number of obstacles. Implementing the simulated annealing
algorithm in dynamic environment can provide a convincing answer. Until now, limited
work has been reported in the open literature on using the simulated annealing algorithm
in path planning in dynamic environments. Therefore, the research in this area is original
5
and can make useful contribution to the research field.
4) The genetic algorithm uses a population that contains a collection of chromosomes rather
than a single solution, implying that it is more complicated and harder to implement than
the simulated annealing algorithm. The processing time of the simulated annealing
algorithm based approach can be shorter than that of the genetic algorithm based approach.
Compared with the genetic algorithm based approach, the simulated annealing algorithm
based approach may give better trade-offs among simplicity, far-field accuracy and
computational cost in some cases as those in [20].
1.3 Aims of the Project
The research focuses on planning path for robots in dynamic environments. The aims of this
project are summarized below:
To develop and implement a simulated annealing algorithm based approach for path planning
for mobile robots in dynamic environments; the approach is expected to be able to determine
the optimal feasible path quickly for a robot in an environment with dynamic obstacles and
to give a result comparable or better than those results in the previous approach [21], which
is a typical path planning method that uses the vertices of the obstacles as search space.
1) The simulated annealing based approach could quickly determine an alternative path for
the robot if the sensor of the robot detects a moving obstacle. The alternative path
generated should also be the optimal result for the robot. The online processing time is
expected to be shorter than those of previous works.
2) To develop a heuristic method for searching the initial feasible path efficiently and to
incorporate the heuristic method to the developed simulated annealing algorithm based
approach.
3) To incorporate the heuristic method into the simulated annealing algorithm based
approach for further improvement of the performance of the approach.
6
1.4 Significance of the Research
The project contributes novelty in implementing a simulated annealing based approach for
robot path planning in dynamic environment that contains moving obstacles. The simulated
annealing based approach solves drawbacks occurred in some previous methods such as:
1) The simulated annealing based approach takes the shape of every obstacle into calculation
that can provide more precise result than other methods. Some methods like [22], [23] and
[24] ignore the dimension of the obstacles. Obstacles in [22], [23] and [24] are considered
as points or simple square blocks which could generate the repulsive force to robots. The
methods are not suitable for optimal path searching in an environment with a variety of
sharp obstacles. For searching the optimal path, the dimensions of the obstacles cannot be
ignored in such a situation. In fact, most of the published researches that use artificial
potential field neglect the dimensions of the static and dynamic obstacles. This
simplification will generate imprecise route for the robot, and the optimal solution is hard
to obtain in the artificial potential field based method. The simulated annealing based
approach takes the vertices of any different sharp obstacles as search space, therefore the
approach takes the shape of the obstacles into account that overcomes the drawback of the
artificial potential field based method.
2) Compared with some methods like [11], [25], [26], [27] and [28], the simulated annealing
based approach reduce the calculation consumption of updating the dynamic environment.
The methods in [11], [25], [26], [27] and [28] focus on dynamic uncertain environments,
in which the change of the environment caused by the absence of obstacles or the
appearance of the unexpected obstacles detected by the sensor of the robot. D* based
methods are used to re-plan path in dynamic environments where the unexpected obstacle
appears or disappears caused by the incorrect information. However, the obstacles
described in the environments are not purely dynamic with speed and moving directions.
Compared with the grids representation methods, it is easier to use vertices to represent the
moving obstacles. Updating the positions and directions of the vertices of the moving
obstacles can be easier than updating every grids in the moving obstacles. In this project,
7
the simulated annealing based method is proposed to do one-time online calculation to
generate an alternative optimal path for robot once a moving obstacle is detected.
According to [29] and simulation result of [27], the computational time of the searching
process in grids based method increases exponentially with the number of the vertices in
the map. The large number of vertices in the map may affect the performance of the D*
based methods. Therefore, the simulated annealing based approach can give a better
performance than grids based methods in dynamic environments with moving obstacles.
3) The simulated annealing based approach provides a more efficient result than previous
approaches which also use the vertices as a search space for path planning. The
performance of the new simulated annealing based approach is all-better than the previous
method [14] and [21] in calculating the optimal path for robot. As the method is able to
quickly determine the optimal path for robots in dynamic environments, it could be used in
industrial research robots such as [3] and [2]. Also, the heuristic method to search initial
path in the simulated annealing based approach in dynamic environments can greatly
improve the performance of the simulated annealing based approach in calculation time
and convergence in online and offline calculation. The heuristic method could also be used
in other optimisation methods which need to generate initial solutions for calculations.
1.5 Contributions of the Thesis
With the completion of the project, the aims of the research have been fully achieved, and an
integrated framework has been successfully developed for robot path planning in dynamic
environments. Main contributions of the Thesis are:
1) Main contribution 1: The simulated annealing algorithm based approach to search
feasible path for robot in static environments with improved performance over existing
methods. Genetic algorithm based path planning method in [21] is a typical path planning
method that uses the vertices of the obstacles as search space and it is the fundamental
algorithm used by [14] in dynamic path planning. Because the genetic algorithm based
method used in [14] and [21] uses the same algorithm mechanism, and the mechanism is
8
stated clearly in [21], so the performance of the simulated annealing algorithm based
approach is compared with that of the genetic algorithm based approach in [21]. The
calculation time for obtaining the optimal path is all-better than genetic algorithm based
approach in [21]. The simulated annealing based approach successfully generates
collision free path for the robot in dynamic environments. The simulated annealing based
approach can quickly determine an alternative path for the robot if the sensor of the robot
detects a moving obstacle. The alternative path generated is also the optimal result for the
robot in the dynamic environment.
2) Main contribution 2: A heuristic selection method for selecting the initial feasible path
more efficiently. It can be used in different types of optimisation methods. Compared
with the random picking method, the heuristic method uses a more intelligent way to
search the initial feasible path for robots. The heuristic method is incorporated into the
simulated annealing algorithm based approach to further improve the efficiency of the
approach. The simulation results show that the new heuristic selection method can greatly
improve the performance of the simulated annealing approach in searching the optimal
path in dynamic environments.
1.6 Structure of the Thesis
The following are the descriptions of the structure of the thesis. Chapter One states the main
issues of the thesis, including a general introduction to the problem and aims and outcomes
of the research. Chapter Two provides detailed literature review and background information
for robot path planning. As one of the main contributions of this thesis, Chapter Three
discusses the implementation details of the simulated annealing based approach. It illustrates
the methodology of the simulated annealing based approach to calculating the optimal path
for a robot in dynamic environments. In Chapter Four, the simulation results of the approach
are compared with the genetic algorithm based approach to test the efficiency of the
simulated annealing based approach. As another main contribution, Chapter Five proposes a
heuristic method to search the initial feasible path. The method is incorporated into the
simulated annealing based approach to further improve the performance of the approach.
Simulation results are also given to demonstrate the effectiveness and high efficiency of the
9
approach with integrated heuristic initial path search method. Chapter Six concludes the
thesis, and briefly discusses future directions on the research topic.
1.7 Related Publications
The following papers are published or in preparation in support of this thesis:
(1) “Robot Path Planning in Dynamic Environments using a Simulated Annealing based
Approach”, accepted for presentation at IEEE 10th International Conference on Control,
Automation, Robotics and Vision (ICARCV 2008), Hanoi, Vietnam, Dec 2008.
This conference is ranked by Core (www.core.edu.au) as a Tier A+ conference in 2007
and a Tier A conference in 2008.
(2) A Journal paper “Path Planning in Dynamic Environment Using a Heuristic Simulated
Annealing Approach” is being prepared to be submitted. It reports the outcomes of the
entire Masters research project.
10
11
Chapter 2 Background and Literature Review The mobile robot path planning task is to find a collision-free route, through an environment
with obstacles, from a specified start location to a desired goal destination while satisfying
certain optimization criteria [30]. This chapter classifies various robot path planning methods
in different ways and gives some general information about traditional path planning
methods in different environments such as the Visibility Graph method, Grid method, and
Artificial Potential Field method.
2.1 Classifications of Robot Path Planning Methods
Path planning for mobile robots is one of the most important aspects in robot navigation
research. The robot path planning methods could be classified into different kinds based on
different situations. Depending on the environment where the robot is located in, the path
planning methods can be classified into the following two types:
12
(1) Robot path planning in a static environment which only contains the static
obstacles in the map; and
(2) Robot path planning in a dynamic environment which has static and dynamic
obstacles in the map.
Each of these two types could be further divided into two sub-groups depending on how
much the robot knows about the entire information of the surrounding environment:
(1) Robot path planning in a clearly known environment in which the robot already
knows the location of the obstacles before it starts to move. The path for the robot
could be the global optimised result because the entire environment is known.
(2) Robot path planning in a partly known or uncertain environment in which the robot
probes the environment using sensors to acquire the local information of the
location, shape and size of obstacles, and then uses the information to proceed
local path planning.
Fig 1 shows the hierarchy of the classifications of the robot path planning methods:
Fig 1: Classifications of the robot path planning methods.
13
2.2 Path Planning in a Static and Known Environment
In a static and known environment, the robot knows the entire information of the
environment before it starts travelling. Therefore the optimal path could be computed offline
piror to the movement of the robot.
The path planning techniques for a static and known environment are relatively mature.
Representative path planning methods for a clearly known static environment include the
Visibility Graph method [4], Voronoi diagrams method [5], the Grids method [6], the Cell
Decomposition method [31], and the Potential Field method [32].
Moreover, the genetic algorithm, the simulated annealing algorithm, and some other
optimisation methods have also been used to obtain the optimal path for the robot. Davidor
[33] developed a tailored genetic algorithm with a modified crossover operator to optimize
robot path. Nearchou [29] used the number of vertices produced in visibility graphs to build
fixed length chromosomes in which the presence of a vertex within the path is indicated by
setting of a bit at the appropriate locus. The method applied a reordering operator for
performance enhancement, and the algorithm was capable of determining a near-optimal
solution. Cai and Peng [34] developed a fixed-length decimal encoding mechanism to
replace the variable-length encoding mechanism and other fixed-length binary encoding
mechanisms used in the genetic approach for robot path planning.
2.2.1 The Visibility Graph Method
In this method, a visibility graph is used in robot motion planning when the geometry of the
environment is known. The main idea of the visibility graph method is that if there is a
collision-free path between two points, then there is a polygonal path that bends only at the
obstacles vertices [31]. As Fig 2 shows, collision-free path (in curves) could be transformed
into line segments (straight line).
14
Fig 2: Visibility graph.
A visibility graph is constituted by nodes and edges. Nodes are the start point, destination
point and the vertices of all obstacles. Edges are strait-line segment between two nodes
which do not path through obstacles. Fig 3 shows the complete visibility graph base on Fig 2.
Fig 3: Complete visibility graph.
The following is the pseudo-code for building a complete visibility graph [31]: Output: G
(visibility graph)
1: for every pair of nodes u, v
2: if segment(u,v) is an obstacle edge then
3: insert segment(u,v) into G;
4: else
5: for every obstacle edge e
6: if segment(u,v) intersects e
7: goto (1)
15
8: insert segment(u,v) into G
Fig 3 shows that there are multiplex paths that could lead the robot from the start point to the
destination. Then, any optimisation algorithms such as the genetic algorithm [35] and the
simulated annealing algorithm [36] could be used to calculate the optimal path for the robot.
The defect of the visibility graph method is that the efficiency of the algorithm is low. From
the pseudo-code for building a complete visibility graph, it can be seen that the visibility
algorithm requires O(n3) time. Furthermore, the obtained path is often very close to obstacles
and thus, may lead to crashing of the robot. However, the problem can be fixed by enlarging
the obstacles by a value according to the dimension of the robot. In this way, the robot can
approaches obstacles without collision.
2.2.2 The Quadatree Method
The Quadatree method is another algorithm for searching the collision free path for a robot.
It uses small non-overlapping grid cells to represent the entire environment. The cells usually
are simple squares. There are three types of cells: empty cell, mixed cell and full cell. An
empty cell is a free space, where the robot could go through, in the environment. A mixed
cell contains obstacles and free space. A full cell is the block of the obstacles. In a
two-dimensional map, a Quadtree is used to decomposition the map, as shown in Fig 4:
Fig 4 [16]: Quadtree decomposition.
16
The cell decomposition method is briefly outlined below [31]:
1. Decompose the map into cells.
2. Search for a sequence of mixed or free cells that connect the initial and goal positions.
3. Further decompose the mixed cells.
4. Repeat (2) and (3) until a sequence of free cells is found.
Then, the method uses an optimisation algorithm, such as A* algorithm, to find the optimal
path for the robot. The A* algorithm [37] is first described early in 1968 by Hart, Nilsson and
Raphael. The algorithm is a best-first, tree search algorithm, and could find the shortest path
from the start point to the destination point. Lingelbach in [38] used the cell decomposition
method combined with probabilistic sampling to plan path for a robot in high-dimensional
static configuration spaces. In [39], a combined artificial potential field and probabilistic
sampling cell decomposition method was proposed. The potential field approach based on
harmonic function is computed over a non-regular grid decomposition of a high-dimensional
space obtained with the probabilistic sampling of cells.
Fig 5 [40]: Cell decomposition
2.2.3 The Artificial Potential Field Method The potential field method was initially proposed by Khatib in 1986 for mobile robot path
planning. The main idea of the method is to imagine that all obstacles can generate repulsive
force to the robot, while the destination point has attractive force to the robot. The robot
applies a force proportional to the negated gradient of the composition of the attractive and
repulsive force. The position and direction vector [ ], TX x y= of robot are fixed on by
composite of attractive force and repulsive force.
17
The attractive potential field function is given by
( ) ( )21 ,2att gU X k X Xρ=
k : positive scaling factor X : position of the robot
gX : goal of the robot
( ), g gX X X Xρ = − : distance from robot to goal Attractive force Fatt is negative grads of the attractive potential field function:
( ) ( ),att att gF U X k X Xρ= −∇ =⎡ ⎤⎣ ⎦ The repulsive potential field function is described by
( ) ( ) ( )
( )
00
0
1 10.5 ,,
0 ,
oorep
o
X XX XU X
X X
η ρ ρρ ρ
ρ ρ
⎧ ⎛ ⎞− ≤⎪ ⎜ ⎟⎜ ⎟= ⎨ ⎝ ⎠
⎪ >⎩
η : positive scaling factor ( ), oX Xρ : the shortest distance between the robot and obstacles
Constant 0ρ : distance of influence imposed by the obstacle; its value depends on the
condition of the obstacle and the goal of the robot, and is usually less than half distances
between the obstacles or shortest length from the destination to the obstacles.
When the robot is not at the goal, the repulsive force is
( ) ( )
( ) ( ) ( )
( )
02
0
1 1 1 ,, ,
0 ,
rep rep
oo o
o
F X U X
X XX X X X
X X
η ρ ρρ ρ ρ
ρ ρ
⎡ ⎤= −∇ ⎣ ⎦⎧ ⎛ ⎞
− ≤⎪ ⎜ ⎟⎜ ⎟= ⎨ ⎝ ⎠⎪ >⎩
The resultant force is att repF F F= +
F navigates the movement of the robot [23]. This is illustrated in Fig 6:
18
Fig 6: Potential field.
The artificial potential field method provides a simple yet effective method to plan paths for
a robot. However, it also has some drawbacks. The major problem is that robots are often
trapped into a local minimum before reaching the destination. Nowadays, the artificial
potential field method is combined with many other computational methods to improve its
efficiency. Park and Lee [41] have integrated the artificial potential field method with the
simulated annealing algorithm to escape local minimum. Zhu, Yan and Xing [15] also use the
artificial potential field method in combination with the simulated annealing algorithm for
similar scenarios. In addition, as will be discussed later, the potential field method is also
used in planning path in dynamic environment, as demonstrated in [8] and [23].
2.3 Path Planning in Static and Unknown Environment
In a static and unknown environment, robot navigation is more difficult than that in static and
known environment. This is due to the uncertainty of the information in the environment,
such that the optimal result is difficult to obtain and the robot has to use local information to
calculate the path. Some path planning methods in static and unknown environment will be
reviewed below.
19
In Ying and Eicher [42], a sensor-based path planning method was proposed to help
underwater robotic vehicles perform real-time path planning in a static and unknown
environment. The environment is constituted by various shaped static obstacles. A
three-dimensional path planning algorithm is developed for the robot in such an environment.
Using sonar to detect the static obstacles in the environment, the robot is controlled by a
group of thrusters. The simulation results show that the on-line path planning method can
plan a collision free path for the robot. The calculation time in the simulations is acceptable
for real-time online planning application.
In [42], the relationships among the robot position, vision zone, obstacles and target position
are described as the following situations:
1) No obstacle is inside the vision zone of the robot, shown as Fig 7:
Fig 7: Obstacle is out of the vision zone.
2) An obstacle partially occupies the vision zone but it does not impede the main line VTuuur
between robot and target. This is shown in Fig 8.
20
Fig 8: Obstacle partially occupies the vision zone.
3) As shown in Fig 9, obstacle partially occupies the vision zone and it impedes the main
line VTuuur
between the robot and the target.
Fig 9: Obstacle impedes the main line between robot and target.
4) Obstacle fully occupies the vision zone, as shown in Fig 10:
Fig 10: Obstacle fully occupies the vision zone.
21
Several path planning rules are used to plan path against above mentioned situations:
(1) Path planning rule 1: If no obstacles exist inside the vision zone or the obstacles
partially occupy the vision zone but do not impede the main line VTuuur
, the robot will
follow the main line VTuuur
to the target.
(2) Path planning rule 2: The best gate rule. If an obstacle partially occupies the vision
zone and it impedes the main line VTuuur
between robot and target, then the robot
search for the best path based on the currently known information. The path is
obtained by the current robot position and the edge point of intersected surface
produced by obstacle surface and vision zone of the robot. The best gate A and the
shortest VATuuuur
are selected, which can be determined by ( )MIN VA AT+uur uuur
. The
following figure demonstrates that:
Fig 11: Path planning rule 2.
(3) Path planning rule 3: The possible shortest path rule. If Obstacle fully occupy the
vision zone, the current location of the robot V, the target position T and the puncture
point G are used for calculating the feasible path:
a): Find puncture point G produced by the VTuuur
and the intersected surface of
the obstacle.
b): Find the minimum gradient direction PGuuur
at the puncture point P on the
intersected surface. G is on the edge of intersected surface.
22
c): The path VGTuuuur
= ( )MIN VG GT+uuur uuur
.
Fig 12: Path planning rule 3.
The path planning algorithm applies each planning rule against each situation robot
encounter until the robot reaches the destination point T.
In [43], Park and Lee proposed a modified virtual hill concept based on the artificial potential
field method to escape local minimums in the local environment. A virtual hill with extra
potential is added at the trapping obstacle when the robot is trapped in a local minimum
position. The extra potential is added to the global potential, which can repel the robot from
the local minimum. However, due to the insufficient information of the obstacles in the
environment, the global optimization result can not be acquired and only the local optimal
result is attained. Recently, Chang and Yamamoto [44] combine the Voronoi diagram
approach with the potential field method to avoid dead-lock problem in the potential field
method. The problem is solved by defining necessary sub-goals between targets on the
constructed map. Xie and Chen [45] use a virtual water-flow concept combine with potential
field method to avoid the local minimum problem in potential field method. Vanualailai and
Sharma [46] developed an asymptotically stable point-mass system to avoid the local
minimum in artificial potential field method. The Lyapunov function in the asymptotically
stable system produces artificial potential fields around the goal and the obstacle, has no
local minimum other than the target.
23
In [47], Wan, Chen and Earnshaw used a robot version hypothesis similar to that in [42] and
proposed an A* based path finding algorithm. The method allocates the points of visibility
dynamically and decentralise the view angle in each step to build state node. Paper [48]
proposed a more practical method using a virtual window in the front of the robot to
calculate the space ahead and any intersections that are done. The robot acquires a
two-dimensional image of the space ahead and processes the information to find a collision
free path. The simulation results show that path planning can be successfully implemented
using the virtual window method and that the computational time limitation can be
overcome.
In [49], an artificial immune network based path planning method is proposed. The method is
capable of obtaining an optimal or near-optimal collision free path in unknown environments
that is presented by grids. According to the simulation results, the method can give a better
performance on complex environments than the Knowledge-based Genetic Algorithm [58]
does.
Nowadays, the path planning problem is still a favorite research topic in robotic and artificial
intelligence studies. The path searching for robot in unknown static environment will still be
study in the future.
2.4 Path Planning in Dynamic and Known Environment
Research on methods that deal with the static environment path planning has been introduced
in previous sections. Currently, the path planning methods to find paths in static environment
have been well developed with hundreds of published papers. Given the entire information of
the environment, the global optimal or near-optimal result path could be found by these
algorithms, such as the genetic algorithm [50].
However, in practical applications, robots such as those described in [2] and [3], often face
obstacles that are not all static in the environment; the status and the movement of the
obstacles change continuously in the map. Moving obstacles in a dynamic environment
24
increase the difficulty of planning path for the robots in the map.
Unlike the situation for path planning in a static environment, limited reports have been
found in the open literature to discuss optimal path planning in dynamic environments.
Complexity and uncertainty increase with the number of the dynamic obstacles. Therefore,
traditional path planning algorithms, such as the visibility graph, the voronoi diagrams and
the grids method, do not perform well in dynamic environments. It is also difficult to gain the
optimal path for the robot using these methods. Robot path planning in a dynamic
environment is thereby an issue for further research. In a dynamic environment, how to
manipulate the robot so that it can travel to the destination safely and optimally without
collision is an important issue of concern.
In 2005, Chestnutt, Lau and Cheung [51] used a modified A* algorithm to calculate path for
a Honda ASIMO humanoid robot. The path planning method is applied to real robots rather
than simulated on software. A grid of cells is employed to represent the environment. Colour
cells represent the obstacles. The cells create a bitmap representing the free spaces and
obstacles in the map. The algorithm plans a sequence of footstep positions to navigate toward
a goal location based on known static and moving obstacles with predictable trajectories. It
uses three cost functions to constrain the step nodes used by the robot.
(1): The first cost is the location cost, which evaluates the footstep location with regard
to the environment to determine if the location is safe place to step.
(2): The second cost is the step cost, which calculates the cost to the robot to make the
desired step.
(3): The last cost is the estimated remaining cost-to-go, which is calculated by planning
backwards from the goal with a standard mobile robot planner as a precomputation
step. This cost is used to avoid the local minimums.
Wang and Sillitoe [14] proposed a vertices genetic algorithm planner in 2007. The planner is
able to rapidly determine optimal or near-optimal solutions for a mobile robot in an
environment with moving obstacles. The method uses the vertices of the obstacles as search
space and produces off-line path planning through the environment with dynamic obstacles.
25
It firstly incorporates the robot speed into the genetic genes, which could optimise both the
travel time and distance of the robot. Before the robot starts movement, the complete motion
knowledge of the moving obstacles in the observed region is available for the robot. The
robot uses the genetic algorithm based planner to calculate the time or distance-optimised
solution and then starts to travel. The assumptions of this method are:
• Obstacles are bounding polygons with vertices.
• The speed of the moving obstacles is fixed value, and
• Physical dimensions of the robot are neglected and regarded as a single point.
The genetic representation of the method is outlined in Fig 13:
Fig 13: The representation of the chromosome in GA planner.
As shown in Fig 13, the possible solution paths are represented by a chromosome which is
constituted by a bunch of genes. The minimum number of genes for a chromosome is two
corresponding to the start point and the end point, and the maximum number of genes is thus
the total number of whole vertices N plus the start and the end points.
Each gene contains four parts with the first two being the x and y coordination positions of
the vertex. The third part is the robot’s speed in the segment of each gene. The robot speed is
selected from a group of predefined discrete speeds. The last part indicates the feasibility of
the path that originates from the genes; if the path segment connecting two consecutive
vertices intersects one or more obstacles, then the infeasibility bit of the gene representing
the originating node is assigned 1 to mark the infeasibility of this segment. The evaluation
functions have two types: the path length and the travel time.
26
Four genetic operators are used in the method; they are crossover operator, mutation operator,
repair operator and robot speed mutation operator. Fig 14 illustrates the first three genetic
operators.
Fig 14: Genetic planners.
The generational operation begins with the random selection of a genetic operator and a
quadratic ranking scheme is used to retain the constant selection differential after evaluation.
The parent (or parents for the crossover operation) involved in the genetic operation is
determined by a roulette wheel whose slots are sized in proportion to the fitness as scaled by
a ranking technique. To form a new generation, the newly generated offspring replace the
worst individual (or pair of individuals if crossover is applied) in terms of fitness in the
existing population. The evolutionary process continues until a termination condition is
satisfied, which can be defined to be a number of generations specified by the user or
determined by monitoring against a specified performance criterion. When the evolution
terminates, the best individual is selected as the path planning solution. The simulation
results show that the planner changes in robot speed significantly improves the efficiency of
the method especially in time management for robot in dynamic environment.
A hybrid navigation method [52] was proposed in 2003. The method consists of two modules:
global and local modules, which could combine two robot path methods to deal with the
global map information and local sensor information.
27
(1) The global module specifies the global route positions. It uses prior information on
the navigation environment and chooses critical points to pass through before actual
navigation takes place. This module uses the A* algorithm to determine the optimal
route to the specified goal position. Using the algorithm, it is possible to find the
optimal route to the goal.
(2) The local module carries out the navigation itself, relying on current sensor data,
thus making it easier to avoid static or dynamic obstacles. It uses a fuzzy neural
representation of the potential field based navigation method.
Firstly, the global planning module finds the optimal route to the goal and proposes the
positions to pass through as intermediary points. These intermediary points are then passed
one by one to the local navigator, which makes the robot reach them while reactively
avoiding the obstacles present in the environment, according to the potential function
previously supplied to the local navigator.
A dynamic environment is more complicated than a static environment in the robot path
planning issue. Several methods were proposed to solve the problem. Because the moving
information and the obstacle information can be known in advance of movement
commencing, the optimal solution can still be obtained.
2.5 Path Planning in Dynamic and Unknown Environment
Path planning in dynamic and unknown environment is the most complicated case in robot
path planning, and is also the most common situation that mobile robots will confront. In the
real world, mobile robots such as undersea unmanned vehicles often need not only to avoid
static rocks in the seabed, but also to avoid colliding with the large sea lives. Due to the
complicated and unknown environment, the robot cannot adopt one time global path
planning for the environment. The global optimised solution is thereby difficult to be
obtained. The robot has to use sensors acquiring the information of surrounding environment
and do online real-time path planning. The planning time for the robot should be short
28
because the robot needs a sufficient time interval to adjust its movement in order to avoid the
coming obstacle. In next few paragraphs, recent research in this area will be reviewed.
In 2006, Lv and Feng [53] introduced a numerical potential field for finding the path in a
dynamic environment for a robot. This research used an ant colony optimisation algorithm to
perform optimal path searching. The movement of the obstacles is taken into considerations
by changing the local potential values. The method firstly uses dynamic valued potential
field to model the dynamic environment, then employs an ant colony optimisation method to
search the path for local path planning.
In 2007, Wang, Sillitoe and Mulvaney [14] introduced a genetic algorithm planner to
determine optimal or near-optimal solutions for a mobile robot in dynamic environments.
The obstacles are described as bounding polygons and the robot dimension is neglected and
regarded as a point. The method firstly uses offline planning to generate a path for the robot,
then the robot starts to travel with its sensors. The online path planning method is activated
to calculate an alternative path for the robot if obstacles are detected. Because of the
complexity and uncertainty involved, very limited work has been conducted to develop
complete solutions for a purely uncertain dynamic environment.
Furthermore, implementing path planning algorithms in real robots in dynamic cases is also
very limited. Cao, Huang and Zhou [23] provided an evolutionary artificial potential field
method for dynamic path planning for soccer robots in RoboCup 2005. The new potential
field method is intended for path planning of mobile robots in a dynamic environment where
the target and obstacles are moving. Firstly, the new force function and the relative threat
coefficient function are defined. Then, a new potential field path planning algorithm based
on the relative threat coefficient is presented. Finally, computer simulation and experiment
are used to demonstrate the effectiveness of the dynamic path planning scheme [32].
Zheng and Zhao [54] proposed an artificial potential field based approach in 2006. The
method could handle path planning for five mobile robots in a dynamic unknown
environment. The robot will pass the intersection area in order according to their priority,
29
which is predefined before movement. The method assumes that the obstacles in the map
have identical sizes. It successfully coordinates the movements of multiple robots, and makes
the robots pass through the static obstacles and other robots without collision. The novelty of
the research is the application of the artificial potential field method to multi-robot path
planning in dynamic environment.
Tarokh [55] develops a hybrid intelligent approach to path planning for high mobility robots
operating in rough environments. Path planning consists of characterization of the
environment using a fuzzy logic framework, and a two-stage genetic algorithm planner. A
global planner determines the path that optimizes a combination of terrain roughness and
path curvature. A local planner uses sensory information, and in case of detection of
previously unknown and unaccounted for obstacles, performs an on-line re-planning to get
around the newly discovered obstacle.
Hu and Gu [56] developed an improved algorithm to solve the problem of optimum route
planning in vehicle navigation systems. The algorithm is based on the standard genetic
algorithm and the lambda-interchange local search method and has evolved from the
improved A* shortest-path algorithm. The algorithm can find the optimum route more
efficiently and quickly without any network constraint condition and can work well either in
continuous or in discrete networks.
In [57], potential field method is applied for both path and speed planning for a mobile robot
in a dynamic environment where the target and obstacles are moving. The robot’s velocity
and trajectory are determined by the relative speed and moving directions of the obstacles
and target. The simulation results verify that the method can efficiently tracking the moving
target while avoiding the obstacles along its path.
The D* (Dynamic A*) [11] method is a typical method used in path planning in dynamic
unknown environments. It plans optimal traverses in real-time by incrementally repairing
paths to the robot’s state as new environment information is known, which makes it possible
to greatly reduce the computational cost. When the robot gathers new information about the
30
environment, it will replan new paths based on the new information and produce a path for
the robot. D* does not require complete re-planning of the path every time when new
information comes in. It updates the path arc cost locally when environment changes.
However, D* still try to obtain the globally optimal path when possible.
Like A*, D* operates on a cost graph. The environment with the obstacles is represented by a
uniform grids map as in Fig 15. The main idea of the method is illustrated as following:
From the initial state, the method repeatedly selects the neighbour with the minimum cost
until propagates to the goal. Each small cell in the map is called state. Each state X has the
arc cost from the state X to the goal given by the path cost function h(X,G). From the start
point (start state), all neighbour states of the current state are listed on the open list. From the
open list, the method calculates the arc cost of the states by h(X,G). Then, select the state
with the minimum h(X,G), go to this state, and new neighbours are added to the open list. As
Fig 16 shows, from the start point, the method repeatedly propagating to neighbours and
selecting the state with the minimum h(X,G) until the goal state is propagated.
In the dynamic uncertain environment, when the robot detects new obstacles or the absence
of expected obstacles, the cost values of the states in the area change. And the adjoining
states are put on the open list for cost correction. Encountering unexpected obstacles, D* will
set off a “raise” wave, a wave of increasing cost, through neighboring states. When this wave
meets with the states that are able to lower path costs, these “lower” states are put on the
open list to recalculate new optimal paths. When it detects the absence of an expected
obstacle, the arc of the path passing through this “missing” obstacle is assigned a small cost,
denoting an empty space, and the adjoining state is put on the open list as a lower state,
setting off a “lower” wave, a wave of decreasing cost. D* is able to determine how far the
raise and lower waves need to propagate while providing the optimal path for robot traverse
continuously [26].
The basic D* method was originally proposed in 1994. However, the cost change
propagation method used in basic D* does not consider which expansions will benefit the
robot at its current location. Thus, the performance of the algorithm [25] can be affected.
31
Therefore, like A*, D* can use heuristic function to focus on the search in the direction of
the robot and reduce the total number of the state (grids) expansion. Focussed D* method [26]
uses the focussing heuristic function to estimate the estimated path cost from the current
location to the goal to help the robot to minimise its search space. In comparison with Fig 16,
Fig 17 shows that with focussed D*, robot uses fewer grids in search the optimal path.
Fig 15: Regular grids representation.
Fig 16: D* Algorithm [25]
32
Fig 17: Focussed D* Algorithm [25]
To enhance the performance of the D* algorithm, framed-quadatree D* [27] and Field D*
[28] methods were proposed.
• The Framed-quadatree method uses quadatree structure to represent the
environment. It uses different dimensions of grids to represent the environment in
order to minimize the search space, and then adding the border-cells to connect
each grid, as shown in Fig 18. The framed-quadatree structure can reduced the
search space for the robot and also the framed-quadatree uses border-cells to over
come the disadvantage of the suboptimal result in using the quadatree structure.
However, because the method has to create more sub-cells for calculation, the
performance of the framed-quadatree method deteriorates greatly as the fractal gain
increases, especially in offline planning [27].
• The Field D* method [28] uses an interpolation-based planning and re-planning
algorithm to generate smooth paths for robots through non-uniform cost grids. It
uses linear interpolation during planning to calculate accurate path cost estimates
33
for arbitrary positions within each grid cell and to produce paths with a continuous
range of headings. The method can produce a smooth optimal path for a robot to
over come the suboptimal problem in using non-uniformed girds method. However,
the method scarifies the performance for the accuracy. It reduces the path length of
the robot in offline and online planning but takes 1.8 times longer in calculation
time than previous D* method [28].
Fig 18: Framed-quadatree structure.
Recently, a grid-based distance-propagating dynamic system is proposed in [58]. The
algorithm is similar to the D* algorithm, but it does not maintain a sorted queue of points to
update. The algorithm uses only local information when computing an update at each point,
and the order of updating is predetermined. This makes the algorithm exceedingly easy to
parallelize by simply assigning a subset of points to each processor. Consequently, in
situations where obstacles and targets are moving at substantial distances from the current
robot location, the grid-based distance-propagating dynamic algorithm is more efficient than
the D* algorithm.
Until now, most research activities on robot path planning in dynamic environments have
been theoretical in focus. Some methods such as those in [11], [25], [26], [27] and [58] are
34
proposed to deal with the issue of robot path planning in uncertain environment. However,
due to the complication of dynamic environment, the methods still have limitations
concerning their assumptions, algorithm efficiency and other issues:
1) The methods in [11], [25], [26], [27], [28] and [58] focus on dynamic uncertain
environments, in which the change of the environment caused by the absence of obstacles
or the appearance of the unexpected obstacles detected by the sensor of the robot. The
robot uses D* or algorithm similar to D* based methods to re-plan path in dynamic
environments where the unexpected obstacle appears or disappears caused by the
incorrect information. However, obstacles described in the environments are not purely
dynamic with speed and moving directions. Compared with the grids representation, it is
easier to use vertices to represent the moving obstacle. Updating the positions of the
vertices of the moving obstacle can be easier than updating every grids in the moving
obstacle. In this project, a simulated annealing based method is proposed to do one-time
online calculation to generate an alternative optimal path for robot once a moving
obstacle is detected. According to [29] and simulation result of [27], the computational
time of the searching process in grids based method increases exponentially with the
number of the vertices in the map. The large number of vertices in the map may affect the
performance of the D* based methods. Therefore, it is worth implementing efficient
vertices based simulated annealing approach for robot path planning.
2) The efficiency of some methods such as that in [14] is not practically acceptable when
applying the method in real-world systems. The methods assume that in order to allow
the robot to be guided so as to avoid any potential collisions with obstacles, there is an
adequately large time interval between the detection of obstacle movements and the
implementation of new generated actions. When dealing with robot path planning in
complex environments, genetic based approach is time consuming. According to the
simulation result of [14], it requires nearly 30 seconds to find the first feasible path for
the robot in an environment with 14 static obstacles and 5 dynamic obstacles. It seems
not practical applying the approach to real robot application.
35
Addressing those drawbacks, this project develops and implements a new approach for more
efficient robot path planning in dynamic unknown environments. The proposed approach
could provide a near or better result than previous solutions like [14] and [21]. The algorithm
is able to quickly determine the optimal path for a robot in dynamic unknown environments.
The robot uses sensors to gain the information of moving obstacle, and compounding it with
the information of static obstacles to acquire the optimal path for the robot in dynamic
environments.
With the time limitation of this research, we will use a robot and environment simulator to
investigate the robot path planning problem, and, implement our algorithms in the simulator.
In this project, we use Matlab to simulate the robot with sensors and the unknown dynamic
environments. For simulating the algorithms, Matlab could be the suitable application,
because of its sophisticate math toolbox. In this paper, the simulation results show that the
approach can give a better performance in path planning than [21], and takes the dimensions
of the static and dynamic obstacles into account comparing with the artificial potential base
methods. The simulated annealing based approach with heuristic search method improves the
searching algorithm in finding the optimal solution in dynamic unknown environments. The
detailed methodology of the approach will be explained in Chapter 3.
2.6 The Simulated Annealing Algorithm and SA Based Approach
2.6.1 The Simulated Annealing Algorithm
The simulated annealing algorithm is a generic probabilistic meta-algorithm for the global
optimization problem, namely locating a good approximation to the global optimum of a
given function in a large search space. It is widely applied in many industrial fields. The
well-described application of the simulated annealing algorithm is for the Traveling
Salesman Problem (TSP) [59]. The TSP problem is that given a number of cities and the cost
of traveling form one city to another, to get the least cost round trip to visit each city exactly
once and then return to the start city.
36
The simulated annealing algorithm is analogous to metal physical cooling process, with each
step of the simulated annealing algorithm replacing the current solution by a random
"nearby" solution. The probability of choosing the random "nearby" solution depends on the
difference between the corresponding function values and on a global parameter T (called the
temperature) that is gradually decreased during the process. The dependency is such that the
current solution changes almost randomly when T is large, but increasingly "downhill" as T
goes to zero [60]. Fig 19 is the flow chart of the general steps of the simulated annealing
algorithm:
37
Fig 19: Procedure of the simulated annealing algorithm.
2.6.2 Simulated Annealing Based Approach
Before this project, the simulated annealing algorithm approach had already been applied to
the robot path planning method to deal with finding path for robot. Zhu, Yan and Xing [61]
proposed a simulated annealing based artificial potential method to deal with the robot path
planning issue in static environments. The artificial potential method is firstly used to map
the environment and find a route for the robot. The simulated annealing algorithm is applied
to the artificial potential method to overcome the local minimum problem that is caused by
obstacles in the map. Zhang and Lu [62] integrate the simulated annealing algorithm into the
artificial potential field method to solve the GNRON and local minima problems. Neachor
[19] used vertices of the static obstacles as search space and implemented the simulated
annealing based approach to find optimal path for robot. The simulated annealing based
approach is compared to genetic based approach in his paper. The performance of the
simulated annealing approach is slightly lower than the genetic approach in his experiment.
There might be two reasons that affect the performance of simulated annealing approach in
Neachor’s model.
(1) The simulated annealing algorithm is more sensitive to the control parameters such
as initial temperature and cooling rate. In Neachor’s research, only one group of
control parameters is set in dealing with three different situations. The performance of
simulated approach could be improved if suitable parameters are set.
(2) The operator that generates the new solutions is too simple. Neachor only used a
swapping operator to generate the new solution. The swapping operator only flips
some bits of the result to generate a new result. In this way, the possibility of jumping
out of the local optimal result is small. The simulation results in this Thesis proves
that adding more algorithm operators to generate the new result gives the simulated
annealing approach better performance. In this thesis, more than one operator were
38
used, the simulation results show that the performance is improved by the
multi-operator in the approach. The detailed approach description will be discussed in
Chapter 3.
Path planning in a dynamic unknown environment is the most common case in the real world.
Therefore, a more effective robot path planning method should be proposed, which can
satisfy the accuracy and the efficiency simultaneously. Our project is trying to apply the
simulated annealing method to quickly produce online calculation and efficiently defines the
optimal path for robot in a dynamic environment.
2.7 Robot and Environment Simulators
Implementing a new method on real robots is not only complicated and time consuming, but
also expensive. For the purpose of this Master research project, we use software simulators to
model both the robot and the environment. Then, the developed approach is implemented in
the simulators. There are many kinds of robot and environment simulators, such as multiple
mobile robot simulators EyeSim [63] and Player/Stage [64]. The following is an introduction
to the Player/Stage appliance.
The Player/Stage Project is an open source project for multiple, distributed robots control and
sensor network system, in which Player provides as robot device server. Stage works as a
multiple robotics simulator, plus some supporting tools. Since that Player/Stage can work
under different platforms and be easily programmed, they have been adopted, modified and
extended by researchers all around the world. In fact, Player can actually control and work in
many popular robot devices, such as Pioneer 2-DX mobile robot and etc [64]. The
Player/Stage project is carried out in Robotics Research Lab of University of Southern
California. Player is an open sourced middleware server that could run on variety of robots,
sensors and actuators. It hides the hardware detail of robots for users (like an operating
system on robots). Users could write their own control program at client to control the robot
through TCP channel.
39
Fig 20: Structure of Player
Researchers do not need to access to the real hardware and environments when they are
focusing on Player and robot research. Stage is used to simulate different kind of robots,
sensors and object for the Player. It provides two-dimensional bitmapped environments, and
also could let research to do experiments with novel devices that do not exist (By adding
library files). Stage simulates a population of simulated robots and sensors operating in a
two-dimensional bitmapped environment. The devices provided by the Stage are accessed
through Player, as if they were real hardware. Stage aims to be efficient and configurable
rather than highly accurate. In practice this means that Stage can simulate tens or hundreds of
robots on a desktop PC, and the simulated robots commonly work similarly to the real robots
in the world [65].
Fig 21: Player/Stage architecture.
40
The implementation of building the new simulated annealing approach on Player/Stage will
not only demonstrates the feasibility of approach but also provide a more practical simulated
environment for the new method. The simulated annealing based approach could be tested by
simulated robot hardware, not only exist in theoretics.
The Stage Library (libstage) provides a C code library for simulating a population of mobile
robots, sensors and environments. Stage is one product of Player/Stage project, which is a
simulating tool used in the field of mobile robot and intelligence sensor system. Stage
simulates mobile robot, sensor, obstacles and other objectives in two-dimensional bit map.
Stage could simulate different equipments and models. It could also simulate virtual robots
and sensors that have not yet been developed. Stage could simulate variety of sensors and
executor including sonar, laser scanning distance measurer, splash display, blob gripper,
mobile robot and so on.
Equipments simulated by Stage can be controlled by Player which is a network robot server.
Interface program provided by Player drives equipments like robot and sensor, and Stage
simulates these equipments. The TCP ports for controlling the real robots provided by Player
are identical with the TCP ports to control the simulated models in Stage. Therefore, Stage
could totally provide comprehensive real environments and robots for the Player and its
clients. It is expected that Stage builds the dynamic environment and simulates real robot and
real sensors for the player sever. The simulated annealing approach is implemented in client
machine, the client acquires the useful information from Player server, and then use
simulated annealing approach to acquire the optimal path.
41
Chapter 3 The Simulated Annealing Algorithm Based Approach
This chapter explains the methodological approach in details, including the modeling of
environments, structure of the approach/algorithm, the generation of the initial feasible path,
the new planner for generating the random path and the procedure of online calculation. One
of the aims of this project is to implement a simulated annealing based approach into robot
path planning in a dynamic environment. The simulated annealing based approach is
expected can quickly determine the optimal feasible path for robot in the environment with
moving obstacles. The research uses the mathematic software Matlab to simulate the
dynamic environment, real world robots and sensors. The research then implements the
simulated annealing path planning method in the environment. Therefore, the entire research
project could be separated into the following three phases:
1) Design the algorithms that could find paths and optimising the paths for the robot in
dynamic environments.
42
2) Use Matlab to build mathematic model and obtaining the simulation results proving
the feasibility of algorithm.
3) Compare the results of the new approach with the those from existing methods, and
discuss the performance and the efficiency of the new approach.
.
3.1 Structure and Assumptions of the Method
First, as in the development of many existing methods, some assumptions are made before
designing the algorithm.
1) There are stationary and moving obstacles in the dynamic environment.
2) As a traditional assumption, both the stationary and dynamic obstacles are bounding
polygons.
3) The search space is the vertices of the static obstacles plus the vertices of the detected
moving obstacles.
4) The movement trajectory of the dynamic obstacles is constituted by a series of line
segments, and is constrained by a straight line.
5) Any changes in motion parameter of the dynamic obstacle are immediately available
to the robot, if the obstacle is in the range of the sensor. The time of the robot
obtaining the moving parameter (such as speed and direction) is minimum that does
not affect the path planning calculation.
6) The robot could change its speed and direction at any time.
7) After the sensor of the robot acquires the motion information of the moving obstacle.
It is assumed that a separated module is used to monitor the movement of the
obstacles for the robot. The module will predict the next movement for the robot based
on the previous movement of the obstacle. It will report any change of direction and
speed of the obstacle. The module is separated from the robot, which can be simply
implemented by using extern sensors in real robot implementation.
The designed structure of the simulated annealing algorithm in dynamic obstacles
environments is divided into two stages: off-line calculation for stationary obstacles, and
online calculation when moving obstacles are detected.
43
Stage 1: Offline calculation for the path on stationary obstacle based on the vertices of the
static obstacles. The simulated annealing based approach firstly calculates the
optimised path for the robot base on the positions of the stationary obstacles. Once
the path is ready, the robot starts to travel through the stationary obstacles with the
sensor that could detect the 360 degree direction of the robot.
Stage 2: Online path calculation once the moving obstacles are detected by the sensor. As
moving obstacle enters the detection range of the robot, the sensor will detect the
obstacle and the robot acquire the moving information such as speed and moving
direction of the obstacle from the sensor of the robot. Then the robot uses the
motion information to calculate the possibility of the moving obstacle clashing
with the robot. If it is calculated that the moving obstacle will not collide with the
robot, the robot will use the current path plan to travel through the map. As Fig 22
shows, the sensor of robot calculates out that the L shaped moving obstacle will not
collide with the robot. Therefore, the robot will not change the current path plan. If
the moving obstacle will clash with the robot as Fig 23 shows, the robot will
activate the algorithm again to calculate a new path online for the robot (Fig 24).
Fig 25 shows that state change of the robot in path planning approach in dynamic
environments.
Fig 22: Scenario 1 of moving obstacle.
44
Fig 23: Scenario 2 of moving obstacle
Fig 24: Online path planning
Fig 25: State transformation diagram of simulated annealing approach.
45
In the actual calculation for either offline or online path planning, the algorithm consists of
two procedures:
• Firstly, using randomly selection to find one initial feasible path which could lead
robot from start point to destination.
• Secondly, based on the initial feasible route, using the optimization algorithm to
search for the optimal or near-optimal path for the robot in the static environment. In
this Thesis, the optimization algorithm used for searching the optimal path is a
simulated annealing algorithm.
3.2 Mathematic Modelling
3.2.1 Environment Modelling
The environment is represented by polygons with vertices and edges. The mathematic model
of locating the edges and vertices of the obstacles in the environment is not complicated, use
the model mentioned in Chapter 2 about visibility graph. For realistic simulations, as in [21],
all obstacles in the map are enlarged by a fixed value that the robot could approach obstacles
without collision. The dimension of the robot is neglected, and consequently the robot is
regarded as a single point. In Fig 26, the black polygons represent the static obstacles and the
hollow polygons are moving obstacles. The vertices of the enlarged polygons form the search
space for the robot.
Fig 26: Enlarged obstacles
46
A mathematic model can be developed to determine the possibility of robot colliding with a
moving obstacle. The model is described as follows:
1) The first crossing point between the robot path proposed by the planner and the
trajectory of a moving obstacle is calculated before examining the possibility of
collision;
2) Based on the time t required for the robot to cover the distance from the current
position to the first crossing point, the instantaneous location of the moving obstacle
can be calculated and. Consequently, the exclusion area for this obstacle can be
obtained;
3) If the robot path between the vertex and the crossing point across the edges of the
moving obstacle in odd times, then a collision would occur between the robot and the
moving obstacle, otherwise no collision will happen between the robot and the
obstacle;
3.2.2 Algorithm Structure and Expressions
Traditionally, the path length Ef is the evaluation criteria for the quality of the solution
derived from the algorithm. The shorter the path, the better the solution. A feasible solution is
expressed by a series of vertices linking the start point through to the end point. Each vertex
of the obstacle has its series number. The path for the robot is represented by a sequence of
vertex numbers. Thus, the feasible solution X is given by:
X = {Vstart, Vstart+1, Vstart+2…Vend-1, Vend};
The evaluation function Ef is given by:
Ef = 1
1( , )i endi i
i startD V V= −
+=∑
Where D(Vi, Vi+1) represents the direct distance from vertex Vi to Vi+1. The pseudo-code of
47
the algorithm is as following:
Step 1: Set an initial temperature T;
Step 2: Generate a initial path order randomly from the start to the destination and calculate
the path length L(initial);
Step 3: Generate a new path which also from the start to the end and calculate path length
L(new);
Step 4: If L(new) < L(initial), accept new path. Else possibly accept new order according to
some scheduling.
Step 5: Repeat step 3 and step 4 until the temperature gets down.
Step 6: Down the temperature and return to step 1.
The following are pseudo-codes for the simulated annealing algorithm:
T = Tinit;
while (T>Tterminate)
randomly generate one feasible solution Xs;
evaluate Xs, Ef = f(Xs);
count = 1;
while (count < Threshold)
generate a new feasible solution Xn base on Xs;
evaluate Xn, En = f(Xn);
if f(Xn) < f(Xs)
Xs = Xn;
else if rand(1) < (exp((f(Xs)- f(Xn))/T)
Xs = Xn;
count =count +1;
endwhile
T = cool_rate * T;
update Xs at each reduction of temperature T
endwhile
48
Xs is the optimal or near-optimal solution for the robot.
Fig 27 is the flow chart of the simulated annealing approach for searching the path for robot
in the environment.
49
Fig 27: Procedure of simulated annealing approach for path planning.
50
3.2.3 Initial Path Selection Process
It is known from Fig 27 that after each reduction of initial temperature T, a new feasible
solution Xn is selected in each new round. It is essential for the algorithm to quickly and
correctly generates a random feasible path in each round. For this purpose, the edges of the
static obstacles should be specified first. An edge is any straight-line between two points on
the edge of or within the obstacles. Any path section crosses the edge is defined as an invalid
path. In the proposed program, a separate array is used to store all the edges of the map.
At the initial stage of the program, except dynamic objects, starting point and end point; a
vertex is chosen randomly from the map, then use a strait-line to connect the start point and
the selected vertex. If the path line intersects with any edges of the obstacle in the map, the
path is recognized as invalid path. Another vertex will be randomly selected again for testing.
If the strait-line to the selected vertex does not intersect with any edges in the map, the
strait-line is recognized as part of a valid path. Then add the vertex into the path. After that,
start from the selected vertex to find next valid vertex. Keep doing the above procedure until
the end point is selected and also the path line to the end point is a valid path, i.e., the path to
the end point does not intersects with any edges. The following pseudo-codes and Fig 28
illustrate the process of the initial path selection process.
initial path IP = {start};
while (1)
randomly select Vs;
if path{IPend, Vs} intersects any obstacles;
continue;
else
IP = {IP,Vs};
endif
if Vs == destination point;
break;
51
else
continue;
endif
endwhile
Fig 28: The initial path selection process.
52
3.2.4 Random Path Planner
As discussed in Chapter 2 (Literature Review), a simpler random path planner directly
affects the performance of the path planning algorithm. Therefore, advanced and more
efficient path planner needs to be implemented. Different from the simple path planner used
in [30], a more efficient random path planner is developed in this thesis.
Additional deleting and switching operators in the planner are used to generate a new
solution by flipping some bits of the Xs. The switching operator randomly switches two
vertices in the feasible path, then check if the new generated path is a feasible path (which
does not intersect with any edge). If the new path is a feasible path, accept the new path;
otherwise, discard it. The algorithm randomly chooses one operator to generate the new path.
Fig 29 shows that the deleting operator randomly deletes one vertex from the initial Xs to
generate a new solution, while the switching operator randomly swaps two vertices in Xs. As
the same selection criteria in generating initial path, each line section generated by the
operators should be firstly tested against the edges in the map in order to generate a valid
path line. This means that the line section created does not intersect with any edges in the
map.
Fig 29: Deleting operation.
When the path length is the evaluation criterion, randomly deleting vertices could contribute
more improving the performance of the solution. Therefore, the possibility of choosing the
deleting operator is set to be higher than the possibility of selecting the switching operator. In
our simulation program that will be discussed later, the possibility of choosing the deleting
operator is set to be 0.78. After generating a new path solution by deleting or switching
53
operator, the new solution will be evaluated using the evaluation function. Ether accept the
new solution if the new solution is better than the previous one, or accept the solution in a
certain probability defined by the current temperature.
3.2.5 Online Path Planning
As stated in Chapter 1 (Introduction), while a robot uses the route generated by offline
planning to travel through static obstacles, the online path planner is triggered automatically
to calculate an alternative optimal path when a dynamic obstacle is detected. As no particular
brand or configuration of the sensor is specified, the sensor range of the robot can be
simulated by a fixed value. The distances between the robot and every vertices of the moving
obstacle real-timely get updated. If the distance between the robot and any vertex of the
moving obstacle is shorter than the fixed value, it can be defined that the moving obstacle
enters the range of the sensor, and can be detected by the sensor of the robot. Then, the
separate module as described in assumption (7) (Section 3.1) can monitor the trajectory and
the shape of the moving obstacle. The moving information of the dynamic obstacles gathered
by the sensor of the robot includes speed and moving direction. After the robot combines the
information of the dynamic obstacle with the moving parameter of itself, the robot could
infer the possibility of collision with the moving obstacles.
The simulated annealing optimization algorithm for finding optimal path is triggered when it
is calculated that the robot will collide with the moving obstacle if no change of movement
will be made in the future. The simulated annealing algorithm will be activated and reloaded
with the updated search space. The search space for the algorithm will be updated. The
current status of the robot, the current location information of the dynamic obstacle, the
location information of the dynamic obstacles that could cause the collision and the location
information of the static obstacles are all combined as a new search space for the robot. In a
realistic case, the planning time should be short enough for the robot to implement motion
changes to avoid collision. With the search space becomes larger, as a result, the time
required of planning a path becomes longer. Therefore, an efficient algorithm is required that
its planning time is relatively short for the robot to change the directions to avoid the
54
collision with the obstacles, which is one main aim of our Thesis.
3.3 Platforms for the Research
Algorithm simulating software and system: Matlab 7 and Windows XP Operating System
Hardware equipment: Pentium Core 2 Duo 1.6Ghz Process, 1G RAM
55
Chapter 4 Simulation Results and Performance Evaluation
This chapter presents the simulation results. With these results, the performance of the
simulated annealing based approach is evaluated. Four environments are tested using the
developed approach. The simulation results are compared with those presented in [21] to see
weather the new approach is efficient for path planning in dynamic environments. The
evaluation results clearly illustrate the first main contribution of the Thesis, that the
simulated annealing based approach successfully generates collision free path for the robot in
a dynamic environment. The approach gives improved performance over existing methods.
The calculation time for obtaining the optimal path all-better than genetic algorithm based
approach in [21].
4.1 Simulation Environments and Algorithm Parameters
The approach is tested in four different environments. Each environment contains static and
dynamic obstacles. The path is optimized for length. The solution derived from the algorithm
56
is optimal or near-optimal. The numbers of static and dynamic obstacles in the four testing
environments are summarized in Table 1. The numbers of the vertices for offline planning
are also given in Table 1. The dynamic obstacles in all four cases have random shapes. The
first two environments simulate simple scenarios where the dynamic obstacles appear
simultaneously and simply travel forward in the same direction. The last two environments
are more complicated scenarios where the dynamic objects do not appear simultaneously but
each appears at a random time and moves forward or backward. Also the numbers of the
moving objects and static objects are larger than those of the simple environments. All the
position information of the static obstacles in the map is known to the robot for offline
planning before it starts to travel.
Table 1: Four testing environments
Environment Number of Static
Obstacles
Number of
Dynamic Obstacles
Number of Static
Vertices
1 3 2 10
2 6 2 25
3 9 4 56
4 14 6 90
The control parameters for the simulated annealing algorithm are set in a traditional way.
According to [18], with bigger initial temperature and smaller cooling rate, it is much more
likely to find the optimal solution, but it will require longer processing time for running.
After many times of testing, we set the control parameters of the algorithm as shown in table
2:
Table 2:Control parameters for the simulated annealing algorithm.
Initial Temperature
Terminate Temperature Cooling Rate Deleting
Operator Rate Switching
Operator Rate
999999999 555555555 0.97 78% 22%
57
4.2 Simulation Results
4.2.1 Case One (Three static obstacles with two dynamic obstacles)
A simple environment that contains three static and two dynamic obstacles is tested firstly.
The total number of vertices of the static environment is ten. Fig 30 shows that the robot
firstly uses offline path planning to obtain the optimal path based on static obstacles. Fig 31
shows that if moving obstacle is detected, an alternative path is generated by the online
planner.
Fig 30: Offline Path Planning Fig 31: Online Path Planning
The black full filled blocks in the map above are the static obstacles in the environment, the
hollow triangles are the trajectories of the dynamic obstacles, the sequence of the points are
the travel trajectory of the robot from start to end point. The above figures illustrate the
optimal path has been found and the robot can perform online path planning to find
alternative path for the robot in online path planning mode. Table 3 concludes the calculation
time of online and offline planning and the optimal result for environment one. The
simulation is run for ten times, the results in the table are the median results of the simulation.
Optimised path length and offline and online calculation time are recorded since the results
will be compared with previous results for discussion.
58
Table 3: Results for environment one
Search Space
(Vertices Numbers) 10
Number of
Dynamic Obstacles 2
Number of
Static Obstacles 3
Optimised Path Length
(Base on Static Obstacles) 145.78
Calculation Time for
Offline Planning (Seconds) 0.453
Calculation Time for
Online Planning (Seconds) 0.573
4.2.2 Case Two (Six static obstacles with two dynamic obstacles)
In case two, a more complex case is presented; the environment contains six static obstacles
with two dynamic obstacles. The same strategy as case one, Fig 32 shows that the robot
firstly uses offline path planning to obtain the optimal path based on static obstacles. Fig 33
shows that if a moving obstacle is detected, an alternative path is generated by the online
planner.
Fig 32: Offline Planning Fig 33: Online Planning 1
59
Fig 34: Online Planning 2
The above figures illustrate the optimal path has been found and the robot can perform online
path planning to find alternative path for the robot in online path planning mode. Fig 34
shows that the robot will use the current route to travel if it is calculated that no collision will
happen between robot and dynamic obstacle. Results in Table 4 show that due to the
increasing search space, the approach needs more time to calculate the optimal path for the
robot.
Table 4: Results for environment two
Search Space
(Vertices Numbers) 25
Number of
Dynamic Obstacles 2
Number of
Static Obstacles 6
Optimised Path Length
(Base on Static Obstacles) 257.25
Calculation Time for
Offline Planning (Seconds)1.201
Calculation Time for
Online Planning (Seconds)1.501
60
4.2.3 Case Three (Nine static obstacles with four dynamic obstacles)
In case three, a more complicated environment was involved. Additional four static and two
dynamic obstacles were added in the map. Other than the last two cases above, the dynamic
obstacles in the map do not appear simultaneously and the trajectory of one dynamic obstacle
is not the strait line. The dynamic obstacle can change moving direction during its movement.
Fig 36 shows that the dynamic obstacles do not appear simultaneously in the map. Fig 37
shows that one dynamic obstacle change its trajectory when it is moving.
Fig 35: Offline Planning Fig 36: Obstacles appear on different times.
Fig 37: Online Planning
61
Fig 37 shows the robot uses the online path planner twice to adjust its route safely travelling
to the final destination. Because the map is more complex, the results obtained by the
simulated annealing based approach are not the same each time. However, all the results
obtained by the approach are the optimal or near-optimal results. The following table shows
the median result of ten runs.
Table 5: Results for environment three
Search Space
(Vertices Numbers) 56
Number of
Dynamic Obstacles 4
Number of
Static Obstacles 9
Optimised Path Length
(Base on Static Obstacles) 288.01
Calculation Time for
Offline Planning (Seconds)3. 418
Calculation Time for
Online Planning (Seconds)4. 784
4.2.4 Case Four (Fourteen static obstacles with six dynamic obstacles)
Case four is the most complicated case in all four cases. Fourteen static obstacles and six
dynamic obstacles are included in the map. Dynamic obstacles appear randomly on different
times and move in different directions. Also, the dynamic obstacles could change their
moving directions during the movement.
62
Fig 38:Offline Planning Fig 39: Obstacles appear on different times
Fig 40: Online Planning 1 Fig 41: Online Planning 2
As in case three, Fig 39 shows that the dynamic obstacles do not appear simultaneously in
the map. Fig 40 and Fig 41 shows that two dynamic obstacles change their trajectory and the
robot uses the online path planner twice to adjust its rout safely travelling to the final
destination. Table 6 below concludes the obtained path length and calculation time for case
four.
63
Table 6: Results of environment four
Search Space
(Vertices Numbers) 90
Number of
Dynamic Obstacles 6
Number of
Static Obstacles 14
Optimised Path Length
(Base on Static Obstacles) 434.2397
Calculation Time for
Offline Planning (Seconds)10.98
Calculation Time for
Online Planning (Seconds)13.57
Fig 42 below illustrates the convergence of the algorithm. It is seen from Fig 42 that the
result converges rapidly in each round of temperature reduction. The algorithm could jump
out of the local minimum and approaches to the global minimum. Table 7 concludes the
calculation time of online path planning in four environments.
Fig 42: Convergence of the simulated annealing approach.
64
Table 7: Processing time for online planning for each case.
Environment 1 (10 vertices)
2 (25 vertices)
3 (56 vertices)
4 (90 vertices)
Processing Time (Seconds) 0.57 1.201 4.784 13.57
4.3 Performance Evaluation
The efficiency of the simulated annealing based approach is discussed in this section. The
simulated annealing based approach is compared with the previous results obtained based on
genetic algorithm in [21]. The new genetic based approach in [21] will be introduced firstly
in this section. The genetic based approach is re-implemented in the same hardware on which
simulated annealing based approach is implemented. The comparison results are also
included in this section.
4.3.1 The Genetic Based Approach
Wang, Mulvaney and Sillitoe [21] proposed a genetic based path planning in 2006. The
approach also uses the vertices of the obstacles as search space. Similar to the simulated
annealing based approach, obstacles are described as polygons. The proposed genetic vertex
planning method is compared with the Evolutionary Navigator/Planner [66], and it is claimed
that the performance is better than the Evolutionary Navigator/Planner results. The rest part
of this section introduces the genetic approach; most of the details are from [21]. In genetic
approach, each gene represents a single obstacle vertex selected as an intermediate node. The
solutions are described as chromosomes in the genetic approach. The chromosome in the
genetic approach contains a total number of genes N, whose minimum value is two (a path
containing only the start and goal nodes) and whose maximum value is L + 2, where L is the
total number of vertices in the static map. The gene in the chromosome has two parts, one is
the vertices reference, and the other is the feasibility bit. The feasibility bit for each gene
65
indicates whether the path segment originating from the vertex referenced by the node is
feasible. If the path segment connecting two consecutive vertices intersects any one or more
obstacles, then the feasibility bit of the first node of the path segment is assigned to 1 to mark
this segment as infeasible. If there is no intersection, 0 is assigned to indicate the segment is
feasible. Each population contains 30 of chromosomes. The following figure [21] shows the
structure of a chromosome:
Fig 43: Representation of chromosome in GA approach [21].
The approach use two evaluation functions, one is Ef which is simply the lengths of the
generated paths. Ef is given by:
where d(Vi, Vi+1) demotes the distance between the referenced pair of vertices. The other
evaluation function Ei indicates how deeply an infeasible path segment intersects with an
obstacle and is given by:
Where denotes the number of the obstacle intersections along the entire path and
is the mean number of intersections per infeasible segment. When the population contains
both feasible and infeasible paths, the infeasible paths are all assumed to be worse than the
worst feasible path.
Three genetic operators are used in the genetic approach, crossover, mutation and repair. The
crossover operator randomly selects two crossover points; the parts after the crossover points
of the two parent individuals are swapped. The mutation operator uses another vertex to
66
replace the selected vertex. The repair operator adjusts a randomly selected infeasible
segment of an infeasible path. Fig 44 illustrates the repair operation:
Fig 44: The Repair Operator [21]
The evaluation process is as the standard GA approach; only one genetic operator is selected
to produce the offspring for the approach. Each generation contains 30 chromosomes and
evolution was terminated when there was no further improvement in the fitness of the best
individual over 300 generations. The following are the copy of the pseudo-code from [21]:
Procedure vertex planning algorithm
begin
t = 0
enlarge the obstacles
encode the vertices of the obstacles
initialise P(t)
decode P(t)
evaluate P(t)
while (not terminating condition) do
67
t = t+1
select operator Oi with probability Pj
select parent (s) from P(t)
apply the operator Oj to produce offspring
decode offspring
evaluate new offspring
replace worst member in P(t) by offspring
end
select the best individual p from P(t)
end
4.3.2 Results Comparison
Three environments are used to make comparison of the approaches. One environment is
selected from the cases in [21], and other two cases are the case 3 and case 4 in the simulated
annealing approach simulation in last chapter. The search space for the two approaches is the
number of the vertices in the map. First environment has 5 static obstacles which has total
number of 22 vertices in the map. Second environment has 9 static obstacles which has 56
vertices in the map. The last environment has 14 obstacles which are constituted by 90
vertices. The termination conditions of the two approaches are:
(1) Termination condition of the simulated annealing approach is the parameter
termination temperature terminT which is predefined at the start of the approach.
(2) As in [21], the genetic based approach terminates when noticing that there is no
improvement on path length after 300 generations.
Every approach was run 10 times on each environment. The results in the Table 9 and Table
10 are the medians obtained over 10 runs on each environment. Table 9 and Table 10 present
the comparison results of the execution time to get the final path and the final path length
from the two approaches:
68
Table 8: Three environments for performance evaluation.
Environment Number of Static
Obstacles
Number of Total
Vertices
1 5 22
2 9 56
3 14 90
Table 9: Comparison of execution time of getting the final path (Seconds).
Environment Simulated Annealing
Algorithm Based Approach
Genetic Algorithm Based
Approach
1 1.01 6.90
2 3.69 21.78
3 12.01 29.78
Table 10: Comparison of the length of the final path.
Environment Simulated Annealing
Algorithm Based Approach
Genetic Algorithm Based
Approach
1 258.477 259.455
2 291.690 351.238
3 429.022 469.580
Fig 45 compares the offline execution time of two approaches base on same vertices numbers.
It shows that the simulated annealing algorithm based approach consumes a shorter time in
path planning than the genetic algorithm based approach.
69
Fig 45:Comparism of execution time based on same vertices number.
Fig 46 compares the final optimal path lengths obtained by two approaches base on same
vertices numbers. It shows the solutions are improved in the simulated annealing algorithm
based approach.
Fig 46: Comparison of final path length based on same vertices number.
For comparing the convergence and the optimisation efficiency of the two approaches, from
each case, selecting the individual runs that produce similar path length results. And
comparing the convergence and efficiency base on the time elapses in two approaches.
70
Environment 1 (22 Vertices):
Final Path Length: 259.27
GA Processing Time: 6.969 Seconds (Dashed Line)
SA Processing Time: 1.107 Seconds (Solid Line)
Fig 47: Convergence comparison in environment one.
Environment 2 (56 Vertices):
Final Path Length: 365.18
GA Processing Time: 13.422 Seconds (Dashed Line)
SA Processing Time: 3.437 Seconds (Solid Line)
Fig 48: Convergence comparison in environment two.
71
Environment 3 (90 Vertices):
Final Path Length: 516.3911
GA Processing Time: 39.015 Seconds (Dashed Line)
SA Processing Time: 10.001 Seconds (Solid Line)
Fig 49: Convergence comparison in environment three.
In paper [21], insufficient information is provided about the implementation of the genetic
operators in the approach. For example, in [21], how to implement the repair operator in the
approach is not clearly specified. Therefore, the differences of implementing the genetic
operators between this paper and [21] may slightly affect the simulation results on our
hardware. Thus, the simulation results of genetic based approach in our hardware may be
slightly different than the simulation results in [21]. However, as Fig 50 shows: environment
one with 25 vertices is established to build a mimic environment one in [21], and the
simulation result in environment one could produce similar results as in [21]. Therefore the
genetic approach implemented on our hardware could illustrate a comparative performance
to [21]. Therefore, the comparison could show that the performance of the simulated
annealing based approach is comparable or even more efficient than the genetic based
approach in [21]. The simulated annealing based approach is able to use shorter processing
time to determine the optimal path in the environment.
72
0 20 40 60 80 100 120 140 160 180 2000
20
40
60
80
100
120
140
160
180
200
Start Point
End Point
Fig 50: Environment One Fig 51: Environment One From [21]
73
Chapter 5 Heuristic Search Method for the Simulated Annealing
Approach
This chapter proposes a heuristic search method to further improve the simulated annealing
approach for robot in dynamic environment. The simulation results show that the heuristic
method can greatly improve the efficiency of simulated annealing approach in offline and
online planning for robots in dynamic environments. The simulation results in this chapter
can totally illustrate the second main contribution of this Thesis.
5.1 The Structure and Implementation of the Heuristic Selecting Method
From Chapter 3 (Implementation of the Simulated Annealing Algorithm Approach), it is
known that in finding the optimal path, the simulated annealing approach needs firstly
selecting a random feasible path, then apply a mathematic operator to generate a new path
base on the selected feasible path. The initial selected path is the beginning of the search
point; the initial solution is replaced by the randomly generated neighbour solution. The
search goes along “down hill” movement from the initial solution. Then, as temperature goes
74
down, the algorithm accepts the “up hill” movement to avoid being stuck at local minimus.
Therefore, a better initial solution means a better initial searching point, which could enhance
the efficiency and the performance of the approach.
From Chapter 3 (Implementation of the Simulated Annealing Algorithm Approach), the way
of generating the initial path was illustrated as follows:
• Firstly, except dynamic objects, starting point and end point, a vertex is chosen
randomly from the map, use strait-line to connect the start point and the selected
vertex.
o If the path line intersects with any edges of the obstacle in the map, the path is
recognized as invalid path. Another vertex will be randomly selected again for
testing.
o If the strait-line to the selected vertex does not intersect with any edges in the
map, the strait line is recognized as part of a valid path. Then add the vertex
into the path.
• Then, start from the selected vertex to find the next valid vertex. Keep performing
the above procedure until the end point is selected and also the path line to the end
point is a valid path (i.e., the path to the end point does not intersects with any
edges).
The method just randomly picks a feasible vertex until the end point is picked. The blindly
picking is not an efficient way of producing the initial path.
To improve the efficiency of the initial solution selecting process, a heuristic process is
proposed in this Chapter. The process makes a modification on the current method. In current
method, a random vertex is selected after one feasible vertex is select, and then tests the
feasibility of the line segment between randomly selected vertex and the feasible vertex.
In the proposed heuristic method, after one feasible vertex is select, an end point feasibility
test will be carried out before selecting another vertex. Before selecting next vertex, the
method will firstly test whether the line segment between the feasible vertex and the end
point is feasible (does not interest with any obstacles). If the segment between the feasible
75
vertex and the end point is feasible, then put the end point as the next vertex and complete
the selection process. The following pseudo-codes and Fig 52 illustrates the process of the
heuristic initial path selection method.
initial path IP = {start};
while (1)
if path{IPend, Destination Point} is a feasible path
IP = {IP, Destination Point};
break;
endif;
randomly select Vs;
if path{IPend, Vs} intersects any obstacles;
continue;
else
IP = {IP,Vs};
endif
if Vs == destination point;
break;
else
continue;
endif
endwhile
76
Fig 52: Process of the heuristic selection method.
Fig 53 and Fig 54 show the difference between the random picking method in normal
simulated annealing algorithm and the proposed heuristic method. Fig 53 shows it is possible
that it requires five times of random picking to find a feasible path to the end point. And Fig
54 shows that if testing the end point feasibility before random selection, a better feasible
path may be obtained.
77
Fig 53: Random picking method. Fig 54: Heuristic picking method.
The heuristic method could improve the approach in two aspects:
• One is that the method could enhance the quality of the solution, i.e., a shorter path
length solution, obtained by the approach. Fig 54 above shows that the heuristic
method may generate a better initial path than the randomly picking method. The
better initial path means the better searching beginning, which may lead a better
solution.
• The other improved aspect is the processing time of getting the solution. After each
feasible vertex is selected, the heuristic method firstly checks the feasibility of the
line segment to the end point rather than randomly select next vertex. Therefore, the
processing time of selecting vertices could be reduced by the heuristic method.
In fact, the heuristic method could be used not only in the simulated annealing approach, but
also in any other mathematic methods which generate initial solution for calculation.
78
5.2 Performance Evaluation of the Heuristic Method
To test the performance of the heuristic method, the random picking process is replaced by
the heuristic method in the simulated annealing approach, and the modified simulated
annealing approach is tested by the same four environments as those in Chapter 4
(Simulation Results and Performance Evaluation). The simulation results of the modified
simulated annealing based approach are compared with those obtained in Chapter 4. The
following are the compared results of the two methods:
5.2.1 Environment One (Three static obstacles with two dynamic obstacles):
Fig 55: Random picking method. Fig 56: Heuristic picking method.
79
Table 11: Results of comparison in case one (3 static and 2 dynamic obstacles).
Environment One
Random
Picking
Method
Heuristic
Method Improvement
Optimal Path Length 145.78 145.78 0%
Calculation Time
(Offline Planning)
(Seconds)
0.453 0. 447 1.34%
Calculation Time
(Online Planning)
(Seconds)
0.573 0. 335 41.54%
5.2.2 Environment Two (Five static obstacles with two dynamic obstacles):
Fig 57: Random picking method. Fig 58: Heuristic picking method.
80
Table 12: Results of comparison in case two (5 static and 2 dynamic obstacles).
Environment Two
Random
Picking
Method
Heuristic
Method Improvement
Optimal Path Length 257.25 246.28 4.26%
Calculation Time
(Offline Planning)
(Seconds)
2.453 0.793 67.67%
Calculation Time
(Online Planning)
(Seconds)
2.573 0.822 68.15%
5.2.3 Environment Three (Nine static obstacles with four dynamic obstacles):
Fig 59: Random picking method. Fig 60: Heuristic picking method.
81
Table 13: Results of comparison in case three (9 static and 4 dynamic obstacles).
Environment Three
Random
Picking
Method
Heuristic
Method Improvement
Optimal Path Length 288.01 277.60 3.62%
Calculation Time
(Offline Planning)
(Seconds)
3.418 1.411 58.72%
Calculation Time
(Online Planning)
(Seconds)
4.784 1.562 67.35%
5.2.4 Environment Four (Fourteen static obstacles with six dynamic obstacles):
Fig 61: Random picking method. Fig 62: Heuristic picking method
82
Table 14: Results of comparison in case four (14 static and 6 dynamic obstacles).
Environment Four
Random
Picking
Method
Heuristic
Method Improvement
Optimal Path Length 434.24 416.36 4.12%
Calculation Time
(Offline Planning)
(Seconds)
10.98 2.796 74.54%
Calculation Time
(Online Planning)
(Seconds)
13.57 2.96 78.19%
5.3 Discussions of the Evaluation Results
From the comparison results presented above in Section 5.2, it can be seen that the heuristic
search method can greatly enhance the performance of the simulated annealing based
approach in dynamic environments. The heuristic method can accelerate the calculation in
online and offline planning modes and can reduce the optimal path length obtained,
especially in complex environments. The evaluation results clearly show the second main
contribution of the Thesis, the heuristic method is successfully incorporated in the simulated
annealing algorithm based approach and can boost the efficiency of the approach.
5.3.1 Offline Planning
In offline planning calculation,
• In the simple environment with 3 static obstacles, the heuristic method improves the
offline processing time by 1.34%.
• In the environment with 5 static obstacles, the heuristic method improves the offline
83
processing time by 67.67%.
• In the environment with 9 static obstacles, the heuristic method improves the offline
processing time by 58.72%.
• The heuristic method gives nearly 74.54% improvement in offline processing time in
finding the optimal path in the environment with 14 static obstacles.
5.3.2 Online Planning
In online planning mode, the heuristic method improves the online processing time by
• 41.54% in environment 1;
• 68.15% in environment 2;
• 67.35% in environment 3 with 9 static obstacles; and
• 78.19% in environment 4 with 14 static obstacles.
5.3.3 The Length of the Path
The heuristic method improves the obtained optimal path by 4.26% in environment 2, and it
improves the optimal path by 3.62% and 4.12% in environment 3 and environment 4
respectively.
Fig 63 compares the optimised path length solution on each case. Fig 64 and Fig 65 compare
the offline and online processing time of the two methods on each environment. Table 15
summarizes the new heuristic method improvements on the processing time and the optimal
path length base on four environments. It can be seen that the heuristic method is far superior
to the random picking method and should be used by the simulated annealing algorithm
based approach in complex environment especially in online path planning calculations.
84
Fig 63: Path solution comparison.
Fig 64:Offline processing time comparison. Fig 65: Online processing time comparison.
Table 15: Conclusion of the improvements.
Environment Improvements
(Offline Planning)
Improvements
(Online Planning)
Improvements
(Optimal Path)
1 (10 Vertices) 1.34% 41.54% 0%
2 (25 Vertices) 67.67% 68.15% 4.26%
3 (56 Vertices) 58.73% 67.15% 3.62%
4 (90 Vertices) 74.54% 78.47% 4.12%
85
5.3.4 Comparison with the Genetic Algorithm Based Approach
Finally, the performance of the heuristic simulated annealing based approach is compared
with the performance of the normal simulated annealing approach and the genetic algorithm
based approach. The heuristic based simulated annealing approach is implemented on the
three test environments in Section 4.3.2 to compare the offline processing time of three
approaches. Compared with the genetic algorithm based approach; the heuristic method
improves the offline processing time by 88.69% in environment 1 with 5 static obstacles. The
heuristic method improves the offline processing time by 93.07% and 90.03% in
environment 2 with 9 obstacles and environment 3 with 14 obstacles respectively. Fig 66
concludes the calculation time performance of the heuristic method based simulated
annealing approach, the basic simulated annealing approach, and the genetic algorithm based
method.
Compared with the genetic algorithm based approach; the heuristic method improves the
obtained optimal path by 0.38% in environment 1 with 22 vertices, and it improves the
optimal path by 20.96% and 11.33% in environment 2 and environment 3 respectively. Fig
67 concludes the performance of obtaining the optimal path in the three methods. Fig 66 and
Fig 67 clearly show that the heuristic method based simulated annealing approach gives the
best performance. Therefore, it can be concluded that the new heuristic method for simulated
annealing algorithm gives superiority to other methods in path planning on vertices based
complicate environment, and it should be used in simulated annealing based approach to deal
with the path planning issue in dynamic environment.
86
Fig 66: The comparison of the three methods in processing time.
Fig 67: The comparison of the three methods in final path length.
87
Chapter 6 Conclusion
This chapter summarizes the Thesis, and discusses the research limitations of the project.
Future research on the simulated annealing based approach will also be discussed.
6.1 Summary
This thesis has developed and implemented a simulated annealing based approach in Chapter
3 to deal with path planning issue for robots in dynamic environment – this is one of the
main contributions of the thesis. The approach uses the vertices of the static and dynamic
obstacles as search space to obtain the optimal path for robots, the approach searches the
initial feasible path for robot in dynamic environment which contains static and dynamic
obstacles, then uses simulated annealing algorithm to obtain the optimal path for the robot in
the dynamic map. Compared with the genetic based approach in [21], the simulation results
in Chapter 4 show that the simulated annealing based approach provides a better
performance in processing time, and is crucial for a robot to quickly response to avoid
88
collision with the dynamic obstacles.
The thesis has also proposed a heuristic search method in Chapter 5 to search the initial path
– this is another main contribution of the thesis. The heuristic method has been incorporated
into the developed simulated annealing algorithm based approach to greatly enhance the
performance of the path planning in both online and offline calculations. The experiment
results show that the heuristic search method can improve not only the processing time of
offline and online planning but also the quality (path length of the solution) of the solution.
The percentage of the improvement in online processing time ranges from 41% to 78% in
our simulated environments for dynamic path planning.
The novelty of the project is the implementation of a simulated annealing based approach
incorporated with a heuristic selecting method for robot path planning in dynamic
environments with moving obstacles. The approach is simpler and easier to implement than
existing genetic algorithm based methods. As the method is able to quickly determine the
optimal feasible path for robot in dynamic environments, it could be used in marine research
robot such as [3] and [2]. Also, the heuristic selecting method can enhance the efficiency of
the simulated annealing based approach. Furthermore, the heuristic selecting method can also
be used in other optimisation methods which need to generate an initial feasible solution for
calculation.
6.2 Research Limitations
The project successfully implements the simulated annealing based approach incorporated
with a heuristic selecting method to plan path for mobile robots in dynamic environments.
However, there are still some research limitations in the project. The limitations of the
project are described below:
1) The shape of the robot is ignored in the project though the dimensions of the obstacles are
considered for calculation. This is a topic that has not been systematically investigated in
the dynamic path planning for mobile robots. Saboori and Menhaj [67] proposed a robot
89
path planning method that is based on fuzzy model. The method implements Dijkstra
algorithm to find an optimum path for in an area with static obstacles for the robot. The
dimensions of the robot and the obstacles have been taken into consideration to have a
more realistic and efficient technique. However, the research and the simulation
experiments are constrained in static environments. Therefore, taking the dimension of
the robot into consideration in dynamic environments is a future issue.
2) In the project, no particular brand or configuration of the sensor is specified. The sensor
is assumed to be able to acquire the motion information such as the speed and direction of
the moving obstacles once the obstacles are in the sensor range. For a more realistic
modelling, how the sensor obtains the motion information of the dynamic obstacle should
be modelled and discussed.
6.3 Future Work
Though the simulated annealing based approach solves the issue of obtaining the optimal
path in dynamic environments, some work is still expected to be done in the future:
1) For a more realistic modelling, a more complex and realistic environment shall be used
and tested using the simulated annealing approach. The environments in the project are
described in a two-dimensional surface and the trajectories of the dynamic obstacles are
constituted by a series of line segments. For modelling a more realistic environment for
robot, a three-dimensional modelling is expected and the trajectories of the dynamic
obstacles can be curves instead of strait line segments. The three-dimensional
environment still uses vertices as search space; the only difference is that the calculation
should add Z coordinate.
2) As stated in Section 6.2 about research limitations, for a more realistic modelling, the
method used by the sensor to obtain the motion information of the dynamic obstacle
should be further investigated. More research should be done to discuss the detailed
method on how the sensor acquires the motion information of dynamic obstacles (e.g.
90
using which sensor? What is the appropriate mathematical method?).
3) Implementing the simulated annealing approach on a more realistic robot simulator, such
as the Player/Stage [65] application. Successfully implementing the simulated annealing
approach into Player/Stage provides a more practical environment than Matlab which
only uses mathematic model to demonstrate the ability of algorithm. The implementation
of the new method on Player/Stage not only demonstrates the feasibility of method but
also provides a more practical testing environment for the new method.
91
References [1] J. Cook, “Adding Intelligence to Robot Arm Path Planning Using a Graph-Match
Analogical Reasoning System” IEEE International Conference on Intelligent Robots and
Systems, vol. 1, pages 657-663, July 1992
[2] J. Ayers, “Underwater Walking”, Arthropod Structure & Development, vol. 33, pages
347-360, July 2004
[3] B. Williams and I. Mahon, “Design of An Unmanned Underwater Vehicle For Reef
Surveying”, Proceedings of IFAC 3rd Symposium on Mechatronic Systems, Sept 2004
[4] H. Mitchell, “An algorithmic approach to some problems in terrain navigation”, Artificial
Intelligence, vol. 37, pages 171–201, Dec 1988
[5] C. O’Dunlaing, and C. Yap, “A retraction method for planning the motion of a disc”,
Journal of Algorithms, vol. 6, pages 104–111, 1982
[6] D. Payton, J. Rosenblatt and D. Keirsey, “Grid-based mapping for autonomous mobile
robot”, Robotics and Autonomous Systems, vol. 11, pages 13–21, 1993
[7] N. Kalra, D. Ferguson and A. Stentz, “Incremental Reconstruction of Generalized
Voronoi Diagrams on Grids”, Robotics and Autonomous Systems, vol. 57, pages 123 –
128, Feb 2009
[8] P. Zhang, T. Lu and L. Song, “Soccer Robot Path Planning Based on the Artificial
Potential Field Approach with Simulated Annealing”, Robotica, vol. 22, pages 563-566,
Oct 2004
[9] B. Stuckman, G. Evans and M. Mollaghasemi, “Comparison of Global Search Methods
92
for Design Optimization Using Simulation”, Proceedings of the Winter Simulation
Conference, pages 937-945, Dec 1991
[10] S. Russell, P. Norvig, “Artificial Intelligence: A Modern Approach”, Prentice Hall, pages
97-104,2003
[11] A. Stentz, “Optimal and Efficient Path Planning for Partially-Known Environments”,
Proceedings of the IEEE International Conference on Robotics and Automation, pages
3310 – 3317, May 1994
[12] D. Rathbun and S. Kragelund, “An Evolution Based Path Planning Algorithm for
Autonomous Motion of a UAV Through Uncertain Environments”, IEEE International
Conference on Digital Avionics Systems Conference, vol. 2, pages 8.D.2.1 – 8.D.2.12,
Oct 2002
[13] Introduction of Simulated Annealing Algorithm, Internet Source,
<http://en.wikipedia.org/wiki/Simulated_annealing >, From Wikipedia, the free
encyclopedia, Viewed in 11th Oct. 2007
[14] Y. Wang, P. Sillitoe, J. Mulvaney, “Mobile Robot Path Planning in Dynamic
Environments”, IEEE International Conference on Robotics and Automation, pages
71-76, May 2007
[15] Q. Zhu, Y. Yan and Z. Xing, “Robot Path Planning Based on Artificial Potential Field
Approach with Simulated Annealing”, Proceedings of the Sixth International
Conference on Intelligent System Design and Applications, vol. 2, pages 622-627, Oct
2006
[16] P. Zhang and T. Lv, “Soccer Robot Path Planning Based on Artificial Potential Field
Approach with Simulated Annealing”, Mechanical Science and Technology, vol. 22,
pages 624-629, Oct 2003
93
[17] Y. Peng and W. Wei, “A New Trajectory Planning Method of Redundant Manipulator
Based on Adaptive Simulated Annealing Genetic Algorithm”, Proceedings of IEEE the
Sixth International Conference on Intelligent System Design and Applications, vol. 1,
pages 262-266, Nov 2006
[18] J. Park, D. Nam and C. Park, “An Empirical Comparison of Simulated Annealing and
Genetic Algorithms on NK Fitness Landscapes”, Proceeds of IEEE the Sixth
International Conference on Evolutionary Computation, pages 147-151, Apr 1997
[19] E. Burke and G. Kendall, “Comparison of Meta-Heuristic Algorithms for Clustering
Rectangles”, Proceedings of the 24th International Conference on Computers and
Industrial Engineering, vol. 3, pages 383-386, Sept 1999
[20] J. Perez and J. Basterrechea, “Comparison of Different Heuristic Optimization Methods
for Near-Field Antenna Measurements”, IEEE Transaction on Antennas and
Propagation, vol. 55, pages 549 – 555, March 2007
[21] Y. Wang; D. Mulvaney; I. Sillitoe, “Genetic-based Mobile Robot Path Planning using
Vertex Heuristics”, IEEE International Conference on Cybernetics and Intelligent
Systems, pages 1 – 6, June 2006
[22] P. Tang, Q. Zhang, Y. Yang, “Studying on path planning and dynamic obstacle avoiding
of soccer robot” Proceedings of the 3rd World Congress on Intelligent Control and
Automation, vol. 2, pages 1244- 1247, July 2000
[23] Q. Cao, Y. Huan and J. Zhou, “An Evolutionary Artificial Potential Field Algorithm for
Dynamic Path Planning of Mobile Robot” International Conference Proceedings on
Intelligent Robots and System, pages 3331-3336, Oct 2006
[24] P. Zhang, T. Lu, “Soccer Robot Path Planning based on the Artificial Potential Field
94
Approach with Simulated Annealing”, Robotica, vol 22, pages 563 – 566, Oct 2004
[25] A. Stentz, “The Focussed D* Algorithm for Real-Time Replanning”, Proceedings of the
International Joint Conference on Artificial Intelligence, pages 1652-1659, Aug 1995
[26] A.Yahja, S. Singh and A. Stentz, “An Efficient Online Path Planner for Outdoor Mobile
Robots Operating in Vast Environments”, Robotics and Autonomous Systems, vol 22,
pages 129-143, 2000
[27] A. Yahia, A.Stentz, S. Singh and B. Brummit, “Framed-Quadatree Path Planning for
Mobile Robots Operating in Sparse Environments”, IEEE Conference on Robotics and
Automation, vol. 32, pages a129-143, May 1998
[28] D. Ferguson and A. Stentz, “Field D*: An Interpolation-based Path Planner and
Replanner”, Intermational Symposium on Robotics Research, vol. 8, pages 239-253, Oct
2005
[29] A. Nearchou, “Path planning of a mobile robot using genetic heuristics”, Robotica, vol.
16, pages 575–588, Sept 1998.
[30] C. Yap, “Algorithmic motion planning” in Advances in Robotics vol. 1: Algorithmic and
Geometric Aspects of Robotics, Eds. Hillsdale, New Jersey: Lawrence Erlbaum, pages
95-143, 1987
[31] W. Regli, “Robot Lab: Robot Path Planning”, Lecture Notes of Department of Computer
Science, Drexel University, Viewed at 20th Oct 2007
[32] O. Khatib, “Real-time Obstacle Avoidance for Manipulators and Mobile Robots”, IEEE
Conference on Robotics and Automation, vol. 2, pages 500-505, Jan 1985
[33] Y. Davidor, “Genetic algorithms and robotics: a heuristic strategy for optimization”,
95
Singapore: World Scientific Publishing, vol. 1, pages 220-225, 1991
[34] J. Cai, M. Peng and S. Ma, “RL-ART2 Neural Network Based Mobile Robot Path
Planning”, Proceedings of the IEEE Sixth International Conference on Intelligent
System Design and Applications, vol. 2, pages 581-585, Oct 2006
[35] M. Scott, “An introduction to genetic algorithms”, Journal of Computing Sciences in
Colleges, vol. 20, pages 115-123, Oct 2004
[36] C. Edelman, M. Franklin, and M. Witte, “Simulated Annealing on a Multiprocessor”,
Proceedings of the 1988 IEEE International Conference on Computer Design: VLSI in
Computers and Processors, pages 540 – 544, Oct 1988
[37] P. Hart, N. Nilsson and B. Raphael, “A Formal Basis for the Heuristic Determination of
Minimum Cost Paths”, IEEE Transaction on Systems Science and Cybernetics, vol. 4,
pages 100–107, July 1968
[38] F. Lingelbach, “Path planning for mobile manipulation using probabilistic cell
decomposition”, Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings,
IEEE, vol 3, pages 2807-2812, 2004
[39] J. Rosell and P. Iniguez “Path planning using Harmonic Functions and Probabilistic Cell
Decomposition”, Proceedings of IEEE International Conference on Robotics and
Automation, pages 1803-1808, Apr 2005
[40] Lecture Note, “Robot Motion Planning”, Department of Computer Science, University
of Mcgill, Viewed at 20th Oct 2007
[41] M. Park and M. Lee, “Experimental evaluation of robot path planning by artificial
potential field approach with simulated annealing”, Proceedings of SICE 2002 Annual
Conference, vol. 4, pages 2190-2195, Aug 2002
96
[42] N. Ying and L. Eicher, “Real-time 3D path planning for sensor-based underwater
robotics vehicles in unknown environment”, OCEANS 2000 MTS/IEEE Conference and
Exhibition, vol 3, pages 2051-2058, Sept 2000
[43] M. Park and M. Lee, “Real-time path planning in unknown environment and a virtual
hill concept to escape local minima”, IEEE Conference on Industrial Electronics Society,
vol 3, pages 2223-2228, Nov 2004
[44] Y. Chang, Y. Yamamoto, “Online Path Planning Strategy Integrated with Collision and
Dead-lock Avoidance Schemes for Wheeled Mobile Robot in Indoor Environment”,
Industrial Robot-An International Journal, vol 35, pages 421 – 434, 2008
[45] L. Xie, H. Chen, “Solution to Reinforcement Learning Problems with Artificial Potential
Field”, Journal of Central South University of Technology, vol 15, pages 552 – 557,
Aug 2008
[46] J. Vanualailai and B. Sharma, “An Asymptotically Stable Collision-avoidance System”,
International Journal of Non-linear Mechanics, vol 43, pages 925 – 932, Nov 2008
[47] T. Wan; H. Chen and R. Earnshaw, “Real-time path planning for navigation in unknown
environment”, International Conference on Theory and Practice of Computer Graphics,
pages 138-145, June 2003
[48] M. Mansor and A. Morris, “Path Planning in Unknown Environment With Obstacles
Using Virtual Window”, Journal of Intelligent and Robotic Systems, vol 24, pages 235 –
251, Mar 1999
[49] X. Hu, Q. Xu, “Robot Path Planning based on Artificial Immune Network”, IEEE
International Conference on Robotics and Biomimetics, pages 1053 – 1057, Dec 2007
97
[50] X. Hu, C. Xie “Niche Genetic Algorithm for Robot Path Planning” The Third IEEE
International Conference on Natural Computation, vol. 2, pages 600-605, Aug 2007
[51] J. Chestnutt and M. Lau, “Footstep Planning for the Honda ASIMO Humanoid”, IEEE
International Conference on Robotics and Automation, pages 629-634, Apr 2005
[52] S. LBszlo and K. Annamiria, “Autonomous navigation in a known dynamic
environment”, The 12th IEEE International Conference on Fuzzy Systems, vol. 1, pages
266-271, May 2003
[53] N. Lv and Z. Feng, “Numerical Potential Field and Ant Colony Optimization Based Path
Planning in Dynamic Environment”, The Sixth IEEE World Congress on Intelligent
Control and Automation, vol 2, pages 8966- 8970, Oct 2006
[54] T. Zheng and X. Zhao, “A Novel Approach for Multiple Mobile Robot Path Planning in
Dynamic Unknown Environment”, IEEE Conference on Robotics, Automation and
Mechatronics, pages 1-5, Dec 2006
[55] M. Tarokh, “Hybird Intelligent Path Planning for Articulated Rovers in Rough Terrain”,
Fuzzy Sets and Systems, vol 159, pages 2927 – 2937, Nov 2008
[56] L. Hu, Z. Gu, “Research and Realization of Optimum Route Planning in Vehicle
Navigation Systems based on a Hybrid Genetic Algorithm”, Proceedings of the
Institution of Mechanical Engineers part D-Journal of Automobile Engineering, vol 22,
pages 757 – 763, May 2008
[57] L. Huang, “Velocity Planning for a Mobile Robot to Track a Moving Target – A
Potential Field Approach”, Robotics and Autonomous Systems, vol 57, pages 55 – 63,
Jan 2009
[58] A. Willms, S. Yang, “Realtime Robot Path Planning via a Distance-Propagating
98
Dynamic System with Obstacle Clearance”, IEEE Transactions on System, Man, and
Cybernetics, vol 38, pages 884 – 893, Jun 2008
[59] J. Schneider, S. Kirkpatrick, “Stochastic Optimization”, Springer Berlin Heidelberg,
pages 299 – 314, 2006 [60] Z. Lo and Z. Bavarian, “Job scheduling on parallel machines using simulated annealing”,
IEEE International Conference on Decision Aiding for Complex Systems, vol. 1,
pages 391-396, Oct 1991
[61] Y. Zhu, S. Yan, “A Knowledge based Genetic Algorithm for Path Planning of a Mobile
Robot”, IEEE International Conference on Robotics and Automation, vol 5, pages 4350
– 4355, May 2004
[62] R. Zhang and P. Lu, “Beyond quadtrees: Cell decompositions for path planning using
wavelet transforms”, The 46th IEEE International Conference on Decision and
Control, pages 1392-1397, Dec 2007
[63] EyeSim: EyeBot Simulator, Free Internet Source,
<http://robotics.ee.uwa.edu.au/eyebot/doc/sim/sim.html>, Viewed in 11th Feb. 2008
[64] X. Yan, W. Li and D. Chen “A New Mechanism for Robots Control Based on
Player/stage” Proceedings of International Conference on Robotics and Biomimetics,
pages 750-754, Dec 2006
[65] P. Gerkey, T. Vaughan, A. Howard, “The Player/Stage Project: Tools for Multi-Robot
and Distributed Sensor Systems” Proceedings of the International Conference on
Advanced Robotics, pages 317-323, July 2003
[66] J. Xiao, “Evolutionary planner/navigator in a Mobile Robot Environment”, Handbook of
Evolutionary Computation, IOP Publishing Ltd, pages G3.11: 1 – G3.11: 13, Jan 1995
99
[67] I. Saboori and M. Menhaj, “Optimal Robot Path Planning Based on Fuzzy Model of
Obstacles”, IEEE 32nd Annual Conference on Industrial Electronics, pages 383-387,
Nov 2006
100