A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based...
Transcript of A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based...
![Page 1: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/1.jpg)
A Comparison of SLAM Algorithms Based on a Graph of Relations
Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian Steder, Rainer Kümmerle, Christian Dornhege, Michael Ruhnke, Alexander Kleiner, Juan D. Tardos
University of Freiburg, Germany and University of Zaragoza, Spain
![Page 2: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/2.jpg)
RAWSEEDS
� Robotics Advancement through Web-publishing of Sensorial and Elaborated Extensive Data Sets� University of Freiburg
� University of Zaragoza
� Politecnico di Milano
� Università degli Studi di Milano-Bicocca
� Funded by the European Commission
� http://www.rawseeds.org/
![Page 3: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/3.jpg)
Motivation
� No gold standard to evaluate and compare the results of SLAM algorithms
� Most approaches depend on the used sensor setup, landmark detector/grid map, and estimation algorithm
� Often, ground truth is not available
� Even if there is ground truth information: how can we compare the results of SLAM algorithms?
![Page 4: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/4.jpg)
Related Work
� SLAM approaches (Smith&Cheeseman, Lu&Milios, Frese, Duckett et al., Dissanayake et al., Howard et al., Eustice et al., Grisetti et al., Stachniss et al., Olson, …)
� Competitions (DGC, RoboCup, NIST, …)
� Data repositories (Radish, RAWSEEDS, OpenSLAM, …)
![Page 5: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/5.jpg)
Measures
� Uncertainty(Entropy, KL-Divergence, …)
� Variance of estimators(Cramer-Rao-bound, Fisher information, …)(Cramer-Rao-bound, Fisher information, …)
� Quality of estimates(observation likelihood, normalized estimation error squared, the measure described in this paper, …)
![Page 6: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/6.jpg)
Sum of Squared Distances between Estimated and Ground Truth Poses
� Let x1:T
be the estimated poses from time 1 to T.
� Let x*1:T
be the reference poses.
� Let be the standard motion composition operator and its inverse.
� Resulting error function
![Page 7: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/7.jpg)
Normalized Estimation Error Squared: Taking into Account Covariance
In SLAM, this measures the absolute error of the landmark estimates (weighted by inverse covariance).
![Page 8: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/8.jpg)
Problems with this Measure
![Page 9: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/9.jpg)
Problems with this Measure� Error in the beginning:
� Error in the very end:
![Page 10: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/10.jpg)
Desired Measure
� that does not automatically count errors multiple times,
� whose results does not depend on the order data is processed, and
� that is compatible with what is actually � that is compatible with what is actually optimized in SLAM approaches.
Inspired by graph-based SLAM: consider the energy needed to transform the pose graph into ground truth
![Page 11: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/11.jpg)
Pose Graph
� A pose-graph encodes the poses of the robot during mapping as well as constraints resulting from observations
� Independent of the kind of observations
� and the type of the map (landmarks, grids, …) � and the type of the map (landmarks, grids, …)
Pose graphRobot poses and observations
![Page 12: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/12.jpg)
Our Approach
� measures the relative errors between (consecutive) poses
� which is exactly what approaches to graph-based SLAM seek to minimize.
![Page 13: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/13.jpg)
Metric Based on Relative Poses
estimated relative movement
ground truth
![Page 14: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/14.jpg)
Application to the Example
![Page 15: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/15.jpg)
Translational Error� Error in the beginning:
vs.
� Error in the very end:
![Page 16: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/16.jpg)
Properties
� Good for incremental errors
� Thus far, we only considered incremental links
� This can be sufficient for navigation,� This can be sufficient for navigation,
� but does not help to close loops and achieve global consistency.
![Page 17: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/17.jpg)
Addition of Loop Closure Links
In the loop closure area, the map will be locally inconsistent, which this is not reflected by
If we introduce an appropriate link, we increasethe error, representing the faulty part of the map
![Page 18: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/18.jpg)
Selection of Relations
� No loop closure links: local consistency only (wrt. time)
� Loop closure links: � Loop closure links: local consistency (wrt. space)
� Links between far away nodes: global consistency
![Page 19: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/19.jpg)
Negative example for naive metric
� Minor angular error somewhere in the trajectory leads to a high error in the naive metric
![Page 20: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/20.jpg)
Our Approach
� Consider the energy needed to transform the result of a SLAM algorithm into the ground truth
� Inspiration from the ideas of graph-based SLAM
[ADD EXAMPLE FROM THE PAPER]
![Page 21: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/21.jpg)
Benefit of (Manually Added) Relations for Global Consistency
![Page 22: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/22.jpg)
Relative Relations from a Pose Graph
� Generate the true relative relations given background knowledge
� Compute the error of these relations given the robot’s poses estimated by the SLAM approach
[ADD FORMULAR FROM THE PAPER]
![Page 23: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/23.jpg)
Obtaining Ground Truth Relations
� Besides in simulation, true relations are hard to obtain
� Highly accurate measurement device (e.g., Simeo positioning system or any other sources)positioning system or any other sources)
� How can we use existing datasets for evaluations (Intel Research, MIT infinite corridor, …)?
� For laser-based SLAM, perform scan alignment paired with manual inspection
![Page 24: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/24.jpg)
Determine Relations Manually
� Scan alignment paired with manual inspection can generate close to ground truth relations
� Significant manual efforts but need to be done only once per dataset
[ADD GUI IMAGE FROM THE PAPER][ADD GUI IMAGE FROM THE PAPER]
![Page 25: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/25.jpg)
Graphical Interface to Confirm & Correct Constraints� Scan alignment paired with manual inspection to
generate close to ground truth relations
![Page 26: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/26.jpg)
Experiments
� Application of measure to evaluate measure for different algorithms and environments
� Demonstrate that the scores correlate with the quality of the map
� Demonstrate that the plots allows us to localize errors in the alignments
![Page 27: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/27.jpg)
Relations Available Online
� We generated relations in this way for
� MIT Killian Court
� ACES Building at the University of Texas
� Intel Research Lab Seattle� Intel Research Lab Seattle
� MIT CSAIL Building
� Building 079 University of Freiburg
These are online available at
http://ais.informatik.uni-freiburg.de/slamevaluation/
![Page 28: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/28.jpg)
ACES Building
![Page 29: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/29.jpg)
Intel Lab, Seattle
![Page 30: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/30.jpg)
MIT CSAIL
![Page 31: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/31.jpg)
Building 79 Univ. of Freiburg
![Page 32: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/32.jpg)
Experimental Evaluation
� Compared three approach using all datasets
� Incremental scan-matching
� GMapping (see http://www.openslam.org)
� Graph-based SLAM system based on TORO � Graph-based SLAM system based on TORO (see http://www.openslam.org)
� We provide resulting scores for all algorithms and datasets
![Page 33: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/33.jpg)
Example
![Page 34: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/34.jpg)
Example
![Page 35: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/35.jpg)
Maps Obtained
Scan Matching RBPF Graph mapping
![Page 36: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/36.jpg)
Analyzing Error Plots
![Page 37: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/37.jpg)
Error Plots
Scan Matching RBPF Graph mapping
![Page 38: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/38.jpg)
Numerical Evaluation
![Page 39: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/39.jpg)
Numerical Evaluation
![Page 40: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/40.jpg)
What if no Trajectory is Available?
� Algorithms such as EKF-based SLAM often do not store the whole trajectory of the robot
� No relations can be evaluated
� Two solutions:� Two solutions:
� Recover the trajectory by localization
� Transfer the idea of relations to landmarks (which yields a landmark-pose-graph)
![Page 41: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/41.jpg)
Offline Trajectory Recovery by Localization
� EKF-based approaches provide landmark estimates
� Data associations are made during SLAM
� To recover the poses of the robot during � To recover the poses of the robot during mapping, run EKF-based localization given the landmark locations, data associations, and motion/sensor model used during SLAM
� Approach using relation can be applied directly
![Page 42: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/42.jpg)
Relations based on Landmark Locations
� Alternatively, one can retrieve from known landmark locations relative relations via a triangle mesh
� Take care: data associations between landmarks might need to be done manuallymight need to be done manually
![Page 43: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/43.jpg)
Towards the Rescue of the Sum of Squared Distances
� Let x1:T
be the estimated poses from time 1 to T.
� Let x*1:T
be the reference poses.
� Error function� Error function
where A is the set of affine transformations
![Page 44: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/44.jpg)
Conclusions
� Approach to comparing SLAM algorithms
� Based on relative relations between poses
� Independent of the used algorithm
� Independent of the sensor setup since it operates � Independent of the sensor setup since it operates on the trajectory estimate
� Manually corrected datasets available online
� Results for comparisons available online
![Page 45: A Comparison of SLAM Algorithms Based on a Graph …€¦ · A Comparison of SLAM Algorithms Based on a Graph of Relations Wolfram Burgard, Cyrill Stachniss, Giorgio Grisetti, Bastian](https://reader031.fdocuments.us/reader031/viewer/2022031006/5b8ae9407f8b9a9b7c8d512a/html5/thumbnails/45.jpg)
Limitations
� The measure mostly captures localization
� Assumption is that good localization implies accurate map
� It does not directly capture, whether the map contains all obstacles in the real world. contains all obstacles in the real world.