74.795 (L01)

Computer Vision for

Mobile Robotics

Winter 2005

Instructor: Jacky Baltes402 E2 EITCPhone: +1 (204) 474-8838Email: jacky@cs.umanitoba.caWWW: http://www.cs.umanitoba.ca/~jacky

Prerequisites: Consent of instructor. Solid programming experience as well as a basicknowledge of AI concepts and Embedded Systems is requiredLecture Times: Mon, Wed, Fri 17:00 - 19:00 304 E2 EITC

Introduction

This contents of this course areThis course covers a variety of topics and issues in applied AI: computer vision, mobile

robotics, and multi-robot systems. These abstract components are grounded in the problemof robotic soccer.

In four assignments, the students will implement the components for a team of intelligentmobile robots capable of performing interesting and challenging tasks such as robotic soccerand urban search and rescue.

The course is based on programming uses physical robots, all topics are covered withspeci�c emphasis on real-time performance.

A selection of topics covered during lectures are:

� Image processing : edge detection, thresholding, color models (RGB, HSI, SCT), �ltercascades.

� Image segmentation: gradient based methods, area based methods algorithm, Houghtransform, watershed algorithm.

� Optical ow analysis : Horn Schnuck algorithm, Lucas Kanade computation.

� Ego-motion estimation: visual servoing, model matching.

� Stereo vision: Camera calibration, epi-polar geometry, fundamental matrix, pixel andregion-based approaches to matching.

� Control : Kinematic models of (car-like) robots, PID control, CMACs, Neural Nets.

� Local Path planning and navigation: Exploration algorithms, obstacle avoidance, landmark-based navigation.

� Localization: Monte-Carlo methods, particle �elds, distance �lters.

� Mapping : occupancy grids, topological maps, simultaneous localization and mapping(SLAM).

Grading

The course mark is determined by practical work (60% Assignments) and a �nal exam forthe underlying theory (40%).

1

Assignments (60%)

Course assessment includes a large practical component. There are four assignments coveringspeci�c topics in the problem domain. Each assignment is worth 15%. The following listcontains some sample assignment topics:

1. Implement a region segmentation algorithm (e.g., watershed)

2. Path following control of a mobile robot. Implement region segmentation and a visualservoing based line tracker,

3. Implement a vision system to estimate the ego-motion of a simple mobile robot.

4. Localization in a structured environment. Follow an unmarked path in a mappedenvironment,

5. Visual SLAM. Create a map of an unstructured environment and �nd an object hiddenin the environment.

6. Compute depth maps using an area-based stereo matching algorithm.

Final Exam (40%)

There will be a �nal exam worth 40%. The exam will be three hours long. The exam willbe held during the examination period at the end of the term. Exact time and location willbe advertised by Student Records.

Textbook

I will use research papers describing both some of the background material as well as thecurrent state of the art. Students are expected to understand the material as well as beingable to implement a simple version of the algorithm.

A reading list is attached to the end of this document. I will be using selections from thethese publications, all of which will be made available online or in the library.

Reading List

[1] P. Bakker and Y. Kuniyoshi. Robot see, robot do: An overview of robot imitation. InAISB96 Workshop: Learning in Robots and Animals, 1996.

[2] D. H. Ballard. Generalizing the hough transform to detect arbitrary shapes. PatternRecognition, 13(2):111{122, 1981.

[3] A. Balluchi, A. Bicchi, A. Balestrino, and G. Casalino. Path tracking control fordubin's cars. In Proceedings of the IEEE International Conference on Robotics andAutomation, 1996.

[4] Jacky Baltes. Yuefei: Object orientation and id without additional markers. InRoboCup-01: Robot Soccer World Cup V, New York, 2002. Springer.

[5] Jacky Baltes and Nicholas Hildreth. Adaptive path planner for highly dynamic environ-ments. In Peter Stone, Tucker Balch, and Gerhard Kraetszchmar, editors, RoboCup-2000: Robot Soccer World Cup IV, pages 76{85. Springer Verlag, Berlin, 2001.

[6] Jacky Baltes and Yuming Lin. Path-tracking control of non-holonomic car-like robotsusing reinforcement learning. In Proceedings of the IJCAI Workshop on RoboCup,Stockholm, Sweden, July 1999.

[7] Jacky Baltes and Yong Joo Park. Comparison of several machine learning techniquesin pursuit-evasion games. In RoboCup-01: Robot Soccer World Cup V, New York, 2002.Springer.

[8] J. Batista, H. Araujo, and A.T. Almeida. Monoplanar camera calibration: Iterativemultistep approach. In BMVC93, page xx, 1993.

[9] Richard Beer. Intelligence as Adaptive Behaviour. Academic Press, 1990.[10] M. Betke. Learning and vision algorithms for robot navigation. Technical Report

MIT/LCS/TR-671, 1995.

[11] Magrit Betke, Ronald L. Rivest, and Mona Singh. Piecemeal learning of an unknownenvironment. Technical Report AIM-1474, 1994.

[12] Robert Brooks. A robust layered control system for a mobile robot. Technical Report864, Massachusetts Institute of Technology, 1985.

[13] Rodney Brooks and Lynn A. Stein. Building brains for bodies. Technical ReportAIM-1439, 1993.

[14] Claudio Castelpietra, Luca Iocchi, Daniele Nardi, Maurizio Piaggio, Alessandro Scalzo,, and Antonio Sgorbissa. Communication and coordination among heterogeneous mid-size players: Art99. In Peter Stone, Tucker Balch, and Gerhard Kraetszchmar, editors,RoboCup-2000: Robot Soccer World Cup IV, pages 86{95. Springer Verlag, Berlin,2001.

[15] D. Fox, W. Burgard, and S. Thrun. Active markov localization for mobile robots, 1998.[16] Dieter Fox, Wolfram Burgard, and Sebastian Thrun. The dynamic window approach

to collision avoidance. Technical Report IAI-TR-95-13, 1, 1995.[17] Dieter Fox, Wolfram Burgard, and Sebastian Thrun. Markov localization for mobile

robots in dynamic environments. Journal of Arti�cial Intelligence Research, 11:391{427, 1999.

[18] K.S. Fu, R.C. Gonzales, and C. S. G. Lee. Robotics: Control Sensing, Vision, andIntelligence, chapter 7.4, pages 306{324. McGraw Hill, 1987.

[19] Kristian J. Hammond. Case Based Planning. Academic Press Inc., 1989.[20] K. Y. Hough. Mehod and means for recognizing complex patterns. U.S. Patent 3069654,

1962.[21] Harukazu Igarashi. Path planning of a mobile robot as a discrete optimization problem

and adjustment of weight parameters in the objective function by reinforcement learn-ing. In Peter Stone, Tucker Balch, and Gerhard Kraetszchmar, editors, RoboCup-2000:Robot Soccer World Cup IV, pages 315{320. Springer Verlag, Berlin, 2001.

[22] M. Isard and A. Blake. Condensation { conditional density propagation for visualtracking, 1998.

[23] S. Julier and J. Uhlmann. A new extension of the kalman �lter to nonlinear systems,1997.

[24] Benjamin Kuipers and Yung-Tai Byun. A robot exploration and mapping strategybased on a semantic hierarchy of spatial representations. Technical Report AI90-120,1, 1990.

[25] J.C. Latombe. Robot Motion Planning. Kluwer Academic Publishers, Boston, 1991.[26] Michael Montemerlo and Sebastian Thrun. Fastslam 2.0: An improved particle �ltering

algorithm for simultaneous localization and mapping that provably converges.[27] A. W. Moore. E�cient Memory-based Learning for Robot Control. PhD thesis, Cam-

bridge, UK, 1990.[28] Stuart Russel and Peter Norvig. Arti�cial Intelligence: A Modern Approach, chap-

ter 20, pages 598{624. Prentice-Hall Inc., Englewood Cli�s, New Jersey 07632, 1995.[29] D. Schulz, W. Burgard, D. Fox, and A. Cremers. Tracking multiple moving targets

with a mobile robot using particle �lters and statistical data association, 2001.[30] R. Sim and G. Dudek. Mobile robot localization from learned landmarks.[31] Robert Sim. Bayesian exploration for mobile robots.[32] Andrew James Thomson. A path following system for autonomous robots with minimal

computing power. Msc., University of Auckland, Private Bag 92019, Auckland, NewZealand, July 2001.

[33] Sebastian Thrun, Wolfram Burgard, and Dieter Fox. A probabilistic approach toconcurrent mapping and localization for mobile robots. Machine Learning, 31(1-3):29{53, 1998.

[34] Sebastian Thrun, Dieter Fox, Wolfram Burgard, and Frank Dellaert. Robust montecarlo localization for mobile robots. Arti�cial Intelligence, 128(1-2):99{141, 2001.

[35] David E. Wilkins. Practical Planning: Extending the classical AI planning paradigm.Morgan Kaufmann Publishers, San Mateo, CA., 1988.

[36] J�urgen Wolf, Wolfram Burgard, and Hans Burkhardt. Robust vision-based localizationfor mobile robots using an image retrieval system based on invariant features.

[37] B. Yamauchi and R. Beer. Spatial learning for navigation in dynamic environments,1996.