Lecture 05: Vision
-
Upload
university-of-colorado-at-boulder -
Category
Documents
-
view
401 -
download
3
description
Transcript of Lecture 05: Vision
![Page 1: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/1.jpg)
Introduction to RoboticsPerception II
CSCI 4830/7000February 15, 2010
Nikolaus Correll
![Page 2: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/2.jpg)
Review: Sensing
• Important: sensors report data in their own coordinate frame
• Examples from the exercise– Accelerometer of Nao– Laser scanner
• Treat like forward kinematics
![Page 3: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/3.jpg)
Today
• Perception using vision• Practical angle:– Why is vision hard– Basic image processing– How to combine image processing primitives into
object recognition• OpenCV / SwisTrack
![Page 4: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/4.jpg)
Why is Vision Hard?The difference between seeing and perception.
Gary Bradski, 2009 4
What to do? Maybe we should try to find edges ….
Gary Bradski, 2005
![Page 5: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/5.jpg)
5
• Depth discontinuity• Surface orientation
discontinuity• Reflectance
discontinuity (i.e., change in surface material properties)
• Illumination discontinuity (e.g., shadow)
Slide credit: Christopher Rasmussen
But, What’s an Edge?
![Page 6: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/6.jpg)
To Deal With the Confusion, Your Brain has Rules...
That can be wrong
![Page 7: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/7.jpg)
We even see invisible edges…
![Page 8: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/8.jpg)
And surfaces …
![Page 9: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/9.jpg)
We need to deal with 3D Geometry
9
Perception is ambiguous … depending on your point of view!
Graphic by Gary Bradski
![Page 10: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/10.jpg)
And Lighting in 3D
Which square is darker?
![Page 11: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/11.jpg)
Lighting is Ill-posed …Perception of surfaces depends on lighting assumptions
11Gary Bradski (c) 2008 11
![Page 12: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/12.jpg)
Contrast
12
Which one is male and which one is female?
Illusion by: Richard Russell, Harvard University
Russell, R. (2009) A sex difference in facial pigmentation and its exaggeration by cosmetics. Perception, (38)1211-1219
![Page 13: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/13.jpg)
Frequency
![Page 14: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/14.jpg)
Color
http://briantobin.info/2009/06/lost-and-found-visual-illusion.html
![Page 15: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/15.jpg)
Pin-hole Model
![Page 16: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/16.jpg)
Pin-Hole Camera
A. Efros
![Page 17: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/17.jpg)
Aperture
![Page 18: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/18.jpg)
Increasing Aperture: Lens
![Page 19: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/19.jpg)
Thin Lens
Objects need to have the right distance to be in focus -> Depth-from-Focus method
![Page 20: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/20.jpg)
Thresholds
2020
Screen shots by Gary Bradski, 2005
http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm
![Page 21: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/21.jpg)
Canny Edge Detector
21Gary Bradski (c) 2008 21
![Page 22: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/22.jpg)
Morphological Operations Examples• Morphology - applying Min-Max. Filters and its combinations
Opening IoB= (IB)BDilatation IBErosion IBImage I
Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)
![Page 23: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/23.jpg)
Stereo Calibration
Gary Bradski (c) 2008 2323
Screen shots and charts by Gary Bradski, 2005
![Page 24: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/24.jpg)
3D Stereo Vision• Find Epipolar
lines:
• Triangulate points:
• Align images:
• Depth:
![Page 25: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/25.jpg)
Example: Tomato-Picking Robot
• Challenges– Foliage– Reflections– Varying size and shape– Varying color– Partly covered fruits
http://swistrack.sourceforge.net
N. Correll, N. Arechiga, A. Bolger, M. Bollini, B. Charrow, A. Clayton, F. Dominguez, K. Donahue, S. Dyar, L. Johnson, H. Liu, A. Patrikalakis, T. Robertson, J. Smith, D. Soltero, M. Tanner, L. White, D. Rus. Building a Distributed Robot Garden. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1509-1516, St. Louis, MO.
![Page 26: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/26.jpg)
Filter-based object recognition
• Filter image– Sobel– Hough transform– Color– Spectral
highlights– Size and shape
• Weighted sum of filters highlights object location
Sobel Hough Color SpectralHighlights
![Page 27: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/27.jpg)
Group exercise
• Object recognition– Goal– Players– Ball– Field
![Page 28: Lecture 05: Vision](https://reader036.fdocuments.us/reader036/viewer/2022062511/54c564f24a79590e7d8b45cf/html5/thumbnails/28.jpg)
Homework
• Read sections 4.2-5 (pages 145-180)• Questionnaire on CU Learn