Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G....
-
Upload
monica-black -
Category
Documents
-
view
225 -
download
2
Transcript of Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G....
![Page 1: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/1.jpg)
Lecture 2Imaging Geometry and OpenCV
Slides by:David A. Forsyth
Clark F. OlsonJean Ponce
Linda G. Shapiro
![Page 2: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/2.jpg)
• Abstract camera model - box with a small hole in it
• Pinhole cameras work in practice
Pinhole Cameras
![Page 3: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/3.jpg)
Distant objects are smaller
![Page 4: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/4.jpg)
Common to draw film planein front of the focal point.Moving the film plane merelyscales the image.
Parallel lines meet
![Page 5: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/5.jpg)
• Cartesian coordinates:– We have, by similar triangles, that:
– (X, Y, Z) ~ (f X/Z, f Y/Z, f)
– f is called the focal length.),(),,(
Z
Yf
Z
XfZYX
[X, Y, Z]
[fX/Z, fY/Z]
The equation of projection
![Page 6: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/6.jpg)
We won’t worry much about lenses in this class.
The reason for lenses
![Page 7: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/7.jpg)
Lens distortion
“Barrel distortion” of rectangular grid is common for inexpensive lenses.
Precision lenses can be expensive.
Zoom lenses often show severe distortion.
Fish-eye lenses also have severe distortion.
![Page 8: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/8.jpg)
• Images are not continuous• Typically captured with a CCD camera (charge-coupled-device)• The amount of light striking each location on a grid is integrated over
some time period• Rows are read out one at a time• For color images, successive
pixels usually correspond to different colors
• High quality color cameras usea beam splitter and 3 separateCCD chips
• APS (active pixel sensor) is acheaper (lower quality) technology
Image capture
![Page 9: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/9.jpg)
Resolution
Resolution often (but not always) refers to the number of pixels in the image.
Lower resolution has fewer pixels.
Interestingly, faces of people you know can usually be recognized at 64x64 (or less) pixels.
Squint and look at the lowest resolution image.
![Page 10: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/10.jpg)
Some of my research
Time permitting, the following slides give a very brief overview of some of my previous and current research.
10
![Page 11: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/11.jpg)
Visual Terrain Mapping for Mars Exploration
11
![Page 12: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/12.jpg)
Pose Sampling for Efficient Model-Based Recognition
12
Careful sampling of the viewpoints reducesthe complexity from cubic to linear in thenumber of craters.
![Page 13: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/13.jpg)
Robust Registration of Aerial Image Sequences
13
Goal: Provide registration between images and maps for persistent aerial surveillance
Registration results
![Page 14: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/14.jpg)
Simple and Efficient Projective Clustering
14
Projective clustering problems have the following properties:• Many dimensions - d• Many points – n• Clusters do not form in
the full d-dimensional space
• No labeled exemplars
![Page 15: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/15.jpg)
Keypoint recognition
15
A popular object recognition technique uses descriptive “keypoints” that have been extracted from images.
We are investigating getting better use out of the color information in the image when creating keypoint descriptors.
![Page 16: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/16.jpg)
16
Image classificationCurrent techniques for image classification cluster image keypoints into “visual words” similar to text retrieval methods.
We are studying the use of projective clustering to improve performance.
http://www.sccs.swarthmore.edu/users/09/btomasi1/tagging-products.html
![Page 17: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/17.jpg)
Programming in OpenCV
In OpenCV, images are represented as matrices (as in linear algebra).
Mat image = imread("photo.jpg"); // Most generic declaration
The image could have a number of underlying data types for each pixel:
uchar – unsigned byte (greyscale image)
Vec3b – vector of 3 bytes (color image)
Point2f – point in two dimensions, float
many others…
![Page 18: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/18.jpg)
Creating images
Images can be created using a number of methods:
using namespace cv; // all my code assumes thisMat image; // creates 0x0 imageMat image = … // uses copy constructorMat image(rows, cols, type); // type is CV_8U, for exampleMat image(rows, cols, type, scalarValue);
Example: Mat allBlue(360, 480, CV_8UC3, Scalar(255, 0, 0));
Mat_<Vec3b> colorImage = imread(“color.jpg”);// Can be convenient, but now limited to Vec3b images (matrices)// Also, must declare as a similar parameter type when passed
![Page 19: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/19.jpg)
Copying images
Be careful to remember that most image copy and pass by value methods do NOT perform a deep copy.
image2 = image1; // shallow copy
void someMethod(Mat imageParam); // shallow copy
If you want a deep copy, then use clone (or copyTo):
image2 = image1.clone(); // deep copyimage1.copyTo(image2); //
deep copy
![Page 20: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/20.jpg)
Memory management
Memory management is handled by the Mat class.• This is different from the IplImage class in OpenCV 1• This works correctly even if multiple images share the same data• A reference count is kept for each image• The data is deallocated only when the reference count goes to
zero• However, this can allow privacy leaks unless you are careful
![Page 21: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/21.jpg)
Backwards compatibility
OpenCV 2 is backwards compatible with OpenCV 1.
IplImage *iplIm = cvLoadImage("photo.jpg");// Do some work with image herecvReleaseImage(&iplIm); // necessary to prevent
memory leak
Can convert to Mat simply:
Mat converted(iplIm); // do not release image until finished
![Page 22: Lecture 2 Imaging Geometry and OpenCV Slides by: David A. Forsyth Clark F. Olson Jean Ponce Linda G. Shapiro.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d0b5503460f949df6f9/html5/thumbnails/22.jpg)
Image manipulation
OpenCV provides many methods to manipulate entire images:
Filtering: blur, smooth, median, gradient, laplacian
Transformations: resize, affine, perspective, color space, threshold, flood fill
Segmentation: grabCut, watershed
Feature detection: edges, corners, lines, circles, template matching, SIFT