Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.
-
Upload
merilyn-heather-sanders -
Category
Documents
-
view
231 -
download
0
Transcript of Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.
![Page 1: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/1.jpg)
Lecture 4: Feature matching
CS4670 / 5670: Computer VisionNoah Snavely
![Page 2: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/2.jpg)
Announcements
• Project 1 artifact due tonight at 11:59pm (CMS)
• Quiz on Friday
![Page 3: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/3.jpg)
Basic idea:• Take 16x16 square window around detected feature• Compute edge orientation (angle of the gradient - 90) for each pixel• Throw out weak edges (threshold gradient magnitude)• Create histogram of surviving edge orientations
Scale Invariant Feature Transform
Adapted from slide by David Lowe
0 2
angle histogram
![Page 4: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/4.jpg)
SIFT descriptorFull version
• Divide the 16x16 window into a 4x4 grid of cells (2x2 case shown below)• Compute an orientation histogram for each cell• 16 cells * 8 orientations = 128 dimensional descriptor
Adapted from slide by David Lowe
![Page 5: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/5.jpg)
Properties of SIFTExtraordinarily robust matching technique
– Can handle changes in viewpoint• Up to about 60 degree out of plane rotation
– Can handle significant changes in illumination• Sometimes even day vs. night (below)
– Fast and efficient—can run in real time– Lots of code available
• http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFT
![Page 6: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/6.jpg)
SIFT Example
siftsift
868 SIFT features
![Page 7: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/7.jpg)
Feature matching
Given a feature in I1, how to find the best match in I2?1. Define distance function that compares two
descriptors2. Test all the features in I2, find the one with min
distance
![Page 8: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/8.jpg)
Feature distanceHow to define the difference between two features f1, f2?
– Simple approach: L2 distance, ||f1 - f2 ||
– can give good scores to ambiguous (incorrect) matches
I1 I2
f1 f2
![Page 9: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/9.jpg)
f1 f2f2'
Feature distanceHow to define the difference between two features f1, f2?
• Better approach: ratio distance = ||f1 - f2 || / || f1 - f2’ || • f2 is best SSD match to f1 in I2
• f2’ is 2nd best SSD match to f1 in I2
• gives large values for ambiguous matches
I1 I2
![Page 10: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/10.jpg)
Feature matching example
51 matches
![Page 11: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/11.jpg)
Feature matching example
58 matches
![Page 12: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/12.jpg)
Evaluating the resultsHow can we measure the performance of a feature matcher?
5075
200
feature distance
![Page 13: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/13.jpg)
True/false positives
The distance threshold affects performance– True positives = # of detected matches that are correct
• Suppose we want to maximize these—how to choose threshold?– False positives = # of detected matches that are incorrect
• Suppose we want to minimize these—how to choose threshold?
5075
200false match
true match
feature distance
How can we measure the performance of a feature matcher?
![Page 14: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/14.jpg)
0.7
Evaluating the results
0 1
1
false positive rate
truepositive
rate
# true positives# correctly matched features (positives)
0.1
How can we measure the performance of a feature matcher?
“recall”
# false positives# incorrectly matched features (negatives)
1 - “precision”
![Page 15: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/15.jpg)
0.7
Evaluating the results
0 1
1
false positive rate
truepositive
rate
0.1
ROC curve (“Receiver Operator Characteristic”)
How can we measure the performance of a feature matcher?
# true positives# correctly matched features (positives)
“recall”
# false positives# incorrectly matched features (negatives)
1 - “precision”
![Page 16: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/16.jpg)
Lots of applications
Features are used for:– Image alignment (e.g., mosaics)– 3D reconstruction– Motion tracking– Object recognition (e.g., Google Goggles)– Indexing and database retrieval– Robot navigation– … other
![Page 17: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/17.jpg)
Object recognition (David Lowe)
![Page 18: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/18.jpg)
3D Reconstruction
Internet Photos (“Colosseum”) Reconstructed 3D cameras and points
![Page 19: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/19.jpg)
Sony Aibo
SIFT usage:
Recognize charging station
Communicate with visual cards
Teach object recognition
![Page 20: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/20.jpg)
Questions?
![Page 21: Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649ea35503460f94ba762a/html5/thumbnails/21.jpg)
Image alignment
Why don’t these image line up exactly?