Interest points
description
Transcript of Interest points
![Page 1: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/1.jpg)
Interest points
CSE 576Ali Farhadi
Many slides from Steve Seitz, Larry Zitnick
![Page 2: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/2.jpg)
How can we find corresponding points?
![Page 3: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/3.jpg)
Not always easy
NASA Mars Rover images
![Page 4: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/4.jpg)
NASA Mars Rover imageswith SIFT feature matchesFigure by Noah Snavely
Answer below (look for tiny colored squares…)
![Page 5: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/5.jpg)
Human eye movements
Yarbus eye tracking
![Page 6: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/6.jpg)
Interest points• Suppose you have to click
on some point, go away and come back after I deform the image, and click on the same points again. • Which points would you
choose?
original
deformed
![Page 7: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/7.jpg)
Intuition
![Page 8: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/8.jpg)
Corners• We should easily recognize the point by looking
through a small window• Shifting a window in any direction should give a
large change in intensity
“edge”:no change along the edge direction
“corner”:significant change in all directions
“flat” region:no change in all directionsSource: A. Efros
![Page 9: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/9.jpg)
Let’s look at the gradient distributions
![Page 10: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/10.jpg)
Principle Component AnalysisPrincipal component is the direction of highest variance.
How to compute PCA components:
1. Subtract off the mean for each data point.2. Compute the covariance matrix.3. Compute eigenvectors and eigenvalues.4. The components are the eigenvectors
ranked by the eigenvalues.
Next, highest component is the direction with highest variance orthogonal to the previous components.
![Page 11: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/11.jpg)
Corners have …
Both eigenvalues are large!
![Page 12: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/12.jpg)
xII x
yII y
yI
xIII yx
Second Moment Matrix
2 x 2 matrix of image derivatives (averaged in neighborhood of a point).
Notation:
![Page 13: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/13.jpg)
The mathTo compute the eigenvalues:
1. Compute the covariance matrix.
2. Compute eigenvalues.
Typically Gaussian weights
![Page 14: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/14.jpg)
Corner Response Function• Computing eigenvalues are expensive• Harris corner detector uses the following alternative
Reminder:
![Page 15: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/15.jpg)
Harris detector: Steps
1. Compute Gaussian derivatives at each pixel2. Compute second moment matrix M in a Gaussian
window around each pixel 3. Compute corner response function R4. Threshold R5. Find local maxima of response function (nonmaximum
suppression)
C.Harris and M.Stephens. “A Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference: pages 147—151, 1988.
![Page 16: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/16.jpg)
Harris Detector: Steps
![Page 17: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/17.jpg)
Harris Detector: StepsCompute corner response R
![Page 18: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/18.jpg)
Harris Detector: StepsFind points with large corner response: R>threshold
![Page 19: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/19.jpg)
Harris Detector: StepsTake only the points of local maxima of R
![Page 20: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/20.jpg)
Harris Detector: Steps
![Page 21: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/21.jpg)
Simpler Response Function
![Page 22: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/22.jpg)
Properties of the Harris corner detector
• Translation invariant?• Rotation invariant? • Scale invariant?
All points will be classified as edges
Corner !
Yes
NoYes
![Page 23: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/23.jpg)
Scale
Let’s look at scale first:
What is the “best” scale?
![Page 24: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/24.jpg)
Scale Invariance
K. Grauman, B. Leibe
)),(( )),((11
xIfxIfmm iiii
How can we independently select interest points in each image, such that the detections are repeatable across different scales?
![Page 25: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/25.jpg)
Differences between Inside and Outside
![Page 26: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/26.jpg)
Scale
Why Gaussian?
It is invariant to scale change, i.e., and has several other nice properties. Lindeberg, 1994
In practice, the Laplacian is approximated using a Difference of Gaussian (DoG).
![Page 27: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/27.jpg)
Difference-of-Gaussian (DoG)
K. Grauman, B. Leibe
- =
![Page 28: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/28.jpg)
DoG example
σ = 1
σ = 66
![Page 29: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/29.jpg)
)()( yyxx LL
1
2
3
4
5
List of (x, y, σ)
scale
Scale invariant interest pointsInterest points are local maxima in both position
and scale.
Squared filter response maps
![Page 30: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/30.jpg)
Scale
In practice the image is downsampled for larger sigmas.
Lowe, 2004.
![Page 31: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/31.jpg)
Results: Difference-of-Gaussian
K. Grauman, B. Leibe
![Page 32: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/32.jpg)
How can we find correspondences?
Similarity transform
![Page 33: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/33.jpg)
CSE 576: Computer Vision
Rotation invariance
Image from Matthew Brown
• Rotate patch according to its dominant gradient orientation
• This puts the patches into a canonical orientation.
![Page 34: Interest points](https://reader034.fdocuments.us/reader034/viewer/2022051317/56815f01550346895dcdbdd9/html5/thumbnails/34.jpg)
T. Tuytelaars, B. Leibe
Orientation Normalization• Compute orientation histogram• Select dominant orientation• Normalize: rotate to fixed orientation
0 2p
[Lowe, SIFT, 1999]