Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge...
Transcript of Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge...
![Page 1: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/1.jpg)
1
Local Image preprocessing (cont’d)
![Page 2: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/2.jpg)
2
Outline
- Edge detectors - Corner detectors
- Reading: textbook 5.3.1-5.3.5 and 5.3.10
![Page 3: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/3.jpg)
3
What are edges?
Edges correspond to relevant features in the image.
An edge corresponds to a change in intensity, surface orientation, shadow etc.
Object recognition and detection use often times edge information (in both CV and biological vision)
![Page 4: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/4.jpg)
4
Why study edges?
Collecting information about edges will help us to identify contours in the image and to retrieve regions enclosed by those contours.
An ‘edge’ image represents a higher level of abstraction (i.e. less information to process)
Edges are features invariant to absolute illumination (as opposed to colour information)
![Page 5: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/5.jpg)
5
Linear filters for edge detection
We need linear filters for: Edge enhancement – image sharpening Removal of false edges (noise
generated)
Edge localization Needs other operations (such as
thresholding)
![Page 6: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/6.jpg)
6
Types of 1D edge profiles
From Forsyth and Ponce
![Page 7: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/7.jpg)
7
Edge representation
From Forsyth and Ponce
![Page 8: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/8.jpg)
8
Edge enhancement Computing the image gradient : vector
composed of first-order partial derivatives
The gradient magnitude gives the amount of the difference between pixels in the neighborhood (the strength of the edge).
The gradient orientation gives the direction of the greatest change, which presumably is the direction across the edge (the edge normal).
Derivatives are linear and shift invariant, thus the gradient can be computed with convolution
![Page 9: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/9.jpg)
9
First order differences of an image
Estimates of derivatives produced by 1st order differences
![Page 10: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/10.jpg)
10
Convolution kernels: Roberts Idea: Approximate the gradient using finite differences
Forward differences
- The Roberts kernels correspond to derivatives with respect to the two diagonal directions
Pros: Only four pixels are needed for the gradient computation
Cons: Masks are too small to obtain reliable results in the presence of noise
![Page 11: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/11.jpg)
11
Convolution kernels: Prewitt
Based on idea of central difference:
very sensitive to noise
Averaging for reducing noise sensitivity:
In y when computing
In x when computing
![Page 12: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/12.jpg)
12
Convolution kernels: Sobel
Also rely on central differences, but give greater weight to central pixels when averaging:
![Page 13: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/13.jpg)
13
From gradient magnitude to edges
The gradient magnitude gives a measure at every pixel of the “edginess” of each pixel:
We still have to find and connect the “best” edge points: Local maxima of gradient magnitude Connected maxima (ridges) Maximum-magnitude contours/paths
Adapted from Brian Morse, http://morse.cs.byu.edu/450/home/index.php
![Page 14: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/14.jpg)
14
From gradient magnitude to edges
The gradient magnitude gives a measure at every pixel of the “edginess” of each pixel:
We still have to find and connect the “best” edge points: Local maxima of gradient magnitude Connected maxima (ridges) Maximum-magnitude contours/paths
Adapted from Brian Morse, http://morse.cs.byu.edu/450/home/index.php
![Page 15: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/15.jpg)
15
From Tomas Svoboda: Image preprocessing in the spatial domain
![Page 16: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/16.jpg)
16
Marr-Hildreth Edge Detection
approximate finding maxima of gradient magnitude (edges) by finding pixels where the Laplacian is zero valued.
Problem: we cannot always find discrete pixels where the Laplacian is exactly zero. Zero-crossings instead
![Page 17: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/17.jpg)
17
Combining first and second order derivatives
Laplacian zero crossings:
Problem: Tells us the gradient magnitude is at a maximum, not how strong it is—lots of spurious edges.
Idea: Combine two measures
From Brian Morse, http://morse.cs.byu.edu/450/home/index.php
![Page 18: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/18.jpg)
18
Edge detection and noise
Partial derivatives are very sensitive to noise image noise results in pixels that look very
different from their neighbours Solution: Gaussian smoothing before filtering
Response of finite differences to noise
σ=0.03 σ=0.09
![Page 19: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/19.jpg)
19
![Page 20: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/20.jpg)
20
Major issues: 1) The gradient magnitude at different values of σ is different; which one should we choose? 2) Large values of the gradient magnitude form thick trails; how do we extract 1 pixel-wide boundaries?
Original image Gradient magnitudes using the derivatives of a Gaussian with σ=1 pixel
σ=2 pixels
![Page 21: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/21.jpg)
21
Laplacian of Gaussian
To include a smoothing Gaussian filter, combine the Laplacian and Gaussian functions to obtain a single equation:
A discrete kernel of the LoG with σ=1.4
![Page 22: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/22.jpg)
22
Laplacian of Gaussian - example
From Efford
![Page 23: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/23.jpg)
23
Limits of LoG
Boundaries of sharp corners are inaccurate; Trihedral vertices are not detected – loops.
Problem with LOG: adds the principal curvatures together; does not determine a gradient of maximum amplitude in any one direction
The Canny Edge Detector defines edges as zero-crossings of second derivatives in the direction of greatest first derivative.
![Page 24: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/24.jpg)
24
The Canny edge detector
Also known as the optimal edge detector Published in IEEE Trans. on Pattern
Analysis and Machine Intelligence, 1986 Intended to enhance the already existent
edge detectors Formulated a list of criteria to enhance
the current methods of edge detection
![Page 25: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/25.jpg)
25
Canny edge detector: criteria
detection: edges occurring in images should not be missed; there should be no responses to non-edges.
edge points have to be well localized. The distance between the edge pixels as found by the detector and the actual edge is to be minimal.
Only one response at a single edge is allowed. Eliminates multiple detections of the same edge.
Note: There is a trade-off between good detection and good localization. Why?
![Page 26: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/26.jpg)
26
Canny edge detector: main steps
1. Filter out noise in the original image before trying to locate and detect any edges.
2. Compute the image gradient to highlight regions with high spatial derivatives.
3. Non-maximum suppression 4. Hysteresis
![Page 27: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/27.jpg)
27
Canny edge detector: Step 1+2
We can implement both steps by filtering with a Difference of Gaussian (DoG) filter
or: Step 1: convolution with a Gaussian Step 2: computation of spatial
derivatives with Sobel masks. Approximate edge strength with: |G| = |Gx| + |Gy|
![Page 28: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/28.jpg)
28
Canny edge detector: Non maxima suppression
The gradient magnitude tends to be large along thick trails in an image.
- To replace this thick trail with a representative thin curve, we have to find a cut direction and to extract the intensity maxima along this direction.
- The gradient direction is a good estimate of the cut direction
![Page 29: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/29.jpg)
29
Canny edge detector: non maximum suppression
Nonmaximum suppression obtains points where the gradient magnitude is at a maximum along the direction of the gradient.
- The gradient direction (forward and backward) does not pass through any pixels on the grid.
- Linear interpolation for computing the values of gradient magnitude at p and r (considering the pixels to the left and right of p and r)
- if G(q)>max(Gint(p), Gint(r)) then q is retained as an edge point;
else q is suppressed.
![Page 30: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/30.jpg)
30
Canny edge detector: hysteresis
In spite of the smoothing step, the non maxima suppressed magnitude image will contain many false edge fragments caused by noise and fine texture.
Solution: thresholding One threshold is not enough, since the gradient
magnitude is variable Thresholding with hysteresis: - retain all edge candidates with gradient
magnitude>TH
- Edge following by linking edge candidates with gradient magnitude>TL
![Page 31: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/31.jpg)
31
![Page 32: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/32.jpg)
32
Strong edges reinforce weak edges
![Page 33: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/33.jpg)
33
Matlab implementation of Canny BW = edge(I,'canny') specifies the Canny method.
BW = edge(I,'canny',thresh) specifies sensitivity thresholds for the Canny method. thresh is a two-element vector in which the first element is the low threshold, and the second element is the high threshold.
If you specify a scalar for thresh, this value is used for the high threshold and 0.4*thresh is used for the low threshold. If you do not specify thresh, or if thresh is empty ([]), edge chooses low and high values automatically.
BW = edge(I,'canny',thresh,sigma) specifies the Canny method, using sigma as the standard deviation of the Gaussian filter. The default sigma is 1; the size of the filter is chosen automatically, based on sigma.
[BW,thresh] = edge(I,'canny',...) returns the threshold values as a two-element vector.
![Page 34: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/34.jpg)
34
Edge detectors: examples
Edge detectors by gradient operators (Prewitt, Roberts, Sobel)
![Page 35: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/35.jpg)
35
Edge detectors: examples (2)
Edge detection with DOG and LOG
![Page 36: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/36.jpg)
36
Edge detectors: examples
Edge detection with Canny: σ=1,2,3 τ1=0.3 τ2=0.7
![Page 37: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/37.jpg)
37
What to remember
Edge = Rapid intensity (color) change Edge information is one of the most
important in CV and Human Vision Three steps in edge detection:
Noise reduction (image smoothing) Edge enhancement Edge localisation
Three types were presented: Based on first-order derivative
Roberts, Prewitt and Sobel Based on second-order derivative
Laplacian of Gaussian Based on linking edges
Canny
![Page 38: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/38.jpg)
38
Corner detection
Corners are important features in the image
Useful for matching two images of the same static scene acquired from different viewpoints (stereo vision, building mosaics)
Useful for object recognition as well
![Page 39: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/39.jpg)
The Moravec corner detector
Developed by Hans Moravec in 1977 in his research about autonomous navigation and obstacle avoidance
http://www.frc.ri.cmu.edu/~hpm/project.archive/robot.papers/1977/aips.txt
- defined the concept of "points of interest” - these interest points could be used to find matching regions in consecutive image frames - interest points=points where there is a large intensity variation in every direction
Slides about Moravec detector use materials from - http://www.cim.mcgill.ca/~dparks/CornerDetector/moravec.htm 39
![Page 40: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/40.jpg)
The Moravec corner detector: measuring the intensity variation
40
![Page 41: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/41.jpg)
The Moravec corner detector: why does it detect corners?
41
![Page 42: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/42.jpg)
42
![Page 43: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/43.jpg)
Cornerness map
43
![Page 44: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/44.jpg)
Moravec-results
44
![Page 45: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/45.jpg)
Moravec-limitations Anisotropic response; large response to diagonal
edges
45
![Page 46: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/46.jpg)
The Harris corner detector
Improves upon Moravec Isotropic More computationally intensive
46
![Page 47: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/47.jpg)
47
Change
No Change
Ideal vertical edge
![Page 48: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/48.jpg)
48
No Change
Change
Ideal horizontal edge
![Page 49: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/49.jpg)
49
Harris corner detector
C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
![Page 50: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/50.jpg)
50
The Basic Idea We should easily recognize the point by looking
through a small window Shifting a window in any direction should change
significantly the intensity distribution inside the window
From Darya Frolova, Denis Simakov The Weizmann Institute of Science Lecture notes on Matching with Invariant features
![Page 51: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/51.jpg)
51
Harris Detector: Basic Idea
“flat” region: no change in all directions
“edge”: no change along the edge direction
“corner”: significant change in all directions
From Darya Frolova, Denis Simakov The Weizmann Institute of Science Lecture notes on Matching with Invariant features
![Page 52: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/52.jpg)
52
Harris Detector: Mathematics
Change of intensity for the shift [Δx, Δy]:
Intensity Shifted
intensity
or Window function w(x,y) =
Gaussian 1 in window, 0 outside
€
S(x, y) = f xi , yi( ) − f xi − Δx, yi − Δy( )[ ](xi ,yi )∈W∑
2
![Page 53: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/53.jpg)
53
Harris Detector: Mathematics
We will show that
Where A captures the intensity structure of the local neighborhood. Its main modes of variation are reflected in its eigenvalues λ1 and λ2
€
S(x, y) = Δx,Δy[ ]AW (x, y)ΔxΔy⎡
⎣ ⎢
⎤
⎦ ⎥
![Page 54: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/54.jpg)
54
Harris Detector: Mathematics
λ1
λ2
“Corner” λ1 and λ2 are large, λ1 ~ λ2; E increases in all directions
λ1 and λ2 are small; E is almost constant in all directions
“Edge” λ1 >> λ2
“Edge” λ2 >> λ1
“Flat” region
Classification of image points using eigenvalues of A:
From Darya Frolova & Denis Simakov
![Page 55: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/55.jpg)
55
Harris Detector: Mathematics (cont’d)
Measure of corner response:
(k – tunable empirical constant, k = 0.04-0.15)
€
R(A) = det(A) − k ⋅ trace2 (A)
Det(A)=λ1λ2
Trace(A)=λ1+ λ2
But: we do not need to compute λ1 and λ2 for the computation of R.
![Page 56: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/56.jpg)
56
Harris Detector: Mathematics
λ1
λ2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flat region
R > 0
R < 0
R < 0 |R| small From Darya Frolova, Denis Simakov Lecture notes on Matching with Invariant features
![Page 57: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/57.jpg)
57
Harris Detector: the algorithm
Filter the image with a Gaussian Compute the horizontal and vertical
gradient images with Sobel masks or a double 1D convolution
Define the neighborhood W (preferably gaussian, thus isotropic)
Find points with large corner response function R(A) (R > threshold)
Take the points of local maxima of R
![Page 58: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/58.jpg)
58
Harris Detector: Workflow Compute corner response R
![Page 59: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/59.jpg)
59
Harris Detector: Workflow Find points with large corner response: R>threshold
![Page 60: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/60.jpg)
60
Harris Detector: Workflow Take only the points of local maxima of R
![Page 61: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/61.jpg)
61
Harris Detector: Workflow
![Page 62: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/62.jpg)
62
Harris detector - examples
![Page 63: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/63.jpg)
63
Harris Detector: Some Properties
Rotation invariance
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response R is invariant to image rotation
From Darya Frolova, Denis Simakov Lecture notes on Matching with Invariant features
![Page 64: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/64.jpg)
64
Harris Detector: Some Properties
Partial invariance to affine intensity change
Only derivatives are used => invariance to intensity shift I → I + b
Intensity scale: I → a I
R
x (image coordinate)
threshold
R
x (image coordinate)
![Page 65: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/65.jpg)
65
Harris Detector: Some Properties
But: non-invariant to image scale!
All points will be classified as edges
Corner !
From Darya Frolova, Denis Simakov The Weizmann Institute of Science Lecture notes on Matching with Invariant features
![Page 66: Local Image preprocessing (cont’d) - UVic.caaalbu/computer vision 2010/L11-12... · 8 Edge enhancement Computing the image gradient : vector composed of first-order partial derivatives](https://reader034.fdocuments.us/reader034/viewer/2022042311/5eda061c4e02a025e73daccb/html5/thumbnails/66.jpg)
Invariance to scale: SIFT
D. Lowe, 2004: Distinctive image features from scale-invariant keypoints
Paper is mandatory reading for ELEC 536; optional for CENG 421.
66