CSE 185 Introduction to Computer Vision Local Invariant Features.
CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.
-
Upload
ferdinand-hall -
Category
Documents
-
view
217 -
download
1
Transcript of CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.
![Page 1: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/1.jpg)
CSE 185 Introduction to Computer
VisionEdges
![Page 2: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/2.jpg)
Edges
• Edges• Scale space
• Reading: Chapter 3 of S
![Page 3: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/3.jpg)
Edge detection• Goal: Identify sudden
changes (discontinuities) in an image– Intuitively, most semantic
and shape information from the image can be encoded in the edges
– More compact than pixels• Ideal: artist’s line
drawing (but artist is also using object-level knowledge)
![Page 4: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/4.jpg)
Why do we care about edges?• Extract information,
recognize objects
• Recover geometry and viewpoint
Vanishing point
Vanishing
line
Vanishing point
Vertical vanishing point
(at infinity)
![Page 5: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/5.jpg)
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
![Page 6: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/6.jpg)
Marr’s theory
![Page 7: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/7.jpg)
Example
![Page 8: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/8.jpg)
Closeup of edges
Source: D. Hoiem
![Page 9: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/9.jpg)
Closeup of edges
![Page 10: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/10.jpg)
Closeup of edges
![Page 11: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/11.jpg)
Closeup of edges
![Page 12: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/12.jpg)
Characterizing edges• An edge is a place of rapid change in the
image intensity function
imageintensity function
(along horizontal scanline) first derivative
edges correspond toextrema of derivative
![Page 13: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/13.jpg)
Intensity profile
![Page 14: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/14.jpg)
With a little Gaussian noise
Gradient
![Page 15: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/15.jpg)
Effects of noise• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?
![Page 16: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/16.jpg)
Effects of noise
• Difference filters respond strongly to noise– Image noise results in pixels that look
very different from their neighbors– Generally, the larger the noise the
stronger the response
• What can we do about it?
![Page 17: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/17.jpg)
Solution: smooth first
• To find edges, look for peaks in )( gfdx
d
f
g
f * g
)( gfdx
d
![Page 18: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/18.jpg)
• Differentiation is convolution, and convolution is associative:
• This saves us one operation:
gdx
dfgf
dx
d )(
Derivative theorem of convolution
gdx
df
f
gdx
d
![Page 19: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/19.jpg)
Derivative of Gaussian filter
* [1 -1] =
![Page 20: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/20.jpg)
Differentiation and convolution• Recall
• Now this is linear and shift invariant, so must be the result of a convolution
• We could approximate this as
(which is obviously a convolution; it’s not a very good way to do things, as we shall see)
),(),(lim
0
yxfyxf
x
f
x
yxfyxf
x
f nn
),(),( 1
000
101
000
H
![Page 21: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/21.jpg)
Discrete edge operators• How can we differentiate a discrete image?
Finite difference approximations:
1, jiI 1,1 jiI
jiI , jiI ,1
Convolution masks :
jijijiji
jijijiji
IIIIy
I
IIIIx
I
,1,,11,1
,,11,1,1
2
12
1
2
1
x
I
2
1
y
I
![Page 22: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/22.jpg)
1, jiI 1,1 jiI
jiI , jiI ,1
1,1 jiI
jiI ,1
1,1 jiI 1, jiI 1,1 jiI
• Second order partial derivatives:
• Laplacian :
Convolution masks :
or
Discrete edge operators
(more accurate)
1 4 1
4 -20
4
1 4 1
0 1 0
1 -4 1
0 1 0
1,,1,22
2
,1,,122
2
21
21
jijiji
jijiji
IIIy
I
IIIx
I
2
2
2
22
y
I
x
II
22 1
I
26
1
![Page 23: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/23.jpg)
Finite differences
Partial derivative in y axis, respond strongly tohorizontal edges
Partial derivative in x axis, respond strongly to vertical edges
![Page 24: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/24.jpg)
Spatial filter
• Approximation
|)||)(|||
|)||)(|||
|)||)(|||
])()[(||
||,
8695
2/18695
6585
2/1265
285
2/122
zzzzf
zzzzf
zzzzf
zzzzf
y
f
x
ff
y
fx
f
f
987
654
321
zzz
zzz
zzz
![Page 25: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/25.jpg)
Roberts operator
One of the earliest edge detection algorithm by Lawrence Roberts
![Page 26: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/26.jpg)
Sobel operatorOne of the earliest edge detection algorithm by Irwine Sobel
![Page 27: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/27.jpg)
Image gradient• Gradient equation:
• Represents direction of most rapid change in intensity
• Gradient direction:
• The edge strength is given by the gradient magnitude
![Page 28: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/28.jpg)
2D Gaussian edge operators
Laplacian of GaussianGaussian Derivative of Gaussian (DoG)
Mexican Hat (Sombrero)• is the Laplacian operator:
Marr-Hildreth algorithm
![Page 29: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/29.jpg)
• Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”.
1 pixel 3 pixels 7 pixels
Tradeoff between smoothing and localization
![Page 30: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/30.jpg)
Designing an edge detector• Criteria for a good edge detector:
– Good detection: the optimal detector should find all real edges, ignoring noise or other artifacts
– Good localization•the edges detected must be as close as
possible to the true edges•the detector must return one point only for
each true edge point• Cues of edge detection
– Differences in color, intensity, or texture across the boundary
– Continuity and closure– High-level knowledge
![Page 31: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/31.jpg)
Canny edge detector• Probably the most widely used edge
detector in computer vision• Theoretical model: step-edges
corrupted by additive Gaussian noise• Canny has shown that the first
derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
http://www.mathworks.com/discovery/edge-detection.html
![Page 32: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/32.jpg)
Example
original image (Lena)
![Page 33: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/33.jpg)
Derivative of Gaussian filter
x-direction y-direction
![Page 34: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/34.jpg)
Compute gradients
X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude
![Page 35: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/35.jpg)
Get orientation at each pixel
• Threshold at minimum level• Get orientation theta = atan2(gy, gx)
![Page 36: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/36.jpg)
Non-maximum suppression for each orientation
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
![Page 37: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/37.jpg)
Sidebar: Interpolation options• imx2 = imresize(im, 2,
interpolation_type)
• ‘nearest’ – Copy value from nearest known– Very fast but creates blocky edges
• ‘bilinear’– Weighted average from four nearest known
pixels– Fast and reasonable results
• ‘bicubic’ (default)– Non-linear smoothing over larger area (4x4)– Slower, visually appealing, may create
negative pixel values
![Page 38: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/38.jpg)
Before non-max suppression
![Page 39: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/39.jpg)
After non-max suppression
![Page 40: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/40.jpg)
Hysteresis thresholding• Threshold at low/high levels to get weak/strong edge pixels• Do connected components, starting from strong edge pixels
![Page 41: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/41.jpg)
Hysteresis thresholding
• Check that maximum value of gradient value is sufficiently large– drop-outs? use hysteresis
• use a high threshold to start edge curves and a low threshold to continue them.
Source: S. Seitz
![Page 42: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/42.jpg)
Final Canny edges
![Page 43: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/43.jpg)
Canny edge detector
1. Filter image with x, y derivatives of Gaussian 2. Find magnitude and orientation of gradient3. Non-maximum suppression:
– Thin multi-pixel wide “ridges” down to single pixel width
4. Thresholding and linking (hysteresis):– Define two thresholds: low and high– Use the high threshold to start edge curves and
the low threshold to continue them
• MATLAB: edge(image, ‘canny’)
![Page 44: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/44.jpg)
Effect of (Gaussian kernel size)
Canny with Canny with original
The choice of depends on desired behavior• large detects large scale edges• small detects fine features
![Page 45: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/45.jpg)
Where do humans see boundaries?
• Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
![Page 46: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/46.jpg)
pB boundary detector
Martin, Fowlkes, Malik 2004: Learning to Detect Natural Boundaries…http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/papers/mfm-pami-boundary.pdf
![Page 47: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/47.jpg)
Brightness
Color
Texture
Combined
Human
![Page 48: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/48.jpg)
Human (0.95)
Pb (0.88)
![Page 49: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/49.jpg)
State of edge detection
• Local edge detection works well– But many false positives from
illumination and texture edges
• Some methods to take into account longer contours, but could probably do better
• Poor use of object and high-level information
![Page 50: CSE 185 Introduction to Computer Vision Edges. Scale space Reading: Chapter 3 of S.](https://reader036.fdocuments.us/reader036/viewer/2022062803/56649f535503460f94c788cc/html5/thumbnails/50.jpg)
Sketching
• Learn from artist’s strokes so that edges are more likely in certain parts of the face.
Berger et al. SIGGRAPH 2013