CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time:...
Transcript of CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time:...
Filtering and Edge Detection
Scott WehrweinCSCI 497/597P: Computer Vision
Reading
• Szeliski, Chapter 4.2
Announcements
Goals
• Understand practical considerations involved in filtering:• Handling the boundaries• Preserving total brightness
• Learn the basics numpy for array and image processing.• Understand the basics of edge detection:• The sobel operator as an approximation of the image gradient in the presence
of noise.
Last time: Convolution and cross-correlation
• Cross correlation
• Convolution
S[f ](m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m+ i, n+ j)
S[f ](m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m� i, n� j)
S[f ] = w ⌦ f
S[f ] = w ⇤ f
Last time: Convolution and cross-correlation
• Properties• Shift-invariant: a sensible thing to require• Linearity: convenient
• Can be used for smoothing, sharpening• Also main component of CNNs
Boundary conditions
• What if m-i <0?• What if m-i > image size• Assume f is defined for [−∞,∞] in both directions,
just 0 everywhere else• Same for w
(w ⇤ f)(m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m� i, n� j)
(w ⇤ f)(m,n) =1X
i=�1
1X
j=�1w(i, j)f(m� i, n� j)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Boundary conditions in practice
• “Full convolution”: compute if any part of kernel intersects with image• requires padding• Output size = m+k-1
• “Same convolution”: compute if center of kernel is in image• requires padding• output size = m
• “Valid convolution”: compute only if all of kernel is in image• no padding• output size = m-k+1
“valid”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
“same”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
“full”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
Padding Modes (OpenCV)
• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii
(with specified constant ‘i’; default = 0)
(image)(L pad) (R pad)
Padding Modes (OpenCV)
• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii
(with specified constant ‘i’; default = 0)
(image)(L pad) (R pad)
The sum of the kernel• The sum of all kernel weights is important.
*1 1 1
1 1 11 1 1
=
The sum of the kernel• The sum of all kernel weights is important.
*1 1 1
1 1 11 1 1
=1
9
Usually want it to sum to either 1 or 0.
Edge detection
• Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
Images as functions…
• Edges look like steep cliffs
Characterizing edges• An edge is a place of rapid change in the image
intensity function
imageintensity function
(along horizontal scanline) first derivative
edges correspond toextrema of derivativeSource: L. Lazebnik
• How can we differentiate a digital image F[x,y]?– Option 1: reconstruct a continuous image, f, then
compute the derivative– Option 2: take discrete derivative (finite difference)
1 -1
How would you implement this as a linear filter?
Image derivatives
-1
1: :
Source: S. Seitz