1

2.3 Feature extraction

Many computer vision systems rely on first the detection of some features in the images.

Image features may be global (e.g. average grey level) or local (e.g. a straight line). They

may or may not be associated to scene elements. Different features demand different

detection methods. In general, feature extraction results in feature descriptors specifying

the locations and properties of the features found in the image.

2.3.1 Edge detection

Edges are pixels at or around which the image values undergo a sharp variation, e.g.

borders of object. Edge detection is often the first operation in finding interesting scene

elements such as lines, contours. Notice that noise can also cause intensity variations. So

a good edge detector should find genuine edges generated by scene elements, not by noise.

In the detection process, noise will be suppressed without destroying true edges, then

edges are enhanced and located.

Types of edge

Step edge occurs where the image intensity abruptly changes from one value on one side

of the discontinuity to a different value on the opposite side. If the intensity change occurs

over some pixels, it is a ramp edge.

Line edge occurs where the image intensity abruptly changes value but then returns to the

starting value within some short distance. If the two changes occur over a finite distance,

it is a roof edge.

Edge descriptor

Edge normal: the direction of the maximum intensity variation at the edge point which is

perpendicular to the edge

Edge direction: the direction tangent to the edge which is perpendicular to the edge

normal

Edge position: the image position at which the edge is located, along the edge normal

Edge strength: a measure of the intensity variation across the edge

2

First derivative operator – Sobel edge detector

Edge detection is the essentially the operation of detecting significant local intensity

changes in an image. Take step edge as an example, it is associated with a local peak in

the first derivative.

The first derivative can be measured by the gradient.

∂

∂∂

∂

=

=

y

Ix

I

G

G))y,x(I(G

y

x

The magnitude of the gradient is 2

y

2

x GG))y,x(I(G += . The calculation can also be

approximated by yx GG))y,x(I(G += or )G,Gmax())y,x(I(G yx= . The direction

of the gradient with respect to the x axis is

=∠

−

x

y1

G

Gtan))y,x(I(G .

For digital image, the gradient is measured by discrete approximation.

]j,1i[I]j,i[IG

]j,i[I]1j,i[IG

y

x

+−≅

−+≅

The equations can be implemented with convolution masks

edge normal edge direction

I

I’

-1

-1

1 1 Gx = Gy =

-1 1 -1

1

3

which estimate the gradient at the coordinates

++

2

1j,

2

1i . To detect edge, you can

perform noise filtering before the gradient computation. Genuine edges can be located

with sub-pixel resolution simply by thresholding the gradient magnitudes.

To avoid having the gradient calculated about an interpolated point between pixels, you

can use a 3 x 3 neighborhood. The Sobel edge detector calculates the gradient magnitude

by 2

y

2

x SS))y,x(I(S += where Sx and Sy can be implemented using convolution masks

Second derivative operator – Laplacian of Gaussian

Another approach is to locate the zero crossing in the second derivative.

The zero crossings can be located by the Laplacian operator.

2

2

2

22

y

I

x

I)y,x(I

∂

∂+

∂

∂=∇

Equations of discrete approximation are

]j,1i[I]j,i[I2]j,1i[Iy

I

]1j,i[I]j,i[I2]1j,i[Ix

I

2

2

2

2

−+−+=∂

∂

−+−+=∂

∂

The Laplacian operator can be implemented using the convolution mask

-1

0

0 1 Sx = Sy =

-2 0 0

2

-1 0

1

2

1 -1 -2

0

1

-1

I

I’

I”

4

In many cases, the actual edge location must be determined by interpolation. Edge

operator involving two derivatives is affected by noise more than an operator involving a

single derivative.

A better approach is to combine Gaussian filtering with the second derivative – Laplacian

of Gaussian (LoG). Steps in edge detection:

• filter out the image noise using Gaussian filter

• enhance the edge pixels using 2D Laplacian operator

• edge is detected when there is a zero crossing in the second derivative with a

corresponding large peak in the first derivative

• estimate the edge location with sub-pixel resolution using linear interpolation

Some methods apply filtering masks of multiple sizes and locate the edge pixels by

analyzing the behavior of edges at different scales of filtering.

Canny edge detector

There is a trade-off between noise suppression and edge localization. An edge detector

can reduce noise by smoothing the image, but this will result in spreading of edges and

add uncertainty to the location of edge. An edge detector can have greater sensitivity to

the presence of edges, but this will also increase the sensitivity of the detector to noise.

The type of linear operator that provides the best compromise between noise immunity

and edge localization, while retaining the advantages of Gaussian filtering, is the first

derivative of a Gaussian. This operator corresponds to smoothing an image with a

Gaussian function and then computing the gradient. The operator is not rotationally

symmetric – it is symmetric along the edge direction and anti-symmetric along the edge

normal.

The Canny edge detector is the first derivative of a Gaussian and closely approximates the

operator that optimizes the product of SNR and edge localization. Steps in edge detection:

• edge enhancement

• non-maximum suppression

• hysteresis thresholding

edge enhancement:

Apply Gaussian smoothing to the image. Compute the gradient of the smoothed image

and estimate the magnitude and orientation of the gradient.

]j,i[I];j,i[G]j,i[S ∗σ=

0 1 ∇2 ≈

1 -4

0 1

0

1

0

5

where G is a Gaussian with zero mean and standard deviation σ. The gradient

components are computed by

2

])1j,1i[S]1j,i[S]j,1i[S]j,i[S(]j,i[Q

2

])j,1i[S]1j,1i[S]j,i[S]1j,i[S(]j,i[P

++−+++−=

+−+++−+=

Magnitude of the gradient is 22 ]j,i[Q]j,i[P]j,i[M +=

Orientation of the gradient is

=θ

−

]j,i[P

]j,i[Qtan]j,i[ 1

non-maximum suppression:

M[i, j] may contain wide ridges around the local maximum. This step is to thin such

ridges to produce 1-pixel wide edges. Values of M[i, j] along the edge normal that are not

peak will be suppressed. Let the possible edge normal orientations be quantized into 4,

e.g. 0°, 45°, 90°, and 135° with respect to the horizontal axis. For each pixel (i, j), find

the orientation which best approximates θ[i, j]. If M[i, j] is smaller than at least one of its

two neighbors along the quantized orientation, change M[i, j] to zero.

hysteresis thresholding:

M[i, j], after the non-maximum suppression step, may still contain local maxima created

by noise. To get rid of false edges by thresholding, some false edges may still remain if

the threshold value is set too low, or some true edges may be deleted if the threshold

value is set too high. An effective scheme is to use 2 thresholds τl and τh, e.g. τh = 2τl.

Scan the non-zero points of M[i, j] in a fixed order. If M[i, j] is larger than τh, locate it as

an edge pixel. Else if any 8-neighbors of (i, j) have gradient magnitude larger than τl,

locate them as edge pixels. Continue until another edge pixel is located by τh. Therefore,

the Canny edge detector performs edge linking as a by-product.

Final words

Edge detection has been one of the most popular research areas since the early days of

computer vision. There is a large literature on edge detection. You should be aware that

specific edge detection method can be very useful for particular computer vision

applications. However, a universal edge detector remains to be seen.

2.3.2 Corner detection

Corners are quite stable across sequences of images. They are interesting features that can

be employed for tracking objects across sequences of images. Consider an image point p,

a neighborhood Q of p, and a matrix C defined as

6

=∑∑

∑∑

Q

2

y

Q

yx

Q

yx

Q

2

x

GGG

GGG

C

We can think of C as a diagonal matrix

λ

λ=

2

1

0

0C

The two eigenvalues λ1 and λ2 are non-negative. A corner is located in Q where λ1 ≥ λ2 >

0 and λ2 is large enough. Steps in corner detection:

• compute the image gradient

• for each image point p, form matrix C over a neighborhood Q of p, compute the

smaller eigenvalue λ2 of C, if λ2 > threshold τ save the coordinates of p into a list L

• sort L in decreasing order of λ2

• scan L from top to bottom, for each corner point p, delete neighboring corner points in

L

References

E. Trucco & A. Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall,

1998, Chapter 4.

R. Jain, R. Kasturi & B. G. Schunck, Machine Vision, McGraw-Hill, 1995, Chapter 5.

Summary

♦ first derivative edge detector

♦ second derivative edge detector

♦ Canny edge detector

♦ corner detection