2.3.3 Line and curve detection

Lines and curves are important features in computer

vision because they define the contours of objects in

the image.

Take the result of edge detection as input, see if we

can link up the edge pixels and detect the existence

of lines and curves.

Line detection - Hough transform (HT)

A line y = mx + n can be represented by a parameter

pair (m, n) which is one point in the parameter space.

You can rewrite the line equation as n = -mx + y. For

a point p = [x, y]T, m and n can vary and form a line in

the parameter space, representing all possible lines

passing through p.

A line containing N edge points p1, …, pN is identified

in the parameter space by the intersection of N lines

associated with p1, …, pN.

imageparameter space

m

np1

p2

m’

n’

As m and n vary, the corresponding entry in the

parameter space will be increased by 1. Ideally, all

entries covered by the lines in the parameter space will

have the count of 1 except that the entry (m’, n’) has the

count of N.

There is a problem using the parameter space

formed by m, n!

Therefore, HT is a voting algorithm. To detect line is

to search for peak in the parameter space.

In implementation, we usually adopt the polar

representation ρ = j cosθ - i sinθ.

discrete image I[i, j]

j

ρ

[0, 0]

i

θ

. . .100

0°

10°

.

.

.

360°

22 NM +

parameter space

θ

ρ

Line detection algorithm:

• input edge detection result (M x N binary image)

• quantize ρ and θ to create the parameter space,

• initialize all entries of the parameter space to zero

• for each edge point [i, j], increment by 1 each entry

that satisfies the polar representation, e.g. for a

specific quantized value of θ, find the closest

quantized value of ρ• find the local maxima (ρ, θ), each has the count >

user-defined threshold τ

]2,0[],NM, 22 πθρ ∈+∈[0

Advantages:

• It is possible to detect the line even some points are

missing.

• False edge points are unlikely to contribute to the

same set of parameters and generate a false peak in

the parameter space.

Hough transform can be implemented using the

MATLAB function hough.

Disadvantages:

• You can generalize HT to detect curve y = f(x, a),

where a = [a1, …, aP]. The parameter space is P-

dimensional. However, search time increases rapidly

with the number of parameters.

• Non-target shapes can produce spurious peaks in

parameter space, e.g. low-curvature curves can be

detected as lines.

Another approach to detect line is to fit edge pixels to a

line model - model fitting approach

For a generic line ax + by + c = 0, or , find the

parameter vector [a, b, c]T which results in a line going

as near as possible to each edge point. A solution is

reached when the distance between the line and all the

edge points is minimum – least squares problem.

Ellipse fitting

Many objects contain circular shapes, which usually

appear as ellipses in image. Therefore, ellipse

detectors are useful tools for computer vision.

According to the model fitting approach, find the best

ellipse that can fit the edge points.

Assume that the set of edge points pi = [xi, yi]T,

i = 1, ...,N belong to a single arc of ellipse

xTa = ax2+bxy+cy2+dx+ey+f = 0

where x = [x2, xy, y2, x, y, 1]T and a = [a, b, c, d, e, f]T.

Find the parameter vector a, associated to the ellipse

which fits p1…pN best in the least squares sense

∑=

N2

T

a

ax

1i

imin

To avoid the trivial solution a = 0 and force the

solution to be an ellipse

aaaa

−

−

==−=−

000000

000000

000000

000002

000010

000200

C1ac4bTT2

where C is called the constraint matrix. C is rank-

deficient.

The problem becomes aaaaaa

SminXXminTTT =

where X is called the design matrix

=

1N

yN

x2N

yN

yN

x2N

x

12

y2

x22

y2

y2

x22

x

11

y1

x21

y1

y1

x21

x

XL

and S is called the scatter matrix S = XTX.

Using Lagrange multipliers, the problem

(generalized eigenvalue problem) can be solved by

Sa = λCa.

Ellipse detection algorithm:

• normalize the edge points

• build X

• compute S

• build C

• compute eigenvalues, a is the eigenvector

corresponding to the only negative λ

Use the MATLAB function eig to compute eigenvalues

and eigenvectors.

Original image Edge detection result

Ellipse detection result

2.3.4 Color

Humans use color information to distinguish objects.

Color model

Color can be represented in 3 bytes – one byte for each

of Red, Green and Blue (RGB). The encoding of an

arbitrary color in the visible spectrum can be made by

combining the encoding of three primary colors RGB.

RGB color model is additive and corresponds well to

monitors.

(255, 0, 0) red

(0, 255, 0) green

(255, 255, 0) yellow

(0, 0, 0) black

(255, 255, 255) whitewhitewhite

Some computer vision algorithms can perform better

using normalized colors.

normalized red

normalized green

normalized blue

BGR

Rr

++=

BGR

Gg

++=

BGR

Bb

++=

Another color model is called Hue-Saturation-Intensity

(HSI). It has the advantage that color information

(chromaticity) represented by H and S, is separated from

intensity. Hue describes the tone of the color. Saturation

provides a measure of its purity.

3

BGRI

++=

B)B)(G(RG)(R2

BG2RcosH

2 −−+−

−−=

B)G,min(R,BGR

31S

++−=

Normalize RGB components to 1. Subtract H from

360° when B/I > G/I. H is not defined when S is zero.

S is not defined when I is zero.

Conversion between RGB and HSI can be implemented

using the MATLAB functions rgb2hsv and hsv2rgb.

An RGB color image in MATLAB corresponds to a

3D matrix of dimensions M x N x 3.

image = imread(filename);

[height, width, color] = size(image);

Histogram

A histogram counts the number of pixels of each kind

(e.g. grey level, color).

Create a histogram by reading the image pixels one by

one and incrementing the appropriate bin of the

histogram. For color image, you can create 3

histograms, one for each of the 3 color components,

e.g. RGB. Each histogram may have 256 bins.

Histogram can be used to determine how similar a test

image T to a reference image R.

Assume both histograms hT and hR have K bins.

intersection ∑=

=K

1i

RT [i])h[i],min(h

match

∑

∑

=

==K

1i

R

K

1i

RT

[i]h

[i])h[i],min(h

The match value indicates how much color content of

the reference image is present in the test image. It is

relatively invariant to translation, rotation and scale

changes of image.

Sometimes, you may want to compute a dissimilarity

measure

distance ∑=

−=K

1i

RT [i]h[i]h

Noise filtering techniques and edge detectors can be

extended to color images under the componentwise

paradigm.

2.3.5 Texture

Same histograms – 50% black, 50% white

Texture feature can be a powerful descriptor of an

image. It gives us information about the spatial

arrangement of the colors or intensities in an image.

Texture is commonly found in natural scenes and man-

made objects.

However, there is not a universally agreed upon

definition of texture.

There are two main approaches to describe texture

properties:

Structural approach – texture is a set of primitive texture

elements (texels) in some regular or repeated

relationship.

Statistical approach – texture is a quantitative measure

of the arrangement of colors or intensities in a region.

The first approach can work well for man-made, regular

patterns. The second approach is more general and easier

to compute and is used more often in practice.

Edgeness

The number of edge pixels in a given region indicates

the busyness of that region.

Edgeness per unit area N

p=

where |p| is the number of edge pixels in a region of N

pixels.

To include both gradient magnitude and gradient

orientation

Edge-based histogram = (hmag, horient)

where hmag is the normalized histogram of gradient

magnitude of that region, and horient is the normalized

histogram of gradient orientation of that region.

Co-occurrence matrices

A co-occurrence matrix is a 2D array C in which both

the rows and the columns represent a set of image

values (intensities, colors). The value Cd[i, j] indicates

how many times value i co-occurs with value j in some

designated spatial relationship. The spatial relationship

is represented by a vector d = (dr, dc), indicating the

displacement of the pixel having value of j from the

pixel having value of i by dr rows and dc columns.

i

2200

2200

0011

0011

2002

0221

2040

210

j

image I C(0,1)

It is common to normalize the co-occurrence matrix

and so each entry can be considered as a probability.

∑∑=

i j

d

d

dj][i,C

j][i,Cj][i,N

Numeric features can be computed from the co-

occurrence matrix that can be used to represent the

texture more compactly.

Energy ∑∑=i j

2

d j][i,N

Entropy ∑∑−=i j

d2d j][i,Nj]log[i,N

Contrast ∑∑=i j

d

2 j][i,Nj)-(i

Homogeneity ∑∑+

=i j

d

j-i1

j][i,N

Correlation ji

i j

dji

σσ

j][i,)Nµ)(jµ(i∑∑ −−

=

where µi, µj are the means and σi, σj are the standard

deviations of the row and column sums

∑=j

dd j][i,N[i]N

∑=i

dd j][i,N[j]N

Summary

♦ Hough transform

♦ ellipse fitting

♦ color models

♦ histogram

♦ texture features