Lecture 6: Segmentation by Point Processing · 9/27/2005 · Lecture 6: Segmentation by Point...

Lecture 6: Segmentation by Point Processing

Harvey RhodyChester F. Carlson Center for Imaging Science

Rochester Institute of [email protected]

September 27, 2005

AbstractApplications of point processing to image segmentation by global

and regional segmentation are constructed and demonstrated. Anadaptive threshold algorithm is presented. Illumination compensationis shown to improve global segmentation. Finally, morphologicalwaterfall region segmentation is demonstrated.

DIP Lecture 6

Image Segmentation

Image segmentation can be used to separate pixels associated with objectsof interest from the image background.

This is an important step in many imaging applications of automatedanalysis and robotics. The opposing requirements of accuracy and speedare always present.

Segmentation on a simple pixel-by-pixel basis using threshold decisions isvery fast but restricted to applications where there is sufficient contrastbetween objects and background.

We will examine techniques for threshold segmentation using global andadaptive methods.

DIP Lecture 6 1

Global Segmentation

The object in this figure is light on a dark background. A histogram showsthat the object and background are well-separated.

A binary image that isproduced by thresholdingat the level indicated bythe dashed line is shownat the right.

DIP Lecture 6 2

Global Segmentation – Example 1

The effect ofthresholding thefingerprint image at thethreshold indicated isshown as the binaryimage to the right.

DIP Lecture 6 3


A greyscale image and its histogram is shown below. Selection of a thresholdfor segmentation is facilitated by examining a surface plot or a contour plot.

Image Histogram

Inverted surface plot Contour plot

DIP Lecture 6 4


The set of pixels S = {p : A[p] ≤ T} with T = 80 are selected in themiddle figure. The objects can be counted using LABEL REGION as shownon the right.

Original Segmented Labeled

DIP Lecture 6 5


The upper figure is an electrophoresis image inwhich the dark areas are of interest.

Global thresholding with T = 70 produces thebottom image. Note that the objects in the lowerright corner are missed. Other thresholds havesimilar difficulty, either being too high for theupper left corner or too low for the lower rightcorner.

DIP Lecture 6 6


The image was divided into four regions as shownin the upper image.

A different threshold was used for each region, asindicated on the figure. The result is shown in thelower image.

There was an improvement in the results becausethe threshold was adapted to the brightness ofeach region.

DIP Lecture 6 7


It appears that the average brightness is a function of position. If we assumethat the image is of the form

A[x, y] = F [x, y] + G[x, y]

where F [x, y] is the image of interest and G[x, y] is a varying background,then perhaps we can remove the baseline variation.

Let G[x, y] be approximated by a surface of the form

G[x, y] = a0 + a1x + a2y + a3xy

The coefficients can be estimated so that G is a least-mean-squareapproximation to A. For this image we find

a0 a1 a2 a3

142 0.06 -0.16 1e-5

DIP Lecture 6 8


Original Image A Compensated Image F

Region Thresholding of A Global Threshold of F

Region thresholding and global thresholding after compensation achievesimilar results.

DIP Lecture 6 9

Global Segmentation with Multiple Thresholds

An image that issegmented by using twothresholds as indicated bythe dashed lines is shownat the right. The brightersegment corresponds tothe region between thethresholds.

DIP Lecture 6 10

Automated Threshold Setting

An algorithm can be made more flexible and more easily used if the thresholdcan be selected automatically. When global thresholding is appropriate, thiscan be done by a simple iterative algorithm.

1. Select an estimate for T . This could be the mean grey value of theimage.

2. Segment the image using T to produce two sets of pixels: G1 containingpixels with grey values > T and G2 containing pixels with grey values≤ T .

3. Compute the average values µ1 and µ2 of sets G1 and G2.

4. Compute a new threshold value

T =µ1 + µ2

2

5. Repeat steps 2 through 4 until changes in T are small.

DIP Lecture 6 11

Difficulty with Global Segmentation

A global threshold may fail if the conditions change across an image.

An image with nonuniformillumination, which increaseslinearly from left to right.

The image histogramshows that there is nosuitable threshold forglobal segmentation.

DIP Lecture 6 12

Difficulty with Global SegmentationShown below is the effect of segmentation with two similar thresholdsettings. Note that neither is satisfactory.

DIP Lecture 6 13

Difficulty with Global SegmentationWe can attempt to clean up the segmentation with other techniques. Shownbelow is the effect of a morphological opening operation.

T = 70 Image T = 80 Image

DIP Lecture 6 14

Effect of Varying Illumination

An image can be considered to be the result of illumination and reflection.

f(x, y) = r(x, y)i(x, y)

The illumination is usually a slowly varying function compared to the scenereflectance.

If we know the illumination function then we can solve for the reflectancedirectly.

r(x, y) =f(x, y)i(x, y)

The reflectance function can then be processed directly.

If we do not know i(x, y) then we can try to determine it by analysis.

DIP Lecture 6 15

Solving for i(x, y)

Consider the function

z(x, y) = ln f(x, y)

= ln r(x, y) + ln i(x, y)

= rl(x, y) + il(x, y)

If r(x, y) and i(x, y) are independent random variables, which would be theusual case, then rl(x, y) and il(x, y) are also statistically independent.

The probability density function of independent random variables is theconvolution of the separate probability density functions. The probabilitydensity function of il(x, y) is likely to be approximately pi(ξ) = δ(ξ − ξ0).Convolution with the probability density function of the reflectance, pr(ξ),will leave the shape essentially unchanged.

Therefore, if r(x, y) can be segmented by global thresholding under constantillumination, we expect that rl(x, y) can be segmented by global thresholding

DIP Lecture 6 16

Solving for i(x, y)We will approximate il(x, y) with a slowly varying function.

il(x, y) = a0 + a1x + a2y + a3xy

We will choose the coefficients to minimize the mean-squared differencebetween z(x, y) and il(x, y).

Let

e(x, y) = z(x, y)− il(x, y) = z(x, y)− (a0 + a1x + a2y + a3xy)

and let the mean-squared error be denoted by

〈e2〉 =1

MN

N−1∑x=0

M−1∑y=0

e2(x, y)

=1

MN

N−1∑x=0

M−1∑y=0

(z(x, y)− (a0 + a1x + a2y + a3xy))2

DIP Lecture 6 17

Solving for i(x, y)

Differentiate with respect to the coefficients ak, k = 0, 1, 2, 3 and set eachequation equal to zero. We will use the bracket notation for the doublesums to simplify the appearance of the equations.

∂〈e2〉a0

= 2〈(z(x, y)− (a0 + a1x + a2y + a3xy))〉 = 0

∂〈e2〉a1

= 2〈(z(x, y)− (a0 + a1x + a2y + a3xy))x〉 = 0

∂〈e2〉a2

= 2〈(z(x, y)− (a0 + a1x + a2y + a3xy)) y〉 = 0

∂〈e2〉a3

= 2〈(z(x, y)− (a0 + a1x + a2y + a3xy))xy〉 = 0

DIP Lecture 6 18

Solving for i(x, y)The equations can be simplified to

a0 + a1〈x〉+ a2〈y〉+ a3〈xy〉 = 〈z(x, y)〉

a0〈x〉+ a1〈x2〉+ a2〈xy〉+ a3〈x2y〉 = 〈xz(x, y)〉

a0〈y〉+ a1〈xy〉+ a2〈y2〉+ a3〈xy2〉 = 〈yz(x, y)〉

a0〈xy〉+ a1〈x2y〉+ a2〈xy2〉+ a3〈x2y2〉 = 〈xyz(x, y)〉

All of the bracketed terms can be computed from the image information.We can then solve for the coefficients in this set of linear equations. Thematrix form is

1 〈x〉〈y〉〈xy〉〈x〉〈x2〉〈xy〉〈x2y〉〈y〉〈xy〉〈y2〉〈xy2〉〈xy〉〈x2y〉〈xy2〉〈x2y2〉

a0

a1

a2

a3

=

〈z〉〈xz〉〈yz〉〈xyz〉

Ca = v ⇒ a = C−1v

DIP Lecture 6 19

Solving for i(x, y)

The terms in the matrix are based only on the image coordinates whilethose on the right depend upon both the image and the coordinates.

;Open the imageF=read_image(’D:\Harvey\Classes\SIMG782\2004\lectures\lecture_06\doc\fig7.png’)sf=size(F,/dim)

;Construct coordinate arrays for the imagex=findgen(sf[0])#replicate(1,sf[1])y=findgen(sf[1])##replicate(1,sf[0])

;Compute the log imageFL=BYTSCL(ALOG(F))

;Construct the moments used to solve for the a coefficientsmx=mean(x) & my=mean(y) & mxy=mean(x*y)mx2=mean(x^2) & my2=mean(y^2) & mx2y=mean(x^2*y)mxy2=mean(x*y^2) & mx2y2=mean(x^2*y^2)C=[[1,mx,my,mxy],[mx,mx2,mxy,mx2y],[my,mxy,my2,mxy2],[mxy,mx2y,mxy2,mx2y2]]CI=invert(C)

DIP Lecture 6 20

Solving for i(x, y)

;Construct an illumination predictor for FL mL=mean(FL) &mLx=mean(FL*x) & mLy=mean(FL*y) & mLxy=mean(FL*x*y)v=transpose([mL,mLx,mLy,mLxy])aL=CI##vGL=aL[0]+aL[1]*x+aL[2]*y+aL[3]*x*y

For this image, which is of size 500× 400 we find

a =

1.0 249.5 199.5 49775.3

249.5 83083.5 49775.3 16575158.0199.5 49775.3 53133.5 13256808.0

49775.3 16575158.0 13256808.0 4414517248.0

−1

147.741253.929511.4

8245948.0

=

95.2278900.207723−0.0011200.000018

DIP Lecture 6 21

Solving for i(x, y)

Having computed the coefficients, the rest is now easy.

;Illumination approximationGL=aL[0]+aL[1]*x+aL[2]*y+aL[3]*x*y

;Compute the log reflectanceRL=bytscl(FL-GL)

;Compute the log reflectance histogramhr=histogram(RL)

;Threshold the log reflectance imageTL=175RLT=RL GE TL

The results are illustrated on the following pages.

DIP Lecture 6 22

Segmentation Example

The figure below has an illumination gradient from left to right acrossthe image. As shown by its histogram, it is difficult to segment. Thelog(reflectance) has a separable histogram.

Original Histogram of Original Segmentation

Log(Illumination) Log(Reflectance) Histogram

DIP Lecture 6 23

Segmentation Example

The figure below has an illumination gradient from left to right acrossthe image. As shown by its histogram, it is difficult to segment. Thelog(reflectance) has a separable histogram.

Original Histogram Segmentation

Log(Reflectance) Histogram Segmentation

DIP Lecture 6 24

Example 3 – Continued

Removal of illumination trend by compensation of the logimage has slightly better results for the electrophoresis image.

Original Image Illumination Subtraction Global Threshold

Log Image Illumination Division Global Threshold

DIP Lecture 6 25

Morphological Waterfall Segmentation

Watershed segmentation is a metaphor for a computational method thatiteratively fills the local minima of a greyscale image to find segmentboundaries.

As regions are filled from the bottom, they reach a level where they wouldmerge. At this point “dams” are constructed to maintain the separation.

The “dams” become the segment boundaries.

The iterative erosion process is illustrated in the figures below (Gonzalez &Woods, Section 10.5).

DIP Lecture 6 26

Digital Image Processing, 2nd ed.Digital Image Processing, 2nd ed.www.imageprocessingbook.com

© 2002 R. C. Gonzalez & R. E. Woods

Chapter 10Image Segmentation

Chapter 10Image Segmentation

Watershed Segmentation – Example 1

The small greyscale image of smooth black blobs is a natural test candidatefor watershed segmentation. This image was segmented using the IDLwatershed.pro function. Code shown on next page.

Original Image

Aftermorphologicalgreyscaleerosion withdisk ofradius 9

Watershedregions

Boundariesoverlaid onoriginalimage.

DIP Lecture 6 27

Watershed Demo Code;Watershed Demo;Radius of disc for greyscale erosionr = 9

;Create a disc of radius rdisc = SHIFT(DIST(2*r+1), r, r) LE r

B=READ_IMAGE(’doc/fig11a.png’)sa=size(B,/dim)window,/free,xs=2*sa[0],ys=2*sa[1]

;window,/free,xs=sb[0],ys=sb[1],xpos=0,ypos=0TVSCL, b,0

;Remove holes of radii less than rc = MORPH_CLOSE(b, disc, /GRAY)TVSCL, c,1

;Create watershed imaged = WATERSHED(c,connectivity=8)

;Display it, showing the watershed regionstek_colorTV, d,2loadct,0,/silent

;Merge original image with boundaries of watershed regionse = a > (MAX(a) * (d EQ 0b))TVSCL, e,3

DIP Lecture 6 28


Revisit segmentation of electrophoresis image

DIP Lecture 6 29


Segmentation of pollen image from IDL demonstration package.

DIP Lecture 6 30

Lecture 6: Segmentation by Point Processing · 9/27/2005 · Lecture 6: Segmentation by Point...

Documents

Transcript of Lecture 6: Segmentation by Point Processing · 9/27/2005 · Lecture 6: Segmentation by Point...