Segmentation of Mango Region from Mango Tree Image

R. Prasath and T. Kathirvalavakumar (Eds.): MIKE 2013, LNAI 8284, pp. 201211, 2013. Springer International Publishing Switzerland 2013

Segmentation of Mango Region from Mango Tree Image

D.S. Guru and H.G. Shivamurthy

Department of Studies in Computer Science, University of Mysore, Manasagangothri, Mysore-570006, Karnataka, India

[email protected], [email protected]

Abstract. In this paper we propose a novel framework for segmentation of mango regions from its tree image. The proposed framework consists of mango localization followed by mapping of boundary information to the located region for segmentation. Initially thresholding is applied to each individual color band R,G and B by adaptive thresholding and later they are combined back. Application of smoothing and binarization to the combined image gives the location of mangoes along with noise. The texture features are extracted from each location then matched with template stored in the database to eliminate the noisy regions. Finally, locations of the mangoes are obtained and edge information is superimposed on to those locations for segmentation. An experiment is performed on our own dataset and efficiency is evaluated by computing the precision, recall and F-measure with respect to the human segmented images considering as a ground truth.

Keywords: Precision agriculture, Segmentation, Mango localization, thresholding, texture features.

1 Introduction

Modernizing agricultural practices with a help of an emerging technology leads to the environmental and economic sustainability with an optimized input in production of agricultural products. The process of identification and interpretation of in-field spatial variability through information and technology for effective management of agricultural practices such as soil mapping, disease mapping, weed mapping, selective harvesting and quality analysis, is known as precision agriculture (PA). Harvesting is the final stage of any agricultural practice. Manual process of selective harvesting will consume more time, manpower and not accurate. Selective harvesting is very popular today because we harvest only the matured crop, and while harvesting we also grade the crop based on the maturity level noticed. The clare valley and Margaret river regions of Australia is famous for wine grape cultivation. Bramley et al. [1] state that economic benefits that may accrue to grape growers and winemakers in Australia through the adoption of selective harvesting. Mango is one of the commonly cultivated commercial crops in many countries including India. Till today, the process of mango harvesting and grading is done manually and it consumes much of human effort and time. Nowadays, researchers are trying to automate the agricultural

202 D.S. Guru and H.G. Shivamurthy

activities through the help of computer vision techniques, as it is very helpful to the formers, dealers and consumers. The mango fruits should be harvested at green mature stage. Selective harvesting of green matured mangos is helpful for maturity grading by distinguishing ripe and unripe mangos so that ripe mangos are used at the earliest and unripe mangos can be stored for some time. Further, quality wise grading of mangos can also be done during selective harvesting.

So, given an image of a mango tree, the problem of selective harvesting of mangos can be defined as a problem of localization and segmentation of mango regions from it. On the other hand it contributes the problem of spatial correlation since both leaf and fruits are green in color. Mango region segmentation is a process of acquiring knowledge about mango regions in an image. It is a very first task in a mango image analysis process such as disease mapping, variable spraying and selective harvesting. Quality of the subsequent tasks will depend on the success of the mango region segmentation process. Color and texture are the two essential features in mango tree image analysis. Texture is an efficient measure to estimate the structure, orientation, roughness, smoothness, and regularity differences of mango regions in the mango tree image. Two usual problems in mango region segmentation are over-segmentation: where an image is segmented into a more number of regions than the actual mango regions in the mango tree image and under-segmentation: where an image is segmented into a less number of regions than the actual mango regions in the mango tree image. The over-segmentation and the under-segmentation of mango regions happen usually in case of images with spatially varying illumination. Also, the leaves are occluded on mangoes in the mango tree image. Hence balancing the over and under segmentation in unconstraint environment of the mango tree image is a challenging issue in the area of color image segmentation.

The automation of agricultural practices for significant increase in food production is emerging out as a new challenge for computer vision community. Ducournau et al. [2] proposed a machine vision approach to count the number of emergent radical tips on seed-lots, under the constrained environment and segmentation is accomplished with the help of thresholding and morphological operators. Green vegetation region segmentation [3] using the IHS (Intensity, Hue, Saturation) and RGB (Red, Green, Blue) color space for color feature extraction and then apply mean shift and BPNN (back propagation neural network) for segmentation. One of the central point of precision agriculture is the selective treatment of weeds [4] and it is achieved in three different stages at which each different agricultural elements is extracted. They are segmentation of vegetation against non-vegetation, crop row elimination and weed region extraction. The segmentation is done by thresholding on the basis of dominant G-component, dominated B-component, minimum and maximum intensity in the image. Segmentation of lesquerella flowers was proposed by Thorp et al. [5] based on thresholding the images in HIS color space and boundary conditions using six parameters, including the maximum and minimum hue, saturation and intensity in image. The popular general unsupervised segmentation of color-texture region in image [6] is presented and it is named as J-seg which consists of two independent steps: color quantization and spatial segmentation.

All the segmentation problems which are related to a precision agriculture are considered to be two class problems (the pixel belongs to vegetation/non-vegetation).

Segmentation of Mango Region from Mango Tree Image 203

In this work, we made an attempt to classify pixel belongs to a region of mango or non-mango in a mango tree image. In our framework we use texture features for template matching and color features for sensing the object of our interest. The proposed framework consists of two major phases they are:

(1) Mango localization, and (2) Mapping the edge information for segmentation. The rest of the paper is organized as follows. In section 2, we present the overview

of the proposed method. The experimentation and results are described in the section 3. Finally paper ends with conclusions in section 4.

2 Proposed Framework

In this section, we propose a new framework for mango region segmentation, which consists of two phases namely, mango localization and edge mapping. In first phase, we apply thresholding to each color band separately then combined them. The combined image (T-image) is said to have j-number of regions. These regions are considered as mango regions. We perform smoothing to eliminate the non-mango regions in the thresholded image. Then apply binarization to the resultant image to get B-image. The B-image has say k-number of regions assumed to be the mango regions. Finally we achieve mango localization through template matching which eliminates the non-mango regions in the B-image and results with an image (L-image) having m-number of mango regions where j k m. In second phase, the edge information from the original image is superimposed on to the regions of the L-image for segmentation. The block diagram of mango region segmentation is shown Fig .1.

2.1 Image Thresholding

The reflectance on the surface of mango regions is higher than that of non-mango regions, so higher intensity will be preserved by the regions belonging to mangos. This is an important clue for us to distinguish mango regions from non mango regions. Based on this clue we apply a simple thresholding to segment the mango regions from its tree image. We separately threshold each R, G & B components in the mango tree image and then combine the resultant images back (T-image). The middle value of the intensity range in a given image intensity distribution will be taken as threshold. The thresholding process can be formulated as shown in Eq. (1) and (2).

A(i, j),A(i, j) thresholdT(i, j)

0, A(i, j) threshold

=


Fig. 1. Block diagram of the proposed framework

Fig. 2. (a) A given mango tree image (RGB). (b) The R-component of the input image. (c) The G-component of the input image (d) The B-component of the input image.

The major difficulty in mango tree image segmentation is varying illumination which can be seen in the mango regions at upper left corner, upper middle and lower middle of the Fig.2a. Initially we assume the regions in the thresholded image as mango regions. Fig.3a-3d depict the thresholded images of each R, G, B component and combined image of the original color image respectively.


Fig. 3. (a) Thresholded image of R-component. (b) Thresholded image of G-component. (c) Thresholded image of B-component. (d) Thresholded image of given input image Fig.2a.

2.2 Smoothing and Binarization The small regions and sharp transitions in the thresholded image are non-mango regions, because the mango regions are relatively larger regions. The sharp transitions in intensity levels can be seen only when there is random noise in the image. So, we perform smoothing to remove small regions and sharp transitions (non-mango regions) in the thresholded image of each of the R,G and B components separately and combined them. We use a 3x3 mask for smoothing because it will not affect lager mango regions and also it results in less blurring effect. The mango region segmentation is a two class problem, we convert each smoothed R, G and B component into binary images where, white pixels represent mango regions and black will represent non-mango regions. Then combine the resultant images into a single binary image (B-image) as shown in the Fig.4. The binarization process will fixes up a contrast break-point between pixels belonging to mango regions and that of non-mango regions.

Fig. 4. Binarized image (B-image)


2.3 Template Matching

Here we use template matching technique to eliminate the non-mango regions in the B-image. For this purpose, we extract the texture properties of the original image from the regions located in the B-image and match with the templates stored in the database. We retain only those regions whose texture properties are similar to mangoes template. The template of mango is as shown in the Fig.5.

Fig. 5. Few templates of mangoes

Texture features are the very fundamental and invariant properties of images. Each image has its own texture properties which describe different image regions present in it. In the image classification and image segmentation literatures texture properties of images have been efficiently used. Statistical texture features proposed by Haralick et al. [7] are used for template matching to eliminate the non-mango region in the binarized image (B-image). Initially Gray Level Co-occurrence matrix (GLCM) is computed for the gray image using the pair-wise occurrences of image resolutions. And the various texture properties are calculated using the GLCM obtained. In our frame work three different texture features are used. Let us assume that P is the gray level co-occurrence matrix obtained from the image region rx, expressions for different texture features which we have used are as follows.

(1). Contrast:

g 1 Ng Ng

i j n

N21

n 0 i 1 j 1f n P(i, j)

== = =

=

(2). Correlation:

x yi j2

x y

(ij)p(i, j)f

=

(3). Entropy:

3 i jf p(i, j)log(P(i, j)).=


Notation:

p(i, j) is the (i, j) th entry in a normalized gray-tone spatial dependence matrix. x , y , x and y are the means and standard deviation of px and py.

Ng is the Number of distinct gray levels in the quantized image.

Below is the algorithm for texture based template matching. Algorithm: template matching.

Input: B-image (k-regions). Output: mango localized image Method:

1. For Ith region in the B-image, extract the corresponding region in the original given image.

2. Compute the texture features (Contrast, Correlation and Entropy) of the region extracted.

3. Compute the dissimilarity between texture features extracted from obtained region and templates stored in the database.

4. If dissimilarity is less than the threshold, then region belongs to mango otherwise it is non-mango region, neglect it.

5. Repeat the above procedure till (I=k). End of algorithm.

The output image of the above algorithm gives location of mangoes. To accurately show the boundaries of each mango located and also to clearly distinguish the mangos in the cases of occlusion, we extract the edges from located regions of the original image and superimpose on to the localized image for segmentation. We use canny edge detection operator for edge extraction. Finally we obtain segmented image of the original image Fig.2a is shown in the Fig.6.

Fig. 6. Segmented image


3 Experimentation and Results

3.1 Dataset

Since, we could not find any dataset with mango tree image in the literature, we have captured 44 natural mango tree images in a natural lighting condition using Kodak digital camera (8.2-megopixel) with resolution 480x480dpi from different directions and with the distance of less than five feet because, the image intensity at pixel depends on the optical properties of the surface material, the surface shape and spatial distribution of the incident illumination. The images we captured consists mango leaves and branches in addition to mangos. We captured images under different illumination conditions and occlusion of leaves on the mangoes. Sample images of our own dataset is show in the Fig.7

Fig. 7. Sample images of mango trees

3.2 Results

The quantitative results of mango region segmentation are computed on our own data set using precision, recall and F-measures as given below,

MRSPr ecision(P)MRS NMRS

=

+ (3)

MRSRecall(R)AMR

= (4)

2*P*RF measureP R

=

+ (5)

Here MRS is the mango region segmented correctly, NMRS is the non mango region segmented as a mango region and AMR is actual mango regions present in the image (ground truth) and human segmented image considered as a ground truth. The more segmented results are shown in Fig.8.

(a) (b) (c) (d)


In the image (3) we exactly segment the mango region, but in the case of image (1), image (2), and image (4) there is a over segmentation due to the occlusions of leaf on the mango which is still a challenging issue in mango region segmentation. The graphical representation of precision, recall and f-measures obtained for our data set is as shown below Fig.9, Fig.10 and Fig.11 respectively.

Input Image Output image

Image 1

Image 2

Image 3

Image 4

Fig. 8. Segmented mango tree images with original input images


Fig. 9. Precision of the proposed method

Fig. 10. Recall of the proposed method

Fig. 11. F-measure of the proposed method


4 Conclusion and Future Work

In this paper, we have proposed a novel approach for mango region segmentation from the mango tree image. The intensity of the pixel location depends on the optical property and shape roughness of the objects present. Texture is the only feature to achieve discrimination between objects in an image. Through this framework we have exploited the intensity distribution and texture features of the objects in images for the purpose of segmentation. The segmentation of mango regions from its tree image taken in an unconstrained environment is a very challenging task. Presently, there are no mango tree datasets available in the literature and hence we have created our own dataset consisting of 44 mango tree images taken in an unconstrained environment. The quantitative performance of the proposed method is analyzed and tabulated using Precision, Recall and F-measures. We have achieved 72.77 % of Precision, 71.43 % of Recall and 66.70% of F-measure. The poor results (image-20) are evidence in the Fig.11 due to the problem of illumination, leaves shadows occluded on the mango regions. In future, we would like to enhance the efficiency and accuracy of the proposed method by making better use of texture features and with the addition of features such as shape, spatial geometry and solidity etc.

References

1. Bramley, R.G.V., Proffit, A.P.B., Hinze, C.J., Pearse, B., Hamilton, R.P.: Generating benefits from precision viticulture through selective harvesting. In: Proceedings of the 5Th European Conference on Precision Agriculture, pp. 891898 (2005)

2. Ducournau, S., Feutry, A., Plainchault, P., Revollon, P., Vigouroux, B., Wagner, M.H.: An image acquisition system for automated monitoring of the germination rate of sunflower seeds. Computers and Electronics in Agriculture 44, 189202 (2004)

3. Zheng., L., Zhang., J., Wang, Q.: Mean-shift-based color segmentation of images containing green. Vegetation Computers and Electronics in Agriculture 65, 9398 (2009)

4. Burgos-Artizzu., X.P., Ribeiro., A., Tellaeche., A., Pajares., G., Fernndez-Quintanilla, C.: Analysis of natural images processing for the extraction of agricultural elements. Image and Vision Computing 28, 138149 (2010)

5. Thorp, K.R., Dierig, D.A.: Color image segmentation approach to monitor flowering in lesquerella. Industrial Crops and Products 34, 11501159 (2011)

6. Deng, Y., Manjunath, B.S.: Unsupervised Segmentation of Color-Texture Regions in Images and Video. Pattern Analysis and Machine Intelligence 23, 800810 (2001)

7. Haralick, R.M., Shanmugam, K., Dinstein, I.: Texture Feature For Image Classification. IEEE Transactions on Systems, Man and Cybernetics SMC-3, 610621 (1973)

Segmentation of Mango Region from Mango Tree Image1 Introduction2 Proposed Framework2.1 Image Thresholding2.2 Smoothing and Binarization2.3 Template Matching

3 Experimentation and Results3.1 Dataset3.2 Results

4 Conclusion and Future WorkReferences

Segmentation of Mango Region from Mango Tree Image

Documents

Transcript of Segmentation of Mango Region from Mango Tree Image