diabetic tongue diagnosis

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 2, FEBRUARY 2014 491

Detecting Diabetes Mellitus and NonproliferativeDiabetic Retinopathy Using Tongue Color, Texture,

and Geometry FeaturesBob Zhang, Member, IEEE, B. V. K. Vijaya Kumar, Fellow, IEEE, and David Zhang, Fellow, IEEE

AbstractDiabetes mellitus (DM) and its complications leadingto diabetic retinopathy (DR) are soon to become one of the 21stcenturys major health problems. This represents a huge financialburden to healthcare officials and governments. To combat thisapproaching epidemic, this paper proposes a noninvasive methodto detect DM and nonproliferative diabetic retinopathy (NPDR),the initial stage of DR based on three groups of features extractedfrom tongue images. They include color, texture, and geometry.A noninvasive capture device with image correction first capturesthe tongue images. A tongue color gamut is established with 12colors representing the tongue color features. The texture valuesof eight blocks strategically located on the tongue surface, with theadditional mean of all eight blocks are used to characterize thenine tongue texture features. Finally, 13 features extracted fromtongue images based on measurements, distances, areas, and theirratios represent the geometry features. Applying a combination ofthe 34 features, the proposed method can separate Healthy/DMtongues as well as NPDR/DM-sans NPDR (DM samples withoutNPDR) tongues using features from each of the three groups withaverage accuracies of 80.52% and 80.33%, respectively. This is ona database consisting of 130 Healthy and 296 DM samples, where29 of those in DM are NPDR.

Index TermsDiabetes mellitus (DM) detection, nonprolifera-tive diabetic retinopathy (NPDR) detection, tongue color features,tongue geometry features, tongue texture features.

I. INTRODUCTION

WORLD Health Organization (WHO) has estimated thatin 2000 there were 171 million people worldwide withdiabetes mellitus (DM), and the number will increase to 366million by 2030 [1] making the disease among the leading causes

Manuscript received December 8, 2012; revised April 26, 2013 and July 31,2013; accepted September 3, 2013. Date of publication September 18, 2013;date of current version January 16, 2014. This work was supported in part by theGRF fund from the HKSAR Government, the central fund from the Hong KongPolytechnic University, and the NSFC Oversea fund (61020106004), China.Asterisk indicates corresponding author.

B. Zhang is with the Department of Electrical and Computer Engineer-ing, Carnegie Mellon University, Pittsburgh, PA 15213 USA and also with theDepartment of Computing, Hong Kong Polytechnic University, Hung Hom,Kowloon, Hong Kong (e-mail: [email protected]).

B. V. K. Vijay Kumar is with the Department of Electrical and ComputerEngineering, Carnegie Mellon University, Pittsburgh, PA 15213 USA (e-mail:[email protected]).

D. Zhang is with the Department of Computing, Hong Kong PolytechnicUniversity, Hung Hom, Kowloon, Hong Kong (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TBME.2013.2282625

of death, disabilities, and economic hardship in the world. Twomain types of DM exist, Type 1 DM and Type 2 DM. Peoplewith Type 1 DM fail to produce insulin, and therefore requireinjections of it. Type 2 DM is the most common type and canbe categorized by insulin resistance. Currently, there is no curefor Type 1 DM or Type 2 DM. However, Type 2 DM can bemanaged by eating well, exercising, and maintaining a healthylifestyle.

A fasting plasma glucose (FPG) test is the standard methodpracticed by many medical professionals to diagnose DM. FPGtest is performed after the patient has gone at least 12 h with-out food, and requires taking a sample of the patients blood(by piercing their finger) in order to analyze its blood glucoselevels. Even though this method is accurate, it can be consid-ered invasive, and slightly painful (piercing process). Diabeticretinopathy (DR) is a microvascular complication of DM thatis responsible for 4.8% of the 37 million cases of blindness inthe world, estimated by WHO [1]. In its earliest stage knownas nonproliferative diabetic retinopathy (NPDR), the disease ifdetected can be treated to prevent further progression and sightloss. Various imaging modalities such as red-free [2], [3], an-giography [3], [4], and color [5][10] fundus imaging are usedto examine the human retina in order to detect DR and sub-sequently NPDR. These methods are based on the detection ofrelevant features related to DR, including but not limited to hem-orrhages, microaneurysms, various exudates, and retinal bloodvessels. These imaging modalities themselves can be regardedas invasive, exposing the eye to bright flashes or having fluores-cein injected into a vein in the case of angiography. Therefore,there is a need to develop a noninvasive yet accurate DM andNPDR detection method.

As a result, this paper deals with the aforementioned prob-lems and proposes a noninvasive automated method to detectDM and NPDR by distinguishing Healthy/DM and NPDR/DM-sans NPDR (DM without NPDR) samples using an array oftongue features consisting of color, texture, and geometry. Thehuman tongue contains numerous features that can used to di-agnose disease [11][25], with color, texture, and geometry fea-tures being the most prominent [11][25]. Traditionally, medi-cal practitioners would examine these features based on years ofexperience [11][25]. However, ambiguity and subjectivity arealways associated with their diagnostic results. To remove thesequalitative aspects, quantitative feature extraction and analysisfrom tongue images can be established. To the best of our knowl-edge, there is no other published work to detect DM or NPDRusing tongue color, texture, and geometry features.

0018-9294 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

492 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 2, FEBRUARY 2014

Fig. 1. Tongue capture device.

Tongue images were captured using a especially designedin-house device taking into consideration color correction [26].Each image was segmented [27] in order to locate its foregroundpixels. With the relevant pixels located, three groups of featuresnamely color, texture, and geometry were extracted from thetongue foreground. To conduct experimental results, a datasetconsisting of 130 Healthy samples taken from GuangdongProvincial Hospital of Traditional Chinese Medicine, Guang-dong, China, and 296 DM samples consisting of 267 DM-sansNPDR, and 29 NPDR processed from Hong Kong Foundationfor Research and Development in Diabetes, Prince of WalesHospital, Hong Kong SAR were used. Classification was per-formed between Healthy versus DM in addition to NPDR versusDM-sans NPDR initially using every feature individually (fromthe three groups), followed by an optimal combination of all thefeatures.

The rest of this paper is organized as follows. Section II de-scribes the tongue image capture device, color correction, andtongue segmentation, while Section III discusses tongue colorfeature extraction. In Section IV, tongue texture feature extrac-tion is given in detail, with tongue geometry feature extractionpresented in Section V. Section VI describes the experimen-tal results and discussion, followed by concluding remarks inSection VII.

II. CAPTURE DEVICE AND TONGUE IMAGE PREPROCESSINGThe capture device, color correction of the tongue images,

and tongue segmentation are given in this section. Fig. 1 showsthe in-house designed device consisting of a three-chip CCDcamera with 8 bit resolution, and two D65 fluorescent tubesplaced symmetrically around the camera in order to producea uniform illumination. The angle between the incident lightand emergent light is 45, recommended by Commission Inter-nationale de lEclairage (CIE). During image capture, patientsplaced their chin on a chinrest while showing their tongue tothe camera. The images captured in JPEG format that rangedfrom 257 189 pixels to 443 355 pixels were color cor-rected [26] to eliminate any variability in color images caused bychanges of illumination and device dependence. This allows forconsistent feature extraction and classification in the followingsteps. The idea of Wang and Zhang [26] (based on the Munsell

Fig. 2. Color gamut in the CIExyY color space depicting the tongue colorgamut inside the red boundary. Furthermore, 98% of the tongue color gamutcan be located within the black boundary.

ColorChecker) is to map a matrix generated from the input RGBvector to an objective sRGB vector, thereby obtaining a trans-formation model. Compared with the retinal imaging modalitiesmentioned previously, this capture device is noninvasive, nei-ther requiring a bright flash nor injection of dye into a patientsblood stream.

Once the tongue images are captured, automatic segmenta-tion [27] is next applied to each image in order to separate itsforeground pixels from its background. This is accomplished bycombining a bielliptical deformable template (BEDT), and anactive contour model known as bielliptical deformable contour(BEDC). In [27], the segmented tongue is obtained by first min-imizing the energy function of BEDT, followed by the energyfunction of BEDC. BEDT captures the overall tongue shape fea-tures, while BEDC can be deformed to match the local tonguedetails. The result is a binary tongue image clearly definingforeground pixels (tongue surface area and its edges) from itsbackground pixels (area outside the tongue edges). This allowsfor three groups of features, color, texture, and geometry to beextracted from a tongue foreground image in the proceedingsteps.

III. TONGUE COLOR FEATURESThe following section describes how color features are ex-

tracted from tongue images. The tongue color gamut is firstsummarized in Section III-A. In Section III-B, every foregroundtongue pixel is compared to 12 colors representing the tonguecolor gamut and assigned its nearest color. This forms the colorfeatures.

A. Tongue Color GamutThe tongue color gamut [28] represents all possible colors

that appear on the tongue surface, and exists within the red

ZHANG et al.: DETECTING DIABETES MELLITUS AND NONPROLIFERATIVE DIABETIC RETINOPATHY 493

Fig. 3. Tongue color gamut can be represented using several points by drawinglines from the RGB color space.

Fig. 4. Twelve colors representing the tongue color gamut with its label ontop.

boundary shown in Fig. 2. It was created by plotting each tongueforeground pixel in our dataset onto the CIE 1931 chromaticitydiagram (refer to Fig. 2), which shows all possible colors in thevisible spectrum. Further investigation revealed that 98% of thetongue pixels lie inside the black boundary. To better representthe tongue color gamut, 12 colors plotted in Fig. 3 were selectedwith the help of the RGB color space. On the RG line, a point Y(Yellow) is marked. Between RB, a point P (Purple) is marked,and C (Cyan) is marked between GB. The center of the RGBcolor space is calculated and designated as W (White), the firstof the 12 colors (see Fig. 3). Then, for each R (Red), B (Blue),Y, P , and C point, a straight line is drawn to W . Each time theselines intersect the tongue color gamut, a new color is added torepresent the 12 colors. This accounts for R, Y,C,B, and P . LR(Light red), LP (Light purple), and LB (Light blue) are midpointsbetween lines from the black boundary to W , while DR (Deepred) is selected as no previous point occupies that area. Moredetails about the tongue color gamut can be found in [28]. GY(Gray) and BK (Black) are not shown in Fig. 3 since both belongto grayscale.

The 12 colors representing the tongue color gamut are ex-tracted from Fig. 3 and shown in Fig. 4 as a color square with

TABLE IRGB AND CIELAB VALUES OF THE 12 COLORS

its label on top. Correspondingly, its RGB and CIELAB valuesare given in Table I.

B. Color Feature ExtractionFor the foreground pixels of a tongue image, corresponding

RGB values are first extracted, and converted to CIELAB [29]by transferring RBG to CIEXYZ using

X

Y

Z

=

0.4124 0.3576 0.1805

0.2126 0.7152 0.0722

0.0193 0.1192 0.9505

R

G

B

(1)

followed by CIEXYZ to CIELAB via

L = 166 f (Y/Y0) 16a = 500 [f (X/X0) f (Y/Y0)]b = 200 [f (Y/Y0) f (Z/Z0)]

(2)

where f (x) = x1/3 if x > 0.008856 or f (x) = 7.787x +16/116 if x 0.008856.

X0 , Y0 , and Z0 in (2) are the CIEXYZ tristimulus values of thereference white point. The LAB values are then compared to 12colors from the tongue color gamut (see Table I) and assigned thecolor which is closest to it (measured using Euclidean distance).After evaluating all tongue foreground pixels, the total of eachcolor is summed and divided by the total number of pixels.This ratio of the 12 colors forms the tongue color feature vectorv, where v = [c1 , c2 , c3 , c4 , c5 , c6 , c7 , c8 , c9 , c10, c11 , c12 ] and cirepresents the sequence of colors in Table I. As an example, thecolor features of three tongues are shown in visual form (referto Figs. 57) along with its extracted color feature vector, wherethe original image is decomposed into one of the 12 colors. Fig. 5shows a Healthy sample, Fig. 6 shows a DM sample, while anNPDR sample is given in Fig. 7. In these three samples, themajority of pixels are R.

The mean colors of Healthy, DM, and NPDR are displayed inTable II along with their standard deviation (std). DM tongueswhich have a higher ratio of DR, LR, and Y are greater in Healthysamples, and GY is higher in NPDR. The rest of the mean colorfeatures are similar. Only seven colors are listed out of the 12 asthe remaining five have ratios less than 1%.


Fig. 5. Healthy tongue sample, its tongue color feature vector and correspond-ing 12 color makeup with most of the pixels classified as R.

Fig. 6. DM tongue sample, its tongue color feature vector and corresponding12 color makeup with most of the pixels classified as R.

IV. TONGUE TEXTURE FEATURES

Texture feature extraction from tongue images is presentedin this section. To better represent the texture of tongue images,eight blocks (see Fig. 8) of size 64 64 strategically located onthe tongue surface are used. A block size of 64 64 was chosen

Fig. 7. NPDR tongue sample, its tongue color feature vector and correspond-ing 12 color makeup with most of the pixels classified as R.

TABLE IIMEAN (std) OF THE COLOR FEATURES FOR HEALTHY (stdHealthy = 11.71),

DM (stdDM = 12.50), AND NPDR (stdNPDR = 12.07)

Fig. 8. Location of the eight texture blocks on the tongue.


Fig. 9. Healthy texture blocks with its texture value below.

Fig. 10. DM texture blocks with its texture value below.

due to the fact that it covers all eight surface areas very well,while achieving minimum overlap. Larger blocks would coverareas outside the tongue boundary, and overlap more with otherblocks. Smaller block sizes would prevent overlapping, but notcover the eight areas as efficiently. The blocks are calculatedautomatically by first locating the center of the tongue using asegmented binary tongue foreground image. Following this, theedges of the tongue are established and equal parts are measuredfrom its center to position the eight blocks. Block 1 is locatedat the tip; Blocks 2 and 3, and Blocks 4 and 5 are on either side;Blocks 6 and 7 are at the root, and Block 8 is at the center.

The Gabor filter is a linear filter used in image processing,and is commonly used in texture representation. To compute thetexture value of each block, the 2-D Gabor filter is applied anddefined as

Gk (x, y) = exp(

x2 + 2 y222

)cos

(2

x

)(3)

where x = x cos + y sin, y = x sin + y cos, isthe variance, is the wavelength, is the aspect ratio of thesinusoidal function, and is the orientation. A total of three (1, 2, and 3) and four (0, 45, 90, and 135) choices wereinvestigated to achieve the best result. Each filter is convolvedwith a texture block to produce a response Rk (x, y):

Rk (x, y) = Gk (x, y) im (x, y) (4)where im (x, y) is the texture block and represents 2-D con-volution. Responses of a block are combined to form FRi , andits final response evaluated as follows:

FRi (x, y) = max (R1 (x, y) , R2 (x, y) , . . . , Rn (x, y)) (5)which selects the maximum pixel intensities, and represents thetexture of a block by averaging the pixel values of FRi .

In the end, equal to 1 and 2 with three orientations (45, 90,and 135) was chosen. This is due to the fact that the sum of alltexture blocks between Healthy and DM had the largest absolutedifference. Figs. 911 illustrate the texture blocks for Healthy,DM, and NPDR samples, respectively. Below each block, itscorresponding texture value is provided.

Table III depicts the texture value mean for Healthy, DM, andNPDR together with their standard deviation. Healthy sampleshave a higher texture value in Block 7, whereas NPDR texture

Fig. 11. NPDR texture blocks with its texture value below.

values are greater for the remaining blocks. The mean of alleight blocks is also included as an additional texture value. Thisbrings the total number of texture features extracted from tongueimages to be 9.

V. TONGUE GEOMETRY FEATURESIn the following section, we describe 13 geometry features

extracted from tongue images. These features are based on mea-surements, distances, areas, and their ratios.

Width: The width w feature (see Fig. 12) is measured as thehorizontal distance along the x-axis from a tongues furthestright edge point (xmax) to its furthest left edge point (xmin):

w = xmax xmin . (6)

Length: The length l feature (see Fig. 12) is measured asthe vertical distance along the y-axis from a tongues furthestbottom edge (ymax) point to its furthest top edge point (ymin):

l = ymax ymin . (7)Lengthwidth ratio: The lengthwidth ratio lw is the ratio of

a tongues length to its width

lw = l/w. (8)Smaller half-distance: Smaller half-distance z is the half dis-

tance of l or w depending on which segment is shorter (seeFig. 12)

z = min (l, w) /2. (9)Center distance: The center distance (cd) (refer to Fig. 13)

is distance from wsy-axis center point to the center point of l(ycp)

cd =(max (yxm a x ) + max (yxm in ))

2 ycp (10)

where ycp = (ymax + ymin) /2.Center distance ratio: Center distance ratio (cdr) is ratio of

cd to l:

cdr =cd

l. (11)

Area: The area (a) of a tongue is defined as the number oftongue foreground pixels.

Circle area: Circle area (ca) is the area of a circle within thetongue foreground using smaller half-distance z, where r = z(refer to Fig. 14):

ca = r2 . (12)


TABLE IIIMEAN (std) OF THE TEXTURE FEATURES FOR HEALTHY (stdHealthy = 1.160), DM (stdDM = 1.238), AND NPDR (stdNPDR = 1.196)

Fig. 12. Illustration of features 1, 2, and 4.

Fig. 13. Illustration of feature 5.



Circle area ratio: Circle area ratio (car) is the ratio of ca toa:

car =ca

a. (13)

Square area: Square area (sa) is the area of a square definedwithin the tongue foreground using smaller half-distance z (referto Fig. 15):

sa = 4z2 . (14)

Square area ratio: Square area ratio (sar) is the ratio of sato a:

sar =sa

a. (15)

Triangle area: Triangle area (ta) is the area of a triangledefined within the tongue foreground (see Fig. 16). The rightpoint of the triangle is xmax , the left point is xmin , and thebottom is ymax .

Triangle area ratio: Triangle area ratio (tar) is the ratio ofta to a:

tar =ta

a. (16)

The mean geometry features of Healthy, DM, and NPDR areshown in Table IV along with their standard deviation.



TABLE IVMEAN (std) OF THE GEOMETRY FEATURES FOR HEALTHY (stdHealthy =

33 013.78, DM (stdDM = 33 723.85), AND NPDR (stdNPDR = 35 673.93)

TABLE VCLASSIFICATION RESULT OF k-NN AND SVM USING EACH COLOR

INDIVIDUALLY TO DISCRIMINATE HEALTHY VERSUS DM

TABLE VICLASSIFICATION RESULT OF k-NN AND SVM USING EACH TEXTURE BLOCK

INDIVIDUALLY TO DISCRIMINATE HEALTHY VERSUS DM

VI. NUMERICAL RESULTS AND DISCUSSION

The ensuing section presents the numerical results. Healthyversus DM classification is first provided in Section VI-A. Thisis followed by NPDR versus DM-sans NPDR classification inSection VI-B.

A. Healthy Versus DM ClassificationThe numerical results were obtained on the tongue image

database comprised of 426 images divided into 130 Healthy,and 296 DM (refer to Section I). Healthy samples were verifiedthrough a blood test and other examinations. If indicators fromthese tests fall within a certain range, they were deemed healthy.In the DM class, FPG test was used to diagnose diabetes.

Half of the images were randomly selected for training, whilethe other half was used as testing. This process was repeated fivetimes. Classification was performed using k-nearest neighbor(k-NN) [30] (with k = 1) and a support vector machine (SVM)[31], where the kernel function (linear) mapped the trainingdata into kernel space. To measure the performance, averageaccuracy was employed

AverageAccuracy = sensitivity + specificity2

(17)with the average of all five repetitions recorded as the finalclassification rate. In the first step, each individual feature (fromthe three groups) was applied to discriminate Healthy versusDM. This result can be seen in Tables VVII. It should be notedthat both k-NN and SVM achieved the same average accuracyfor all 34 features. The highest average accuracy of 66.26%from this step was obtained using geometry feature ta (refer toTable VII).

In the next step, optimization by feature selection using se-quential forward selection (SFS) was performed. SFS is a featureselection method that begins with an empty set of features. Itadds additional features based on maximizing some criterion J ,and terminates when all features have been added. In our case,J is the average accuracy of the classifier (k-NN and SVM).Tables VIII (k-NN) and IX (SVM) illustrate this result appliedto each of the three main feature groups. From color features,the best combination is 3 and 12, which obtained an averageaccuracy of 68.76% using an SVM (see Table IX). In texture


TABLE VIICLASSIFICATION RESULT OF k-NN AND SVM USING EACH GEOMETRY

FEATURE INDIVIDUALLY TO DISCRIMINATE HEALTHY VERSUS DM

TABLE VIIIOPTIMIZATION OF HEALTHY VERSUS DM CLASSIFICATION USING FEATURE

SELECTION WITH k-NN

TABLE IXOPTIMIZATION OF HEALTHY VERSUS DM CLASSIFICATION USING FEATURE

SELECTION WITH THE SVM

features, 14, 15, 16, and 17 attained an average accuracy of67.67%, again using the SVM. With geometry features, 22, 30,32, 33, and 34 distinguished Healthy versus DM with an aver-age accuracy of 69.09% (in Table IX). Combining the featuresin these three groups by applying SFS, an average accuracy of77.39% was achieved in Table IX using 3, 12, 14, 15, 17, 22, 30,32, 33, and 34. Finally, by examining the best combination fromall features (SFS), the highest average accuracy of 80.52% canbe accomplished (via SVM), with a sensitivity of 90.77% and a

Fig. 17. ROC curves for Healthy versus DM (red), and NPDR versus DM-sansNPDR (blue).

specificity of 70.27%. Receiver operating characteristic (ROC)analysis was also performed on this classification as shown bythe red ROC curve in Fig. 17. The average accuracy of thisresult is higher than the optimal combination from the threefeature groups (77.39%), and contains fewer features. At thesame time, it significantly improves upon the use of all featureswithout feature selection, which obtained an average accuracyof 58.06% (k-NN) and 44.68% (SVM). The mean of features, 3,5, 12, 15, 22, 27, 30, 33, and 34 from the best overall groupingfor Healthy and DM, is shown in Table X, while Fig. 18 depictsthree typical samples from Healthy and DM.

B. NPDR Versus DM-Sans NPDR ClassificationOf the 296 DM samples, 29 were marked as NPDR (refer to

Section I). The NPDR samples were verified by medical profes-sionals after examining the retina of patients. Using the sameexperimental setup as in Section VI-A, the results of NPDRversus DM-sans NPDR (267 samples) classification are illus-trated in Tables XITables XIV. Since it was established inthe previous section that the SVM outperforms k-NN, only theformer classifier was used. The average accuracies of apply-ing each feature individually from the three groups are shownin Tables XIXIII, and Table XIV displays the optimized re-sult using SFS. As can be seen in the last row of Table XIV,the best result of 80.33% was achieved with features 7, 10,11, 14, and 25 (sensitivity82.76% and specificity77.90%based on its blue ROC curve in Fig. 17). This compares to59.05% and 53.35% average accuracies for k-NN and SVM, re-spectively, using all features without feature selection. Mean ofthe five optimal features for DM-sans NPDR and NPDR can befound in Table XV along with its three typical samples (refer toFig. 19).

For completeness, NPDR versus Healthy classificationwas also conducted. An average accuracy of 87.14% was


TABLE XMEAN OF THE OPTIMAL TONGUE FEATURES FOR HEALTHY AND DM

Fig. 18. Typical Healthy (top) and DM (bottom) tongue samples.

TABLE XICLASSIFICATION RESULTS OF USING EACH COLOR INDIVIDUALLY TODISCRIMINATE NPDR VERSUS DM-SANS NPDR USING THE SVM

TABLE XIICLASSIFICATION RESULTS OF USING EACH TEXTURE BLOCK INDIVIDUALLY TO

DISCRIMINATE NPDR VERSUS DM-SANS NPDR USING THE SVM

accomplished using SFS with the SVM, achieving a sensitivityof 89.66% and a specificity of 84.62% via features 3, 9,15, 16,and 33.

TABLE XIIICLASSIFICATION RESULTS OF USING EACH GEOMETRY FEATURE INDIVIDUALLY

TO DISCRIMINATE NPDR VERSUS DM-SANS NPDR USING THE SVM

TABLE XIVOPTIMIZATION OF NPDR VERSUS DM-SANS NPDR CLASSIFICATION USING

FEATURE SELECTION WITH THE SVM

TABLE XVMEAN OF THE OPTIMAL TONGUE FEATURES FOR DM-SANS NPDR AND NPDR

Fig. 19. Typical DM-sans NPDR (top) and NPDR (bottom) tongue samples.


VII. CONCLUSIONIn this paper, a noninvasive approach to classify Healthy/DM

and NPDR/DM-sans NPDR samples using three groups of fea-tures extracted from tongue images was proposed. These threegroups include color, texture, and geometry. A tongue colorgamut was first applied such that each tongue image can berepresented by 12 colors. Afterward, eight blocks strategicallylocated on the tongue were extracted and its texture value calcu-lated. Finally, 13 geometry features from tongue images basedon measurements, distances, areas, and their ratios were ex-tracted. Numerical experiments were carried out using 130Healthy and 296 DM tongue images. By applying each fea-ture individually to separate Healthy/DM, the highest averageaccuracy achieved (via SVM) was only 66.26%. However, em-ploying SFS with the SVM, nine features (with elements fromall the three groups) were shown to produce the optimal result,obtaining an average accuracy of 80.52%. As for NPDR/DM-sans NPDR classification, the best result of 80.33% was attainedusing five features: three from color, one from texture, and onefrom geometry. This lays the groundwork for a potentially newway to detect DM, while providing a novel means to detectNPDR without retinal imaging or analysis.

ACKNOWLEDGMENT

The authors would like to thank Xingzheng Wang for his helpin the data collection.

REFERENCES

[1] Prevention of Blindness From Diabetes Mellitus, World Health Organiza-tion, Geneva, Switzerland, 2006.

[2] J. H. Hipwell, F. Strachan, J. A. Olson, K. C. McHardy, P. F. Sharp, andJ. V. Forrester, Automated detection of microaneurysms in digital red-free photographs: A diabetic retinopathy screening tool, Diabetic Med.,vol. 7, pp. 588594, 2000.

[3] M. E. Martnez-Perez, A. Hughes, S. Thom, A. Bharath, and K. Parker,Segmentation of blood vessels from red-free and fluorescein retinal im-ages, Med. Image Anal., vol. 11, pp. 4761, 2007.

[4] T. Spencer, J. A. Olson, K. C. McHardy, P. F. Sharp, and J. V. Forrester,An image-processing strategy for the segmentation of microaneurysmsin fluorescein angiograms of the ocular fundus, Comput. Biomed. Res.,vol. 29, pp. 284302, 1996.

[5] M. Niemeijer, B. van Ginneken, J. Staal, M. S. A. Suttorp-Schulten, andM. D. Abramoff, Automatic detection of red lesions in digital color fun-dus photographs, IEEE Trans. Med. Imag., vol. 24, no. 5, pp. 584592,May 2005.

[6] M. Niemeijer, B. van Ginneken, M. J. Cree, A. Mizutani, G. Quellec,C. I. Sanchez, B. Zhang, R. Hornero, M. Lamard, C. Muramatsu, X. Wu,G. Cazuguel, J. You, A. Mayo, Q. Li, Y. Hatanaka, B. Cochener, C. Roux,F. Karray, M. Garcia, H. Fujita, and M. D. Abramoff, Retinopathy onlinechallenge: Automatic detection of microaneurysms in digital color fundusphotographs, IEEE Trans. Med. Imag., vol. 29, no. 1, pp. 185195, Jan.2010.

[7] J. J. Staal, M. D. Abramoff, M. Niemeijer, M. A. Viergever, and B. vanGinneken, Ridge based vessel segmentation in color images of the retina,IEEE Trans. Med. Imag., vol. 23, no. 4, pp. 501509, Apr. 2004.

[8] B. Zhang, F. Karray, Q. Li, and L. Zhang, Sparse representation classifierfor microaneurysm detection and retinal blood vessel extraction, Inf. Sci.,vol. 200, pp. 7890, 2012.

[9] B. Zhang, X. Wu, J. You, Q. Li, and F. Karray, Detection of mi-croaneurysms using multi-scale correlation coefficients, Pattern Recog.,vol. 43, pp. 22372248, 2010.

[10] B. Zhang, L. Zhang, L. Zhang, and F. Karray, Retinal vessel extractionby matched filter with first-order derivative of Gaussian, Comput. Biol.Med., vol. 40, pp. 438445, 2010.

[11] C. C. Chiu, The development of a computerized tongue diagnosis sys-tem, Biomed. Eng.: Appl., Basis Commun., vol. 8, pp. 342350, 1996.

[12] C. C. Chiu, A novel approach based on computerized image analysis fortraditional Chinese medical diagnosis of the tongue, Comput. MethodsPrograms Biomed., vol. 61, pp. 7789, 2000.

[13] B. Kirschbaum, Atlas of Chinese Tongue Diagnosis. Seattle, WA, USA:Eastland Press, 2000.

[14] D. Zhang, B. Pang, N. Li, K. Wang, and H. Zhang, Computerized diag-nosis from tongue appearance using quantitative feature classification,Amer. J. Chin. Med., vol. 33, pp. 859866, 2005.

[15] Y. Zhang, R. Liang, Z. Wang, Y. Fan, and F. Li, Analysis of the colorcharacteristics of tongue digital images from 884 physical examinationcases, J. Beijing Univ. Traditional Chin. Med., vol. 28, pp. 7375, 2005.

[16] D. Zhang, Automated Biometrics: Technologies and Systems. Seattle,WA, USA: Kluwer, 2000.

[17] W. Su, Z. Y. Xu, Z. Q. Wang, and J. T. Xu, Objectified study on tongueimages of patients with lung cancer of different syndromes, Chin. J.Integrative Med., vol. 17, pp. 272276, 2011.

[18] K. Q. Wang, D. Zhang, N. M. Li, and B. Pang, Tongue diagnosis basedon biometric pattern recognition technology, in Pattern Recognition FromClassical To Modern Approaches, 1st ed. S. K. Pal and A. Pal,, Eds.Singapore: World Scientific, 2001, pp. 575598.

[19] B. Huang, J. Wu, D. Zhang, and N. Li, Tongue shape classification bygeometric features, Inf. Sci., vol. 180, pp. 312324, 2010.

[20] B. Li, Q. Huang, Y. Lu, S. Chen, R. Liang, and Z. Wang, A method ofclassifying tongue colors for traditional Chinese medicine diagnosis basedon the CIELAB color space, in Proc. Int. Conf. Med. Biometrics, 2007,pp. 153159.

[21] C. H. Li and P. C. Yuen, Tongue image matching using color content,Pattern Recog., vol. 35, pp. 407419, 2002.

[22] N. M. Li, The contemporary investigations of computerized tongue diag-nosis, in The Handbook of Chinese Tongue Diagnosis. Peking, China:Shed-Yuan Publishing, 1994.

[23] G. Maciocia, Tongue Diagnosis in Chinese Medicine. Seattle, WA, USA:Eastland Press, 1995.

[24] N. Ohta and A. R. Robertson, Colorimetry: Fundamentals and Applica-tions. Hoboken, NJ, USA: Wiley, 2006.

[25] B. Pang, D. Zhang, and K. Wang, Tongue image analysis for appendicitisdiagnosis, Inf. Sci., vol. 17, pp. 160176, 2005.

[26] X. Wang and D. Zhang, An optimized tongue image color correctionscheme, IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 6, pp. 13551364, Nov. 2010.

[27] B. Pang, D. Zhang, and K. Q. Wang, The bi-elliptical deformable con-tour and its application to automated tongue segmentation in Chinesemedicine, IEEE Trans. Med. Imag., vol. 24, no. 8, pp. 946956, Aug.2005.

[28] X. Wang and D. Zhang, Statistical tongue color distribution and its appli-cation, in Proc. Int. Conf. Comput. Comput. Intell., 2011, pp. 281292.

[29] H. Z. Zhang, K. Q. Wang, X. S. Jin, and D. Zhang, SVR based colorcalibration for tongue image, in Proc. Int. Conf. Mach. Learning Cybern.,2005, pp. 50655070.

[30] R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. Hoboken,NJ, USA: Wiley, 2000.

[31] C. Cortes and V. Vapnik, Support-vector networks, Mach. Learning,vol. 20, pp. 273297, 1995.

Bob Zhang (M12) received the M.A.Sc. degree ininformation systems security from Concordia Uni-versity, Montreal, QC, Canada, in 2007, and subse-quently the Ph.D. degree in electrical and computerengineering from the University of Waterloo, Water-loo, ON, Canada, in 2011.

He is currently a Postdoctoral Researcher in theDepartment of Electrical and Computer Engineering,Carnegie Mellon University, Pittsburgh, PA, USA.His research interests include pattern recognition,machine learning, and medical biometrics.


B. V. K. Vijay Kumar (F10) received the B.Tech.and M.Tech. degrees in electrical engineering fromthe Indian Institute of Technology, Kanpur, Kanpur,India, and the Ph.D. degree in electrical engineeringfrom Carnegie Mellon University (CMU), Pittsburgh,PA, USA.

Since 1982, he has been a faculty member in theDepartment of Electrical and Computer Engineering(ECE), CMU, where he is currently a Professor andthe Associate Dean for Graduate and Faculty Affairsin the College of Engineering. He served as the As-

sociate Department Head in the ECE Department from 1994 to 1996 and asthe Acting Head in the Department during 20042005. His publications includethe book entitled Correlation Pattern Recognition (coauthored with Dr. AbhijitMahalanobis and Dr. Richard Juday, Cambridge, U.K.: Cambridge UniversityPress, November 2005), twenty book chapters, 370 conference papers, and 180journal papers. He is also a coinventor of eight patents. His research interestsinclude computer vision and pattern recognition algorithms and applicationsand coding and signal processing for data storage systems.

Dr. Bhagavatula served as a Pattern Recognition Topical Editor for the In-formation Processing division of Applied Optics and as an Associate Editor forthe IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY. He hasserved on many conference program committees and was a Co-General Chairof the 2005 IEEE AutoID Workshop, a Co-Chair of the 20082010 SPIE con-ferences on Biometric Technology for Human Identification and a Co-ProgramChair of the 2012 IEEE Biometrics: Theory, Applications and Systems (BTAS)conference. He is a Fellow of SPIE, a Fellow of Optical Society of America,and a Fellow of the International Association of Pattern Recognition. In 2003,he received the Eta Kappa Nu Award for Excellence in Teaching in the ECEDepartment at CMU and the Carnegie Institute of Technologys Dowd Fellow-ship for educational contributions and he was also a corecipient of the 2008Outstanding Faculty Research Award in CMUs College of Engineering.

David Zhang (M89SM95F08) received theGraduation degree in computer science from PekingUniversity, Beijing, China, the M.Sc. degree in com-puter science in 1982, and the Ph.D. degree in1985 from the Harbin Institute of Technology (HIT),Harbin, China. In 1994, he received the secondPh.D. degree in electrical and computer engineeringfrom the University of Waterloo, Waterloo, Ontario,Canada.

From 1986 to 1988, he was a Postdoctoral Fel-low with Tsinghua University and then an Associate

Professor at the Academia Sinica, Beijing. He is currently a Chair Professor atthe Hong Kong Polytechnic University, Kowloon, Hong Kong, where he is theFounding Director of the Biometrics Technology Centre (UGC/CRC) supportedby the Hong Kong SAR Government in 1998. He also serves as a Visiting ChairProfessor at Tsinghua University, Beijing, and an Adjunct Professor at PekingUniversity, Shanghai Jiao Tong University, Shanghai, China, HIT, and the Uni-versity of Waterloo. He is the author of more than 10 books and 250 journalpapers.

Dr. Zhang is the Founder and Editor-in-Chief, International Journal of Imageand Graphics (IJIG); Book Editor, Springer International Series on Biometrics(KISB); Organizer, the International Conference on Biometrics Authentication(ICBA); and Associate Editor of more than ten international journals includingIEEE TRANSACTIONS AND PATTERN RECOGNITION. He is a Croucher SeniorResearch Fellow, Distinguished Speaker of the IEEE Computer Society, and aFellow of International Association of Pattern Recognition.

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 600 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description >>> setdistillerparams> setpagedevice

diabetic tongue diagnosis

Documents

Transcript of diabetic tongue diagnosis