Using the Mathematical Morphology and ShapeMatching for Automatic Data Extraction in Dental

X-Ray Images

Pedro H. M. Lira ∗, Gilson A. Giraldi∗ and Luiz A. P. Neves†,∗National Laboratory for Scientific Computing

Petropolis, RJ, BrazilEmail: {pedrohml,gilson}@lncc.br†Federal University of Parana

Department of Technological and Professional EducationCuritiba, PR, Brazil

Email: {neves}@ufpr.br

Abstract—Dental X-Ray images are very popular as a firsttool for diagnosis in odontological protocols. Automating theprocess of analysis of such images is important in order to helpdentist procedures. In this practice, teeth segmentation fromthe radiographic images and feature extraction are essentialsteps. In this paper, both these tasks are addressed by usingmathematical morphology and matching technique. Firstly, itis applied top-hat and bottom-hat morphological operations forimage enhancement. Then, the Otsu’s thresholding is used to geta raw teeth segmentation. This result is processed by a labelingtechnique for identification of the target teeth. In this stage, thebinary image is eroded to get a set of seeds which are the input fora distance transform operator. From the obtained distance fielda mask is generated by using the watershed operator. This maskis used to separate objects in the Otsu’s result. In the final stepof the segmentation, a morphological close operation is appliedover the small regions remaining. The boundary of the obtainedregions are extracted and aligned with a reference shape in orderto perform the feature extraction. The experimental results showthe efficiency of the proposed method.

I. INTRODUCTION

The panoramic x-ray is a cheap and very useful toolfor dentist diagnosis. It uses a low level of radiation, beingcomfortable for the patient to have taken. The panoramicx-ray shows the dentist a patient’s nasal area, sinuses, jawjoints, teeth and surrounding bone (Figure 1). It can revealcysts, tumors, bone irregularities, among other problems. So,automating the process of analysis of such images is animportant and a difficult task due to the structures overlappingand texture patterns commonly observed in such images.

There are few researches dedicated to the specific problemof image segmentation of dental radiographs. In [1], Jain andChen presented a semi-automated method for bitewing andpanoramic dental images. Authors apply projection histogramsin order to separate the upper jaw and the lower jaw as wellas to isolate each tooth from its neighbors. This problemcan be addressed through 2-dimensional wavelets generatedby composing 1-dimensional low-pass (L) and highpass (H)filters in the horizontal and vertical direction, named LL, LH,HL and HH as performed by HajSaid et al. [2]. An LH filter

Fig. 1. A typical panoramic x-ray image.

can be used to simplify the upper and lower jaw separation,while an HL filter is used before individual teeth separationusing projection operators also. Besides, in [3] Nomir andAbdel-Mottaleb introduce a fully automated approach basedon thresholding for teeth segmentation as well as projectionsto separate each individual tooth. Image contrast enhancementand mathematical morphology has been also applied by Zhouand Abdel-Mottaleb in [4]. Lira et al. [5] proposed a work thatapplies zonal masks in the Fourier domain to address overlap-ping. Recently, Wanat and Frejlichowski [6] applied a pipelinecomposed by image enhancement (Laplacian pyramid), imagesegmentation (using the technique presented in [1]) and featureextraction (search method based on the line that separateslower/uper jaws) to address the automatic segmentation ofpanoramic images.

Teeth segmentation from dental x-ray films is an essentialstep for automating feature extraction for diagnosis as wellas forensic procedures like postmortem identification [7], [8].Segmentation is the partition of an image into multiple anddisjoint regions (sets of pixels) according to some criteriaof homogeneity of features such as color, shape, texture andspatial relationship.

This paper improves the segmentation and feature ex-traction approach for panoramic x-ray images presented by

Neves et al. [9] and Lira et al. [10], [5]. Specifically, it isdeveloped a segmentation technique that can remove overlaps.Besides, the method takes advantage of a matching techniqueto extract morphometric data. The segmentation stage is basedon mathematical morphology and thresholding filters.

The paper is organized as follows. The proposed method ispresented on section II. The experimental results are discussedon section III and the efficiency of the proposed pipelineis highlighted. Finally, section IV gives the conclusions andfuture works.

II. PROPOSED METHOD

The proposed method is divided into three main stages: (a)Preprocessing; (b) Teeth Segmentation; (c) Morphometric dataextraction. The Figure 2 describes the segmentation pipeline.

Fig. 2. Pipeline of the segmentation stage.

Firstly, a low-pass version of the original image is pro-cessed by top-hat and bottom-hat morphological operators[11], [12]. Then, the Otsu’s thresholding [13] is used and alabelling is applied. The obtained regions are searched and thelarger ones are selected to find out the teeth candidates. Next,an erode operation [11] is performed for detection of the seedsfor the teeth. Then, a distance transform [11] is used to geta distance field that will be processed by watershed [12] inorder to produce a mask to address the overlapping problem

between the teeth. Thus, the AND operation [11] is carriedout between Otsu image and the mask which gives the objectsthat are candidates to be teeth.

Sometimes, the splitting process fails and small regionsare generated. This problem is fixed using a reconstructiontechnique based on morphological close operation [11]. Theboundaries of those regions are extracted, through a simple 2Dmarching-cubes like technique [14]. Each obtained polygonalcurve is aligned with a reference model of a tooth. Thatreference shape is the mean shape obtained from a database ofmanually segmented teeth provided by Dr. Marcelo de CastroCosta and his team from the Federal University of Rio deJaneiro, Brazil.

Next, the feature extraction is started to compute the twomeasures illustrated in Figure 3: the crown-body (CB) and root(R) measures [15]. The former is obtained through the distancefrom the deepest pit to the furcation, given by the points Cand B of Figure 3, respectively. The position of these pointsare known in the reference shape. Therefore, after performingthe alignment between the extracted teeth and the referenceshape, these points can be found by a simple search. Finally,the ratio CB/R is computed. Next, the details of each step ofthe proposed pipeline are presented.

Fig. 3. The scheme for crown-body (CB) and root (R) measurements (source[15]).

A. Pre-Processing

Panoramic X-ray images have a complex intensity fieldand texture patterns, as observed on Figures 1 and 5-a. So, aGaussian filter is used to reduce noise and smooth the imagebefore performing the segmentation (Figure 5-b). The filteredimage is denoted by Ig .

B. Teeth Segmentation

In this stage the morphological top-hat and bottom-hatoperations [16] are applied. The top-hat is defined by equation[12]:

Ith = Ig − (Ig ◦B) (1)

where Ig is the smoothed version of the input image, Bthe structuring element and ”◦” is the morphological openingoperation. The Figure 4-a shows an input image and Figure 4-b pictures the result of this operation that highlights the localpeaks of the image.

Similarly to top-hat, the bottom-hat operation aims tohighlight the valleys of image [12]. This means that objectsare emphasized which simplifies their segmentation from thebackground image. This operation is defined by equation:

Ibh = Ig − (Ig •B) (2)

where Ig and B means the same as above and ”•” is themorphological closing operation. Figure 4-c shows an exampleof bottom-hat application with cross structuring element of size9× 9.

By using top-hat transform, it is possible to obtain detailsof the teeth as the edge, surface and size. This process allowsextracting the dark features. In this case, the background imageis enhanced for better identification of the teeth.

(a)

(b) (c)

Fig. 4. (a) Input image; (b) Top-hat result; (c) Bottom-hat operation result.

Therefore, an image of high definition contrast can beobtained by combining the two images Ith, Ibh as follows:

I = Ith − Ibh, (3)

This operation generates an enhanced image I as shownin Figure 5-c. The obtained image allows us to identify moredetails in the teeth image. Next, the Otsu’s thresholding isused, returning a threshold T = 99 which renders the binaryFigure 5-d.

The Figure 6 shows the result of Otsu’s thresholdingapplied on the original image I (Figure 5(a)). When comparedwith Figure 5-d it is possible to notice that the enhancedimage I allows a result where the regions of interest are betterpreserved.

Following, the labelling is applied for identification ofthe teeth in the binary image, as shown in Figure 5-e. Theobtained regions are searched and the larger ones are selected,as illustrated in Figure 5-f, in order to find out the teethcandidates. The image resolution of Figure 5-a is 1361×1411and the area threshold adopted in this paper is computed by1361·1411

30 = 64012 pixeis (see section III for details).

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Fig. 5. Proposed Method. (a) Original image; (b) Gaussian Filter’s result; (c)Combining top-hat and bottom-hat transforms according to expression (3); (d)Otsu’s image (T=102); (e) Labelling’s image; (f) The larger ones are selectedin labeling result; (g) Seeds of teeth and (h) Distances transform field.

Now, an erode operation is performed (three iterations),using cross structuring element with size 5 × 5 for detectionof the seeds of the teeth, which are shown in Figure 5-g. Otherstructuring elements can be used [11] but the cross one givesbetter results in the tests.

The seeds offer an effective way to eliminate overlaps fromthe segmented image through the distance transform operator.The result of this operator is a distance field, pictured inFigure 5-h, which is flood by the watershed algorithm [12],with 8-connected neighborhoods, using the seeds as maskerpoints. The basins frontiers generate a mask (Figure 7-a).This mask can be used to address the overlapping problembetween the teeth, using the AND logical operation betweenOtsu image and the mask (Figure 7-b). The last step of thesegmentation corrects over-segmentation problems by usingmorphological closing operation over small regions (area lessthan Image Area

30 for the images of this paper), with cross

Fig. 6. Otsu segmentation applied to the original image pictured in Figure5(a). The Otsu’s threshold is T = 93.

structuring element of size 9× 9 (Figure 7-c).

(a) (b)

(c) (d)

Fig. 7. Proposed Method. (a) Mask image; (b) AND operation between maskand Otsu images (threshold T = 102); (c) Small regions are recovered byclose operation and (d) Segmented image with all teeth identified.

The Figure 7-d highlights the result of the teeth segmenta-tion. It is important to observe that in the proposed methodthe segmentation is done automatically, without the use ofheuristics.

C. Morphometrics Data Extraction

At this stage all the remaining objects in the image areconsidered teeth. For each of these teeth it is performedboundary extraction through a simple 2D marching-cubes [14].After this step each tooth boundary is represented by a setof ordered points {(xi, yi) , i = 0, ..., N − 1}. The key ideabehind the data extraction process is to perform a matchingbetween each obtained boundary and a known reference toothboundary where the deepest pit, furcation and root (see Figure3) are marked.

Therefore, the correspondence between each curve and thereference shape must be computed. So, a feature vector iscalculated for each point according to the following expression[17]:

Fi = [ fi1 fi2 · · · fiM ] , i = 1, 2, ..., N

fij =

∣∣∣∣∣ xi−j xi xi+jyi−j yi yi+j

1 1 1

∣∣∣∣∣ .where M =

⌊N−12

⌋.

Then, at the end of this computation, two (global) featurevectors are found, Φk = [ F1 F2 · · · FN ] with k = 1, 2,by concatenating the (local) feature vectors of the curve 1,the reference model, and the feature vectors of the curve 2,the tooth boundary. In [17], the correspondence between thetwo curves is obtained through an optimization algorithm thatsearches for the cyclic permutation π of points of curve 2 thatsolves the problem:

minπ

∥∥π (Φ2)− Φ1

∥∥ . (4)

The output of this process is a set of N pairs[(x1i , y

1i ); (x2j , y

2j )] of corresponding points in both curves.

They are used to seek for the best affine transformation thataligns the input curve to the model one. The target transforma-tion A is obtained solving the following optimization problem:

minA

∥∥X2 −AX1∥∥ . (5)

where X2 is the vector of points of the tooth boundary and X1

is the vector of corresponding points in the reference curve.

After the alignment, two measures are determine alongthe main axes: crown-body (CB) and root (R) measures (seeFigure 8). The former is obtained through the distance fromthe deepest pit to the furcation. The latter is the distance fromthe furcation to the root apex. Once the reference points aremarked in the reference tooth boundary its identification in thetarget boundary is straightforward. Finally, it is computed theratio CB/R as shown in the Figure 8. These measurements arethe morphometric data focused in this research. Each extractedtooth will be represented by a CB/R ratio which can be usedin further researches for tooth classification or diagnoses [15].

III. EXPERIMENTAL RESULTS

The Figures 5 and 7 show a typical image that is usedin the experiments. The implementation was developed usingMatLab, version 7.4.

It is executed the matching method, described in sectionII-C, between the reference tooth boundary and each of theextracted boundaries. The reference shape has the points ofinterest (see Figure 3) marked a priori. So, it is possible toidentify the corresponding points in the extracted boundariesand compute the ratio CB/R according to the scheme picturedon Figure 8. Some measurement results are shown in Table I.The obtained CB/R ratio can be used for dentist diagnosis.Besides, given a population, the average value of CB/Rand its standard deviation can be used to get insights aboutmorphometric data variation of the specific population. Theresults of Table I will be used to evaluate the accuracy of thedata extraction process in further researches.

Fig. 8. Automatic measurement of interest regions.

Image CB R CB/R1 415,2 320,10 1,292 432,36 333,01 1,293 437,02 350,19 1,244 430,8 315,05 1,365 435,83 328,05 1,326 448,00 327,50 1,367 445,20 314,10 1,418 450,10 318,30 1,419 448,24 325,87 1,37

TABLE I. RESULTS OF CROW-BODY (CB) AND ROOT (R)MEASUREMENTS.

The main advantage of the proposed methodology if com-pared with the previous ones [9], [10], [5] is the ability to solveoverlapping between the teeth. Figure 9 shows the final resultobtained through the segmentation pipeline applied to Figure4-a: it is possible to observe that the mathematical morphologyis able to separate the segments of the image.

(a) (b)

Fig. 9. Overlapping problem. (a) Original image. (b) Segmentation resultwithout overlapping.

In order to highlight this process, the Figure 10 shows im-ages with overlapping that were corrected by the methodologyand its comparison with the Otsu segmentation alone. The res-olution N×M of these images are such that 1056 ≤ N ≤ 1361and 1101 ≤M ≤ 1623

Also, the whole segmentation pipeline may yield smallregions. These can be eliminated by using an area thresholdexperimentally obtained. In the examples of this section,objects that have areas smaller than A

30 are exclude whereA is the image area. For instance, for the most top leftimage, whose resolution is 1361 × 1411 the area thresholdwas 1361·1411

30 = 64012 pixeis.

The obtained result by applying the mask for overlapcorrection may also generate over-segmentation. This problemcan be fixed by taking the smallest regions (area less than A

30 )remaining and performing a close operation between them andthe surrounding objects [11].

The identification of seeds is sensitive to the number ofiterations of erosion operation. There are cases where seedsare wrongly deleted due to the amount of erosion. This mayhappens in images where the size of teeth goes from small tobig one. Also, Figure 10 shows that details of the regions ofinterest are lost and some artifacts were not eliminated. Othermorphological operators and a more suitable pre-processingstage should be considered to address this problems.

IV. CONCLUSIONS

In this paper it is presented a method for automaticsegmentation and feature extraction for dental x-ray images.The proposed method has been implemented using traditionalmathematical morphology and matching techniques.

In the experimental results, a visual inspection shows that itis a promising method. Further works include the developmentof a more sophisticated pre-processing pipeline, test of othermorphological operators to improve the segmentation and thedevelopment of a more robust technique for seed identification.

ACKNOWLEDGEMENTS

Authors would like to thank Dr. Marcelo de Castro Costaand his team from the Faculty of Odontology of FederalUniversity of Rio de Janeiro, Brazil for the databases usedin this research. In addition, the authors thank the supportprovided by PCI-LNCC, FAPERJ (grant E-26/170.030/2008)and CAPES (grant 094/2007).

REFERENCES

[1] A. Jain and H. Chen, “Matching of dental X-ray images for humanidentification,” Pattern Recognition, vol. 37, no. 7, pp. 1519–1532,2004.

[2] E. HajSaid, D. Nassar, G. Fahmy, and H. Ammar, “Dental x-ray imagesegmentation,” in SPIE Technologies for Homeland Security and LawEnforcement conference, 2004.

[3] O. Nomir and M. Abdel-Mottaleb, “A system for human identificationfrom X-ray dental radiographs,” Pattern Recognition, vol. 38, no. 8, pp.1295–1305, 2005.

[4] J. Zhou and M. Abdel-Mottaleb, “A content-based system for humanidentification based on bitewing dental X-ray images,” Pattern Recog-nition, vol. 38, no. 11, pp. 2132–2142, 2005.

[5] P. H. M. Lira, G. A. Giraldi, and L. A. P. Neves, “Panoramic DentalX-Ray Image Segmentation and Feature Extraction,” in Proceedings ofV Workshop of Computing Vision, Sao Paulo, Brazil, 2009.

[6] R. Wanat and D. Frejlichowski, “A problem of automatic segmentationof digital dental panoramic x-ray images for forensic human identifica-tion,” in The 15th Central European Seminar on Computer Graphics,2011.

[7] E. Said, D. Nassar, G. Fahmy, and H. Ammar, “Teeth segmentationin digitized dental X-ray films using mathematical morphology,” IEEETransactions on Information Forensics and Security, vol. 1, no. 2, pp.178–189, 2006.

[8] J. Cameron, B. Sims, and K. Simpson, Forensic dentistry. ChurchillLivingstone Edinburgh, 1974.

(a) (b) (c) (d)

Fig. 10. Examples of the segmentation process from the enhanced image. (a) Original image. (b) Enhanced image. (c) Segmented image without overlapping.(d) Otsu’s threshold T applied to images of the first column (From top to bottom: T=93, T=97, T=88, T=101, T=85).

[9] L. A. P. Neves, G. Giraldi, A. G. Costa, E. C. Kuchler, and D. E. M. .Oliveira, “Automatic Data Extraction in Odontological X-Ray Imaging,”in Proceedings of VISAPP International Conference on Computer VisionTheory and Applications, Lisboa, Portugal, vol. 1. INSTICC PRESS- Institute for Systems and Technologies of Information, Control andCommunication, 2009, pp. 141–144.

[10] P. H. M. Lira, G. A. Giraldi, and L. A. P. Neves, “An AutomaticMorphometrics Data Extraction Method in Dental X-Ray Image,” inProceedings of I International Conference on Biodental Enginnering,Porto, Portugal, 2009.

[11] A. Jain, Fundamentals of digital image processing. Prentice-Hall, Inc.Upper Saddle River, NJ, USA, 1989.

[12] J. Serra, Image Analysis and Mathematical Morphology. AcademicPress, 1988.

[13] N. Otsu, “A THRESHOLD SELECTION METHOD FROM GRAY-LEVEL HISTOGRAMS,” In: Proceedings of the IEEE Transactionson Systems, Man and Cybernetics, pp. 62–66, 1979.

[14] E. L. Allgower and K. Georg, Numerical Continuation Methods: AnIntroduction. Springer-Verlag Berlin Heidelberg, 1990.

[15] W. K. Seow and P. Y. Lai, “Association of taurodontism with hypodon-tia: a controlled study.” Pediatr Dent, vol. 11, no. 3, pp. 214–9, 1989.

[16] G. Fahmy, D. Nassar, E. Haj-Said, H. Chen, O. Nomir, J. Zhou,R. Howell, H. Ammar, M. Abdel-Mottaleb, and A. Jain, “Toward an

automated dental identification system,” Journal of Electronic Imaging,vol. 14, p. 043018, 2005.

[17] H. Ip and D. Shen, “An affine-invariant active contour model (AI-snake)for model-based segmentation,” Image and Vision Computing, vol. 16,no. 2, pp. 135–146, 1998.