Computers in Biology and Medicine 42 (2012) 135–146

Contents lists available at SciVerse ScienceDirect

Computers in Biology and Medicine

0010-48

doi:10.1

n Corr

E-m

ari@dee

jorgeme

journal homepage: www.elsevier.com/locate/cbm

Computational methodology for automatic detection of strabismus in digitalimages through Hirschberg test

Jo~ao Dallyson Sousa de Almeida n, Aristofanes Correa Silva, Anselmo Cardoso de Paiva,Jorge Antonio Meireles Teixeira

Federal University of Maranh ~ao—UFMA, Applied Computing Group—NCA/UFMA, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga 65085-580, S ~ao Luıs, MA, Brazil

a r t i c l e i n f o

Article history:

Received 9 September 2010

Accepted 6 November 2011

Keywords:

Medical image

Strabismus

Hirschberg test

Geostatistical functions

Image processing

Pattern recognition

Support vector machine

25/$ - see front matter & 2011 Elsevier Ltd. A

016/j.compbiomed.2011.11.001

esponding author. Tel.: þ55 98 33018243; fa

ail addresses: jdallyson@yahoo.com.br (J. Dall

.ufma.br (A. Correa Silva), paiva@deinf.ufma.

ireles1@bol.com.br (J. Antonio Meireles Teixe

a b s t r a c t

Strabismus is a pathology that affects about 4% of the population, causing aesthetic problems, reversible

at any age; however, problems that can also cause irreversible muscular alterations, and alter the vision

mechanism. The Hirschberg test is one of the exams used to detect this pathology. The application of

high technology resources to help diagnose and treat ophthalmological conditions is, lamentably, not

commonly found in the sub-specialty of strabismus. This work presents a methodology for automatic

detection of strabismus in digital images through the Hirschberg test. For such, the work was organized

into four stages: (1) finding the region of the eyes; (2) determining the precise location of the eyes;

(3) locating the limbus and brightness; and (4) identifying strabismus. The methodology has produced

results on the range of 100% sensibility, 91.3% specificity and 94% for the correct identification of

strabismus, ensuring the efficiency of its geostatistical functions for the extraction of eye texture and

for the calculation of the alignment between the eyes on digital images obtained from the

Hirschberg test.

& 2011 Elsevier Ltd. All rights reserved.

1. Introduction

Strabismus is an abnormal condition that makes the eyes losetheir parallelism between. While an eye stares at a frontal point,the other turns aside, or even upwards and downwards. Becauseof this, the brain receives two images with different focuses,instead of two images that converge into a single spot. There areseveral types of strabismus: the affected eye can be yawed towardthe nose (convergent strabismus); it can turn aside (divergentstrabismus); or it turns upwards or downwards (vertical strabis-mus). There can be a combination of horizontal and verticalyaw in the same patient, as, for example, toward the nose andupwards.

In general, it can be said that the mechanical component ofstrabismus, in other words, the esthetic aspect of the yaw, can betreated at any age. On the other hand, the sensorial disturbancesare more significant, and are only treatable at a certain period inone’s life—the stage of plasticity of the visual system, whichlingers on till the age of nine. Thus, as the main sensorialcomplication of a yaw is the strabismic amblyopya, its treatment

ll rights reserved.

x: þ55 98 33018841.

yson Sousa de Almeida),

br (A. Cardoso de Paiva),

ira).

must be initiated as soon as a strabismus condition with amblyo-genic characteristics is detected [1,2].

To diagnose strabismus, the following exams are performed:visual acuity, eye background, external examination of the eyes(cornea, sclera, conjunctiva, iris, lens, etc.), and eye movementexam, obtained by means of the Cover test and the Hirschbergtest. The Hirschberg test consists basically of sending a thin beamof light into the patient’s eyes in order to verify if the reflection ineach eye is located at the same place on both corneas. Besidesthese exams, there are the devices called electronic synopto-phores, which measure strabismus via the projection of twoseparate and dissimilar images in the same position in space.

Despite the increasing use of cutting-edge resources to helpwith the diagnosis and the treatment of various ophthalmologicalconditions, the sub-specialty of strabismus has not been given thesame importance. Considering the fact that it is not easy to findprofessionals with enough experience in this sub-area away fromlarge urban centers – a fact that makes precocious diagnosesmore difficult – these technologies have become essential in citiesfather away from those more advanced centers.

In 1998, the Brazilian government created the Unified HealthSystem (SUS). This system provides health assistance – fromsimple ambulatory assistance to organ transplantations – ensur-ing integral, universal and cost-free health benefits for the entirepopulation. Within this program, and in harmony with theprinciples and dictums of the SUS, the Health Program at Schools

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146136

(PSE) was created in September, 2008, aiming at reinforcinghealth actions in the primary sphere, caring for the preventionand the promotion of better health assistance in Brazilian schools.

The PSE structure lies upon four platforms. Let us consider thefirst of these, from where there stems the main objectives of ourwork. This includes, among the ever so large numerous healthissues, visual acuity tests. Consequently, the relevance of ourwork may be directed mainly toward the precocious diagnoses ofstrabismus and also toward the implementation of the ophthal-mologic acuity test. This project can be further extended to theactivities developed in the Family Health Program (PSF). Besides,this is a test that can be safely applied by non-specialized stuff,helping with patient triage, and contributing to the reduction ofwaiting queues and public expenses.

The field that involves the use of computational tools tosupport the diagnosis of strabismus has come under considera-tion recently. However, some tools have already been or will bedeveloped so that health professionals can make reliable deci-sions concerning a number of sight pathologies.

In [3], the development of a device called Trophorometer isdescribed. The Trophorometer is used to measure the position andthe movement of the eye by employing computerized imageprocessing to help diagnosing phorias and tropias. The movingwindow thinning technique was used to detect the edges of thepupil and limbus, and Hough’s Transform was applied to locatethe pupil.

In [4], it is proposed a method that uses telemedicine to treatstrabismus in locations where a specialist is unavailable. For such,digital photographic cameras were employed to capture patients’images, and computers were used to send the images via e-mailto a strabismologist1 so that the images could be analyzed.

Eye motion research laboratories use ocular trackers or mag-netic devices to measure yaws and eye movements, but, despitethe precision of these devices, these methods are expensive andhard to apply in a real situation [5].

There is also the report of the devices used to measurestrabismus, or any other devices that use the same basis as thatof a synoptophore. These devices basically work like a commonoptical synoptophore, but the fixing images are generated elec-tronically on video, and the measurements are done by means of acomputer [6]. Nevertheless, synoptophores are difficult to use foreye motion by non-specialized people. Also they are neithercompact nor easy to transport. They can only be used oncollaborative patients. Finally, such devices have not been usedfor the evaluation of the yaw in these past decades. This wouldrequire a very special universe of equipments, something like alaboratory, far from the patient’s daily reality.

In order to develop a method capable of helping the specialistin the detection of strabismus, one would require initially todetermine the position of the eyes. Many approaches have beendeveloped to automatically detect the position of the eyes fromdigital images. In [7], a method to detect the eyes in facial imagesusing Zernike’s moments with support vector machines (SVM) ispresented. Here, the eye and non-eye patterns are represented interms of the magnitude of Zernike’s moments, and are classifiedby means of the SVM. Zernike’s moments are invariant torotation, that is, they can detect the eyes, even if the face hasbeen rotated. The orthogonal property of Zernike’s polynomialallows each moment to be unique and independent as to theinformation provided by an image. This method has achievedmatching rates of 94.6% for detection of eyes in the face imagesfrom the ORL base.

1 Physician specialized in the treatment of strabismus.

With similar goal in mind, the authors in [8] have proposed amethod for the automatic detection of human faces’ digitalimages by using the semivariogram, geostatistical function torepresent the region of the eyes, and a support vector machine toclassify eye candidates. The detection obtained matching rate of88.45% for images from the ORL base.

Geostatistical functions have been applied to other works. In[9], a method to identify people through the analysis of iristexture by using semivariogram and correlogram functions wasproposed. This method produced success rate of 98.14%, and itused an iris base called CASIA. In [10], the geostatistical functionswere used to classify lung nodules, as either malignant or benignin computerized tomography images.

Differently from the equipment and methods presented in theintroduction, and presently being used by ophthalmologists, thiswork proposes the development of an easy, fast and cheap way ofautomatically diagnosing someone with strabismus. For thisreason, this is a method most useful for the average ophthalmol-ogist. A digital camera and a computer – either portable or not –will be used with strabismus detection software installed, incompliance with the methodology proposed in the present work.

This work, based on a master’s dissertation developed by theauthor [11], aims to evaluate the efficiency and effectiveness ofthe use of image processing and pattern recognition techniques toautomatically diagnose strabismus based on digital images ofhuman faces. The geostatistical measurement emivariogram,semimadogram, covariogram and correlogram have been usedtogether with image processing techniques (Canny’s Method andHough’s Transform), selection of features (stepwise DiscriminantAnalysis)and pattern recognition (Support Vector Machines) toverify and determine if a person is strabic, by using the Hirsch-berg test as reference.

The remaining of the present work was organized into foursections. Section 2 provides the theoretical basis, without which itwould be difficult to understand our approach. Section 3describes the four stages (detection of eye region, eyes location,location of limbus and brightness, and the identification ofstrabismus), which comprises the methodology used to detectstrabismus from digital images based on the Hirschberg test. InSection 4, the results obtained by the proposed methodologyare shown and discusses. Finally, Section 5 presents the workconclusions, analyzing the efficiency of the techniques used.

2. Theoretical basis

This section presents the theoretical basis necessary for theunderstanding of the proposed methodology.

2.1. Strabismus

Strabismus, one of the commonest ophthalmologic alterna-tions in childhood, can be defined as an abnormal binocularinteraction between the eyes, where the same image does notreach the fovea2 of both eyes at the same time; consequently, theeyes do not fixate on the same image.

Once the position of each eye (center of the pupil) is deter-mined, relative to a reference (either the observed point or theobservation point), i.e. the directions of each axis (eitherthe visual or the pupillary point), strabismus may be defined asthe difference between the expected alignments, i.e. the anglebetween the ocular directions, corresponding to a disturbance of

2 The fovea is located in the optical axle of the eye, on which is projected the

image of the focused object, and the image formed on it is very sharp [12].

Fig. 1. Example of strabismus.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146 137

the binocular positional relation, relative to a given point (nor-mally, the object toward which the sight is directed) [13]. Fig. 1illustrates the occurrence of strabismus.

The symptoms and the consequences of strabismus differaccording to the age at which it appears and the way it manifestsitself. Strabismus that appears before the age of 6 has anadaptation mechanism that makes the image created in theyawed eye be suppressed, and, as a result, the patient does notpresent diplopia.3 However, sight diminishment occurs (amblyo-pya or ‘‘lazy sight’’) in the yawed eye. On the other hand, if aperson becomes strabic after 6 years of age, then this person willpresent diplopia: each eye will focus the image on to differentpositions, relative to the yaw. In a child, diplopia is periodical andleads to suppression. This suppression consists of a corticalmechanism of elimination of the image caught by the yawedeye, something that occurs only in children who still havecerebral plasticity.

Several techniques can be applied to the treatment of strabis-mus with the objective of restoring muscular balance and solvingthe problem of amblyopya. The medical treatment commonlyused is: prescription of glasses, execution of orthoptical exercises,and obstruction of the fixating eye, alternating with the other eye.If the medical treatment does not suffice, surgery may berecommended to ensure the retrocession of the weakened ocularmuscles.

2.2. The Hirschberg method

In order to evaluate the strabismus yaw by using luminousfocus, one should initially describe the Hirschberg test thatcalculates the approximate magnitude of the yaw relative tothe luminous reflection displacement of the cornea in thenon-fixating eye, taking into account the center of its ocularglobe. Depending on the reflection location incidence, withrespect to the complex limbus-iris-pupil, one can infer themagnitude of the yaw. Alternatively, in order to avoid thevariations resulting from the size of the pupil, one may correlatethe luminous reflection to the center of the cornea and the limbus[14]. The term corneal luminous reflection is unsuitable, for it isnot a reflection from outside the cornea. What we can first see asa luminous reflection is actually the reflection of Purkinje’s image,which is a virtual image located behind the pupil [15].

When examining an individual by means of the Hirschbergtest in order to diagnose strabismus, one must observe that thefixating eye has the first Purkinje’s image aligned to its opticalcenter, and consequently the other eye, the non-fixating eye, isthe eye in which the yaw must be observed. The yaw is inferredby comparing the reflection of the light in the anterior surface ofthe cornea with its optical center and by detecting whether thereis a misalignment. As it is difficult to determine on a non-fixatingeye its precise location, the yaw will be evaluated in relation tothe anatomic center of the eye, or, in other words, in relation tothe center of the pupil. One can notice from this description theexistence of another variable interfering with the observation of

3 Diplopia consists in the perception of the same object in two different

spatial locations (in retina).

the yaw; and that is the Kappa angle.4 This angle must bemeasured for that eye and must be taken into considerationwhen examining the reflex. However, other factors interferewith the relative positioning of the luminous reflection on thenon-fixating eye in relation to the position the reflex assumes inthe fixating eye. These factors are: corneal curvature, the size ofboth cornea and eye, and refraction. If the data obtained fromboth eyes are too dissimilar, then these can disturb the evalua-tion; so much so that, when attempting to analyze or quantify theyaw by means of the Hirschberg method, one must take all ofthese factors into consideration [16,17].

2.3. Geostatistical functions for extraction of features

This work proposes the analysis of digital image textures bymeans of geostatistical functions, so as to form a textural pattern.Such functions, largely known as the Geostatistics study area, areemployed in this study to describe and recognize the patternidentified by regions of eyes and non-eyes (the other areas of theface: nose, mouth, ears, etc.).

In this context, we use four geostatistical functions (semivar-iogram, semimadogram, covariogram and correlogram) and acombination of these measurements in the extraction of featuresto identify and elicit the region of the eyes [10]. The advantage ofthese functions is that the spatial variability and correlationfeatures are analyzed all together. These functions encompassthe association between the function distance and a possibledirection.

In statistics, texture can be described in terms of two maincomponents related to pixels (or any other unit): spatial varia-bility and autocorrelation. The advantage of the use of spatialstatistical techniques is that both aspects can be measuredtogether, as they will be discussed in the following sections.These measurements describe the texture obtained from a certainimage through the degree of spatial association present inside theimage’s geographically referenced elements. The pixels’ organiza-tional correlation, taken as independent points, can be analyzedwith several measurements, such as those described in thesequence of this section.

2.3.1. Semivariogram

The curve relating the semivariance as a function of thedistance of a point is called Semivariogram. The greater thedistance between the samples, the greater will be the semivar-iance; and the smaller the distance between them, the smallerwill be the semivariance.

Semivariogam is defined by

gðhÞ ¼ 1

2NðhÞ

XNðhÞi ¼ 1

ðxi�yiÞ2

ð1Þ

where h is the distance vector (lag distance) between the values oforigins, xi; and the values of extremity, yi, and N(h) is the numberof pairs in the distance h.

The other values used to calculate the semivariogram, such aslag spacing, lag tolerance, direction, angular tolerance and maximum

bandwidth are illustrated in Fig. 2.

2.3.2. Semimadogram

Semimadogram is the mean of the absolute difference mea-sured in the pairs of the sample, as a function of distance and

4 Angle formed by the visual line and the axle of the pupil.

Y

X

Lag tolerance

Lag increment

Angular tolerance

BW

Bandwith

Direction vector (h)

h Lag1Lag2

Lag3Lag4

Fig. 2. Parameters used in the calculation of geostatistical functions [10].

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146138

direction [10]. The function is defined by

mðhÞ ¼1

2NðhÞ

XNðhÞi ¼ 1

9xi�yi9 ð2Þ

where h is the distance vector (lag distance) between the values oforigins, xi, and the values of extremity yi, and N(h) is the numberof pairs in the distance h.

5 Biomicroscopy corresponds to the outside view of the eye (cornea, sclera,

conjunctive) as well as of all components of the anterior chamber (iris, aqueous

humor, crystalline and its capsules) and even part of the posterior segment

(anterior vitreous and retina), through proper lens.6 Fundoscopy is the exam of the eye background.7 Tonometry is the measurement of the pressure inside the eye.8 The evaluation of the eye motion is performed through cover test (of the

manual occlusor) or through the corneal luminous reflection, asking the patient to

stare at one point (cover test) or light (luminous reflection), to verify the yaw of

the eye near and far.

2.3.3. Covariogram

The covariogram measures the correlation between two variables.In Geostatistics, covariance is calculated as the variance of the sampleminus the value of the variogram. The covariance function tends toincrease as the variables’ values are closer to each other i.e., whenh¼0; and tends to decrease as these values are farther away fromeach other, or nearer to the limit. Covariogram is defined by

CðhÞ ¼1

NðhÞ

XNðhÞi ¼ 1

xiyi�m�hmþh ð3Þ

where m�h is the mean value of the vectors’ origins, and mþh is themean value of the vectors’ extremities

m�h ¼1

NðhÞ

XNðhÞi ¼ 1

xi ð4Þ

mþh ¼1

NðhÞ

XNðhÞi ¼ 1

yi ð5Þ

2.3.4. Correlogram

The correlation function (correlogram) is the normalizedversion of the covariance function. The coefficients of correlationrange from �1 to 1. The correlation is expected to be higher forunits close to each other (correlation¼1 for distance zero) and ittends to zero when the distance between the units increases [10].

Correlation is defined by

rðhÞ ¼ CðhÞ

s�hsþhð6Þ

where s�h is the standard deviation of the values of the vectors’origins, and sþh is the values’ standard deviation of the vectors’extremities

s�h ¼1

NðhÞ

XNðhÞi ¼ 1

x2i �m2

�h

" #1=2

ð7Þ

sþh ¼1

NðhÞ

XNðhÞi ¼ 1

y2i �m2

þh

" #1=2

ð8Þ

2.4. Validation

The methodology uses sensitivity, specificity and accuracy analysistechniques. These are the metrics commonly used to analyze theperformance of systems based on image processing. Sensitivity (SE) isdefined by TP/(TPþFN), specificity (SP) is defined by TN/(TNþFP),and accuracy (AC) is defined by (TPþTN)/(TPþTNþFPþFN), whereTN is true-negative, FN is false-negative, FP is false-positive, and TP istrue-positive.

3. Materials and methods

This section describes the procedures used for the automaticdetection of strabismus via digital images from the patient’s facialimages. First, we introduce the image base used in tests. Next, wedescribe a sequence of the stages developed in order to achievethe goals of the proposed methodology.

3.1. Patients

The images used in the present study were taken of patientsfrom a private ophthalmologic clinic specialized on strabismus inthe city of So Lus—MA, Brazil. The images were taken over aperiod of 8 months. The patients who volunteered to participatein the study signed a Consent Term.

3.1.1. Acquisition protocol

The patients were submitted to an acquisition protocol estab-lished by the physician. This protocol determines the criteriaobserved for the patients’ inclusion or exclusion from the data-base. These criteria are described as follows:

The patients included in the database were examined, after thefollowing criteria have been observed: visual acuity with the bestvisual correctness, Biomicroscopy,5 fundoscopy6 (posterior pole),planning tonometry7 (as often as possible) and eye motion exam8

through Cover Test.The Cover Test may reveal two situations:

1.

There is no yaw: the strabologic evaluation is completed with4D test and the sensorial evaluation is done by means of theTitmus stereoscopic acuity test:(a) if these last two evaluations are normal, the patient will be

included in the control group (without strabismus);(b) if at least one of these two exams presents alterations, the

patient will be included in the test group (with strabismus).

2. In case of a yaw, the method of alternated prism and cover will

be applied to quantify the yaw. The patient is included in thetest group if his/her vertical and/or horizontal yaw is below orequal to 15D. Above this value, the patient will be submitted toone of the exclusion criteria.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146 139

The following criteria were applied to exclude patients from thetests:

�

bot

use

Test group (with strabismus)J Having horizontal and/or vertical yaw above 15D.J Opacity or any other alteration of the cornea and/or eyelid

that compromises the observation of luminous reflectionsin both corneas.

J Irregularity on the limbus contour.J Alterations in the size of one of the eyes, such as micro-

phtalmia. Iris or pupil alterations (leucocoria, aniscoria,discoria) have not been included as exclusion criteria, aswell as alterations of the posterior segment, with or with-out visual failure.

J Nystagmus9 present.

9 N

h ey10 T

d to11 V

Fig. 3. Acquisition of the patient’s face picture.

� Control group (without strabismus)

J Inability to achieve 1.0/1.0 (on the Snellen table10) of visionwith the best visual correction or to inform visual acuity.

J Inability to achieve on arc of 40 s in the Titmus’ stereo-scopic visual acuity test or inability to perform the test.

J Nistagmo present.

3.1.2. Image acquisition

The acquisition of the images was done in the same ophthal-mologic clinic by using a SonyR Cyber-shot camera with 8.1 mega-pixels, with optical zoom of 3� , adjusted to image capturingmode (with accurate details and sharpness) at a resolution of2048 �1536 pixels.

The picture is taken with the patient sitting on the examina-tion chair. The patient’s face must be centered at about 40–50 cmaway from the camera. The lights of the room are on, but withoutany complementary light focus. The patient is asked to stare at anaccommodative picture stuck laterally to the camera’s objectivelens. The flash will be on to provide the necessary brightness, orthe first Purkinje’s image.11 The function macro is used to securethe perfect focusing of the acquired image, even when the patientis close to the camera. If the patient wears corrective lens, thepicture will be taken with the lens on. Fig. 3 illustrates the pictureof the patient’s face, taken in the clinic.

The images acquired for the tests comprised 45 pictures ofpatients of both genders and of different ages, with and withoutcorrective lens. From these 45 patients, 15 were pronouncedstrabic by the specialist.

3.2. Proposed methodology

The detection of strabismus in digital images depends on theprecise location of the limbus and on the brightness detected onthe images. Thus, to meet such needs, the proposed methodologyhas been organized into four stages, represented in Fig. 4.

On the first stage one seeks to obtain the region of the eyes, tominimize the search space, and to exclude the regions that playan important role in the methodology. Next, the precise locationof the eyes is established, reducing still further the search space.On the third stage, the location of the limbus and brightness isdetermined, leading to a diagnosis of the patient’s condition i.e.,whether he/she has strabismus or not. All these stages will bediscussed in the following section.

ystagmus are repeated and involuntary rhythmic oscillations of one or

es in some or all of the fixating positions.

he Snellen table, also known as Snellen’s optometric scale, is a diagram

evaluate the visual acuity of a person.

irtual image located behind the pupil [15].

3.2.1. Detecting the eye region

On the preliminary stage of automatic detection of the eye region,when adjusting what has been proposed in [18], one aims to reducethe search space, generating a sub-image within the eye region, andexcluding the non-interesting regions (mouth, nose, hair, back-ground), in order to make the next stage easier; and by that wemean the eye location. The methodology starts with the acquisition ofthe image followed by the re-dimensioning of the image from2048�1536 pixels, which is the original resolution, to a resolution10 times lesser, 205�154 pixels, the objective of which is tominimize the computational costs of image processing. This reducedimage is used until the stage of eye location is reached; since, on thestage of limbus location, a higher resolution image is needed to avoiddata loss that would corrupt the stage of limbus and brightnessdetection.

The image is also converted from RGB color to a gray level color soas to promote computational efficiency. After the conversion, thehomomorphic filter [19] is applied in order to prevent luminositydivergences. On the next stage, the image is smoothed out by meansof a 3�3 mask Gaussian filter. Next, the input image gradient,generated on the previous stage, is calculated by using the Sobelfilter [20].

A horizontal projection of this gradient is applied obtaining as aresult the mean of three of the projection’s highest peaks. It isrelevant to know that the eyes are found in the superior part of theface and that together with the eyebrows they correspond to the twopeaks closer to each other. This physiologic information, known a

priori, can be used to identify the area of interest, and the peak of thehorizontal projection will supply the horizontal position of the eyes.

At the same time, the vertical projection is applied to the gradientimage. There are two peaks to the left and to the right thatcorrespond to the limits of the face. From these two limits, the lengthof the face is estimated. Combining the results of these projections weobtain the image coordinates representing the region of the eyes.

3.2.2. Location of the eyes

The stages proposed for locating the eyes, in the region delimitedby the previous stage, follow the sequence represented in Fig. 5.

To use the pattern recognition technique, one needs todetermine the location of the eyes. This is done along two stages:training and testing. On the training stage, a classification modelis created; while on the testing stage, the samples are classifiedvia the trained model. The only difference in the implementationof these two stages lies in the sample used on the next stages. Forthe training stage, 1008 samples of 28 images of patients wereselected: being 18 of eye regions, 9 of each eye, and 18 of theother regions of the face areas. This means that for every imageused on the training stage, 36 samples were manually extracted.These were formed by a 30�30 pixels window.

Fig. 4. Stages of the proposed methodology.

Fig. 5. Eye location stages.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146140

For the testing stage, the samples were automatically detectedby the Hough transform, which was employed to locate eyecandidates by using radiuses intervals12 from 4 to 10 pixels. Sixcoordinates with more votes in the accumulation vector of thefirst and second half of the image were extracted, correspondingto the left eye and the right eye, respectively.

The samples are then pre-processed via histogram equaliza-tion [20]. After the previous stage pre-processing, the regions ofinterest are then carried on to the features extraction stage. Atpresent, geostatistical functions are used to describe the textureof objects representing eyes and other regions of the face,extracted from face images. The functions used were: correlo-gram, covariogram, semivariogram and semimadogram.

The geostatistical function parameters used to extract featuresfrom each sample were the directions 01, 451, 901 and 1351 withan angular tolerance of 22.51 and an increment lag (distance)equal to 1, 2 and 3, corresponding to 29, 14 and 9 lags andtolerance of each lag distance equivalent to 0.5, 1.0 and 1.5,respectively. The directions adopted were the ones most used inthe literature for the analysis of images; because to choose the lag

tolerance in accordance with [21], the commonest procedurewould be to adopt half the lag increment.

To build the Features Vector (FV), which represents the samplesignature, 208 features per sample were extracted, correspondingto the four directions of 52 lags ð29þ14þ9Þ for each geostatisticalfunction. A combination of the four geostatistical functions usedin this work results in an FV of 832 features (4�208). Beforeimplementing their selection, the features undergo a normal-ization process along a common range of values, such as �1 toþ1. This mechanism helps the classifier to converge more easilyduring the training stage. Besides, this will standardize the

12 The radiuses intervals considered in this work were determined through an

analysis done on the image base used in tests.

distribution of variable values, which may assume differentdomains.

Feature extractions performed by the geostatistical functionsgenerate many variables. So, a selection of the features that betterdistinguish the eye and the non-eye classes (the other areas of theface) is carried out by applying the stepwise DiscriminantAnalysis technique [22].

On the final stage objects such as eye and non-eye areidentified by means of pattern recognition techniques. Thismethodology uses Support Vector Machine (SVM) [23].

The image base used for training and testing the SVM classifieris composed of manually selected image samples taken from 28patients, and from eye candidates identified with the applicationof the Hough method after the region detection in the imagessubmitted to the test. We used the LIBSVM [23] library. The SVMwas used with radial basic function (RBF).

The recognition process comes to an end with the training andtesting of the SVM, which generates the location of the eyes. Afterrecognition, if necessary, the calculation of similarity among theregions classified as eye is done by using the Absolute MeanError [24].

Similarity is used to locate the two regions corresponding toboth the right eye (RE) and the left eye (LE) when there is morethan one region classified by SVM. Similarity is also used todetermine the corresponding eye in the candidates, when onlyone eye is found.

3.2.3. Location of limbus and brightness

The location of the reflection generated by the Hirschberg testis used as a parameter for verifying the alignment of both eyes.For such, we applied the Canny edge detection algorithm, and theHough Transform (HT). Optimum result on this stage depends ona reasonable percentage of well-defined limbus edges, so that HTcan detect it (the limbus) more accurately.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146 141

In this step, the acquired images resized to an 819�614 pixelsresolution are used. This is done to reduce the computational costof the image processing, without missing the details of the limbusedge. The images are also converted from RGB to gray levels.

The coordinates of the eyes found on the previous stage arere-scaled to determine the corresponding values for the resolu-tion of 819�614. Next, the Canny method is applied after beingconfigured with a derivation factor of 1.2, mask of 5�5 used inthe Gaussian function: with lower bound of 100 and upper boundof 136. To determine the edges’ location of the limbus, the HTtechnique is used. This technique employs the map of edgescreated on the previous stage.

To determine the location of the limbus’ edges, the HTtechnique is used. This technique uses the edges map generatedon the previous stage. We considered the points in the intervalfrom 0 to 601, 300 to 3601, corresponding to a gap of 1201 on theright side of the circle drawn on the points of edges, and 2101 to2401 corresponding to a gap of 1201 on the left side. The pointsoutside these intervals were excluded from the accumulationvector. By performing this procedure, we dismiss the influence ofthe eyelids on the location of the limbus. To find the edge of thelimbus, we used radiuses intervals13 of 15 to 37 pixels, fromwhere we selected the 80 most voted coordinates14 in theaccumulation vector. The coordinates were ordered in relationto voting. Next, the most voted coordinate was put aside, and itwas verified if, among the remaining coordinates, there wereothers with the same number of votes. If there are othercoordinates with a similar number of votes, we then select thosewith smaller radii. After doing so, we proceed with our choice of acircular region. This procedure is applied to left and rightlimbuses.

Having the two main limbus candidates, called, for example,right limbus (Rl) and left limbus (Ll) with radiuses Rr and Lr,respectively, we then check if the difference between Rr and Lr isgreater than 2 pixels, since the limbuses must present approx-imate radiuses. If a difference is detected, we take the smallestlimbus as a reference. By considering the smallest Rr, we searchamong the most voted in the accumulation vector of Ll the firstradius to present the maximum difference of 2 pixels, comparedto Rr. Then, we verify if there are more peaks with radiuses equalto Lr, in which case we select the one that presents closeralignment in relation to the horizontal axis of Rl. In this way,the precise location of both limbuses is ensured.

The HT was initially used for detecting the center and theradius of the limbus in both eyes in the delimited image. Next, HTis applied again in the region previously detected to determinethe center of the reflection. To locate brightness, we used radiusesintervals of 2–4 pixels, and considered all the points of the circlesdrawn in the edges map, and then projected on to the Houghspace (accumulation vector). After this, we selected the 80 mostvoted coordinates15 in the accumulation vector. The coordinateswere ordered according to voting.

Taking as example the location of the brightness of the righteye, we have taken into account, considering the six greater peaksin the accumulation vector, the ones that were closer to the centerof the right limbus, bearing in mind the coordinates ðx,yÞ. So, weguarantee that the location of the limbus does not fall on theregion of the eyelids, when the eyes are partially open i.e., it willnot be disturbed by the reflections caused by corrective lens. Thisprocedure is applied to find the brightness of both the left and theright eye.

13 The radiuses intervals considered in this work were determined through an

analysis done on the image base used in tests.14 This number was chosen after exhaustive tests.15 This number was chosen after some tests.

3.2.4. Detection of strabismus

In order to detect strabismus, we use the location of theluminous reflection of the cornea or the first Purkinje imagegenerated by the Hirschberg test (Section 2.2) along with thelocation of the limbus as parameters to verify the alignment ofboth eyes.

As its main goal, the present study attempts to create an easy,fast and low-cost method for the automatic detection of strabis-mus. In order to make such method accessible to the averageophthalmologists – not necessarily to those whose specialtyincludes strabismus – one needs to presuppose the lack ofviability of any of the methods that require the measurement ofthe Kappa angle16 in each eye, ceratometry17 and/or cerato-scopy18, or even the axial length of each ocular globe, as demon-strated in other studies [16,17]. In this context, one shouldconsider mainly the refraction of each eye when analyzing thephotographs, for this represents basic information that can beeasily gathered by the ophthalmologist. In this way, the evalua-tion of the position of the first Purkinje image in each eye iscarried out as follows:

1.

thro

can

The distance of the center of the reflection to the center of thelimbus in the vertical (VD) and horizontal (HD) directions ismeasured.

2.

It is evaluated, also, the corneal diameter at 1801, and then thisis compared to the other eye diameter.

3.

If there is no difference between the diameters, the VD and HDof the eyes are compared.

4.

If there is a difference between the diameters of the corneas,then the proportion of the yaw of the reflection in the non-fixating eye in relation to the position of the reflection in thefixating eye is calculated based on the diameter differencebetween both corneas by using CPC ¼ rl=RL. Where CPC is thecorneal proportionality constant, rl is the radius measurementof lower limbus area and RL is the radius of the measure of thegreater limbus area.

Thus, it is possible to measure the positioning of the firstPurkinje image, taking into consideration the differences in thesize of the corneas of both eyes. We start by considering thedifferences between refractive errors (anisometropias), exempt-ing the use of contact lenses, when photographing theanisotropic. This happens because anything done to restraindiminishment in the size of the cornea, artificially caused bya corrective lens of elevated spherical equivalent, will bediscounted when calculating the CPC between the eyes for theimage captured under the influence of the dioptrically power ofthe corrective lens in use.

The major problem of evaluating these patients by theproposed method is the anisometropy, for the differences in the sizeof the corneal curvature, or the axial length are not so significant inmost strabric patients. By solving this problem, we can state that theproposed method is applicable to the patients concerned.

As illustrated in Fig. 6, the distances in pixels for each eye arecalculated from the center of the reflection to the center of thelimbus along both vertical and horizontal directions, representedrespectively by distX and distY. To diagnose the alignment, theCPC is multiplied by the distances, distX (distance calculated withrespect to the x-axis) and distY (distance calculated with respectto the y-axis) of the eye with greater limbus, replacing the original

16 The angle formed by the optical axle and the fixating line.17 Computerized exam to measure the curvature of the corneal surface.18 The computerized ceratoscopy or topography of the cornea is the exam

ugh which a qualitative and quantitative analysis of the corneal astigmatism

be done.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146142

distances for these results. Next, the absolute difference of thedistance between the eyes is calculated for both vertical (VDIF)and horizontal (HDIF) directions.

The application of the Hirschberg test presents a differencewhich ranges from 21 to 41 in the visual axis with respect to theanatomic axis. This may cause the false impression of horizontalyaw. Thus, we define cutoff points, or thresholds of up to 1.0 pixelof VDIF and 2.0 pixels of HDIF for a patient considered normal.The threshold of VDIF is smaller than that of HDIF, because thevertical yaws have worse aesthetic effects than those ofhorizontal yaws.

Table 1Results of the classification of the SVM for patients’ images.

Measurements %

TP TN FP FN SE SP PPV NPV AC

Semimadogram 408 99 8 25 94.23 92.53 98.08 79.84 93.89

4. Results and discussion

This section presents and discusses the results obtained by theproposed methodology, which is based on the Hirschberg test forthe detection of strabismus with the aid of digital pictures. InSections 4.1–4.4. we discuss, respectively, the results of the stagesof detection of the region of the eyes, location of the eyes, locationof limbus and brightness and detection of strabismus.

4.1. Detection of the eye region

Using a base of patients’ images, formed by 45 images, weobtained matching rate of 100% in the detection of eye region. InFig. 7b and d we have shown some examples of automaticdetection of eye region in pictures of patients with and withoutglasses.

4.2. Location of the eyes

With the reduction of the search space achieved by theautomatic detection of the region of the eyes, the stage of eyelocation begins (Section 3.2.2).

As shown in Section 3.1.2, the image base is formed by 45photographs. From these photographs, a training base was formedand introduced in Section 3.2.2, followed by the extraction andselection of features, and the training of the SVM. The featuresextraction approach using just one of the geostatistical functions

Fig. 7. Automatic detection of the region of the patients’ eyes. (a) Patient without glasse

of the patient’s eyes (b) detected.

Fig. 6. Calculation of alignment.

generates a set of 208 features; however, when combining thefour functions, it generates a total of 832 features.

After the stage of features selection using stepwise discrimi-nant analysis, one can obtain a statistically significant reductionin the number of features. Of the 208 variables for each geosta-tistics measure, 37 for semimadogram, 29 for semivariogram, 17for correlogram and 30 for covariogram have been selected. Fromthe 832 variables, using all measures, we have selected 59.

Following the flow of the proposed methodology in Section3.2.2, the next step is the stage of classification and validation ofresults. The results obtained by the SVM classifier using theparameters above are listed in Table 1.

The best result is the one with 94.14% sensitivity, 95.38%specificity and a matching of 95.19% for the configuration of allgeostatistical functions. The rates of 98.78% for PPV (Positivepredictive value) and 83.07% for NPV (Negative predictive value)indicate that this approach classified eye regions far moreeffectively than any other regions of the face; this justifies theuse of combined geostatistical functions on the stage of eyelocating.

Fig. 8a presents examples of images for which the methodol-ogy, using combined measurements, succeeded in locating theeyes, having TP of 411. Fig. 8b, on the other hand, shows wherethe methodology has failed by obtaining FP of 5. Analyzing theresults, one may notice that the errors occurred mainly in theregions of eyeglasses frames.

Analyzing the classification of non-eye regions, we can observethat the amount of TN and FN were, respectively, 103 and 21.Fig. 8c shows examples of eye regions which were classified asnon-eye. We have noticed that most of the errors occurred inimages of patients that presented reflections in the lenses of theirglasses, and those with eyes partially open.

s, (b) region of the patient’s eyes (a) detected. (c) Patient wearing glasses, (d) region

Semivariogram 392 90 16 42 90.32 84.96 96.08 68.19 89.26

Correlogram 383 76 29 52 88.05 72.38 92.96 59.38 85.00

Covariogram 369 70 31 70 84.06 69.30 92.25 50.00 81.30

All 411 103 5 21 95.14 95.38 98.78 83.07 95.19

Fig. 8. Location of the eyes. (a) Correct location of the eyes, (b) failure in the

location of the eyes and (c) eye region classified as non-eye.

Fig. 11. Analysis of Fig. 10a. (a) Output image of the accumulation vector after

application of HT to the image of the eye region. (b) Right eye candidate wrongly

classified as non-eye by the SVM.

Fig. 12. Analysis of Fig. 10b. (a) Output image of the accumulation vector after

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146 143

Having the regions of the eyes duly classified, and followingthe methodology cited in Section 3.2.2, we have obtained amatching of 91.11% in the location of both eyes, with theoccurrence of errors in only four images. Fig. 9 presents examplesof images where the location of the eyes was performed correctly.

Fig. 10a–d, on the other hand, presents images where it wasnot possible to locate both eyes correctly.

By analyzing the result shown in Fig. 10a, we have noticed inthe output of the accumulation vector used in the HT asrepresented in Fig. 11a that the methodology did not succeed inlocating the left eye in candidates through HT. Nevertheless, itlocated the right eye (Fig. 11b), but failed on the SVM classifica-tion stage. One needs at least one eye classified by the SMV;otherwise it would not be possible to find the other eye byapplying our methodology proposed.

The second picture to be analyzed is Fig. 10b, from which wecan observe that it is not possible to identify the circular region ofthe eyes through HT, but only the left and right central corner ofthe eyeglasses frame, as illustrated in Fig. 12a–c. However, theclassification of the eye candidates was done correctly, as inFig. 12b and c, which do not represent eyes, these were classifiedas non-eye. Besides, because the images did not show any eye, themethodology ignored the search for new candidates.

Fig. 10c, on the other hand, shows that the right eye waslocated correctly by the methodology which classified Fig. 13correctly. However, there was a failure in the classification of theleft eye candidate as illustrated in Fig. 13c.

The result shown in Fig. 10d is then discussed. Fig. 14apresents the output image of the accumulation vector after theapplication of the HT to the image of the region of the eyes. In thisfigure, we can notice that the HT could not find the circular regionof the eyes. Fig. 14b illustrates the left eye candidate that hadbeen erroneously classified as eye by the SVM. Fig. 10d points outthe failure on the location of the right eye as having been causedby the application of similarity between the right eye candidateand the sample classified by the SVM.

application of HT to the image of the region of the eyes. (b) and (c) left and right

eye candidates, respectively, classified correctly as non-eye by the SVM.

Fig. 13. Analysis of Fig. 10c. (a) Output image of the accumulation vector after the

application of the HT to the image of the region of the eyes. (b) Right eye

candidates correctly classified as eye. (c) Left eye candidates erroneously classified

as eye by the SVM.

4.3. Location of limbus and brightness

The correct diagnosing of strabismus is directly connected tothe result of the location of limbus and brightness, since thisposition is used in the calculation of the alignment (Section 3.2.4).On this stage, after considering the 41 images that went throughthe previous stage, we obtained a matching rate of 95.56% forlocation of limbus on both eyes i.e., the methodology failed in justone of the 41 images. In Fig. 15a and b examples of images arepresented, where the methodology correctly found the region oflimbus.

In Fig. 16, we can see the image where the methodology hasfailed to locate the limbus correctly in both eyes. We have alsonoticed that the error occurred mainly because of the presence ofluminous reflections in the right lens of the glasses, covering, inthis way, the patient’s limbus. Analyzing the visible part of the

Fig. 9. Examples of images where the location of the eyes was performed

correctly.

Fig. 10. Examples of images wher

limbus of the right eye (RE), we can see that its radius is smallerthan that of the limbus in the left eye (LE), with a difference above2 pixels. Thus, considering that the methodology takes thesmallest limbus for reference, in the case of this possibility, itwould be impossible to locate the limbus in the LE correctly.

Considering the location of brightness on the 40 images takenof the correct location of the limbus, we have obtained a matchingrate of 100% for location of brightness on both eyes. In Fig. 17aand b we show some examples of the correct location of bright-ness by using the methodology.

4.4. Detection of strabismus

In this section, we present the results obtained on the detec-tion stage of strabismus. We will consider the 40 patients whoselimbus and brightness were located on the previous stage.

e the methodology has failed.

Fig. 14. Analysis of Fig. 10d. (a) Output image of the accumulation vector after the

application of the HT to the image of the region of the eyes. (b) Left eye candidates

erroneously classified as eye by the SVM.

Fig. 15. Examples of images where the methodology correctly found the region of limbus.

Fig. 16. Image where the methodology failed in locating the limbus.

Fig. 17. Examples of correct location of brightness obtained by applying the

methodology.

Table 2Result obtained from the processing of the 40 patients’ images for verifying the

alignment of the eyes compared to the specialist’s analysis. P—patient, S—spe-

cialist and M—methodology.

P RE LE CPC HDIF VDIF Result

RR distY distX LR distY distX S M

1 34 4 4 34 0 8 1 4 4 Yes Yes

2 26 2 5 28 0.92 1.85 0.93 1.07 3.14 Not Yes

3 33 1.87 1.88 31 1 0 0.93 1.87 0.87 Not Not

4 25 3 1 25 0 3 1 3 2 Yes Yes

5 30 3.73 0 28 3 0 0.93 0.73 0 Not Not

6 34 4 2 34 4 1 1 0 1 Not Not

7 31 2.80 3.74 29 0 4 0.93 2.80 0.25 Yes Yes

8 32 6.78 1.93 31 3 1 0.96 3.78 0.93 Not Yes

9 34 0.97 1.94 33 6 1 0.97 5.02 0.94 Not Yes

10 27 3.85 0 26 3 0 0.96 0.85 0 Not Not

11 28 1.92 0.96 27 1 0 0.96 0.92 0.96 Yes Not

12 34 0 2 34 5 0 1 0 1 Not Not

13 31 3.87 0 30 2 1 0.96 1.87 1 Not Not

14 25 1.92 1.92 24 2 1 0.96 0.08 0.92 Not Not

15 33 5.82 0 32 3 0 0.97 2.82 0 Yes Yes

16 26 0 4 26 2 1 1 2 3 Yes Yes

17 20 3.0 1.0 20 1.0 0 1.0 2.0 1.0 Not Not

18 32 4 1 33 2 0 0.97 2 1 Not Not

19 27 1 1 27 3 0 1 2 1 Not Not

20 21 2 1 21 1 0 1 1 1 Not Not

21 20 2 1 20 3 2 1 1 1 Not Not

22 24 2 1 25 1.92 0.96 0.96 0.08 0.04 Not Not

23 21 1.90 0 20 1 1 0.95 0.90 1 Yes Not

24 18 2 0 18 3 2 1 1 2 Yes Yes

25 23 2 4 23 1 2 1 1 2 Yes Yes

26 20 1 2 20 1 1 1 0 1 Not Not

27 25 2 2 25 1 2 1 1 0 Not Not

28 19 2 0 19 0 0 1 2 0 Not Not

29 21 1 0 21 1 0 1 0 0 Not Not

30 23 1.91 0.95 22 2 3 0.95 0.08 2.04 Not Yes

31 16 2 0 16 8 0 1 6 0 Yes Yes

32 26 1 0 26 1 1 1 0 1 Yes Not

33 22 1 1 23 1.91 0 0.95 0.91 1 Not Not

34 21 2 0 21 2 1 1 0 1 Not Not

35 22 3.82 0 21 5 2 0.95 1.18 2 Yes Yes

36 24 0.95 0 23 0 0 0.96 0.96 0 Yes Not

37 22 1 0 24 1.83 0 0.91 0.83 0 Not Not

38 25 2 1 25 0 1 1 2 0 Yes Not

39 24 2.87 0.95 23 2 1 0.95 0.87 0.04 Not Not

40 19 1 2 20 0 0.95 0.95 1 1.2 Yes Yes

Fig. 18. Images where the application of the Hirschberg test failed to: identify the

normal patient (a and b) and the strabic patient (c and d).

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146144

In Table 2 we present the results achieved by the methodologyalong the processing of the 40 images. Where RR is the rightradius, LR is the left radius, distX is the vertical direction, distY isthe horizontal direction, CPC is the corneal proportionality con-stant, VDIF is the vertical alignment difference and HDIF is thehorizontal alignment difference. All these values are expressed inpixels.

After analyzing Table 2, we obtained the values of TP¼10,FP¼4, TN¼21 and FN¼5. So now we can state that our metho-dology has achieved 67% of sensitivity, 84% of specificity and77.5% of matching for the 40 patients’ images; such that, from thenine errors occurred, seven were caused by the limitations of theHirschberg test, which can only detect aesthetic strabismuses,since this test examines only the anatomical axis of the eye andnot its visual axis.

In Fig. 18a and b examples of images of one of a patient’s leftand right eyes are presented, in which case the Hirschberg testfailed in view of the apparent yaw. In this case, according to thespecialist, the patient did not present strabismus. However,the methodology revealed a yaw, and referred the patient to thestrabic group. From the seven patients missed, just two did notpresent strabismus, according to the specialist. This mismatch ofthe Hirschberg test can be explained by the fact that there is nostrabismus in the presence of an apparent yaw (pseudostrabis-mus), which is caused by pupillary axis angled to each other evenwhen the visual axis has been correctly positioned in relation tothe viewed object [13].

Fig. 18c and d, on the other hand, is examples of images of leftand right eyes of five strabic patients on whom the Hirschbergtest failed to diagnose that condition. In this case, according to thespecialist, the patient presents strabismus. However, the metho-dology could not detect the misalignment, including the patientin the normal group. In these cases, the Hirschberg test is limited tothe fact that there can be strabismus masked by a Kappa angle of

opposite signal, annulling the appearance of the yaw and giving thenotion of an adequate binocular position, in spite of a yaw [13].

Analyzing Table 2, without considering the patient on whomthe Hirschberg test failed, we have obtained the values of TP¼10,

Fig. 19. Images where the methodology failed in determining the precise location

of the limbus. (a) and (b) RE and LE of patient 2. (c) and (d) RE and LE of patient 30.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146 145

FP¼2, TN¼21 and FN¼0. Thus, we can verify that the methodol-ogy achieved 100% of sensitivity, 91.3% of specificity and 94% ofmatching for the 33 remaining images. The two patientsconsidered strabic, even if they were not strabic, have beenconsidered as such because of the precision error that occurredwhen locating the limbus.

Analyzing Fig. 19a, which presents the right eye of patient 2 fromTable 2, we have noticed that the region of the limbus was notprecisely located, because the center of the limbus was to be closerto the center of the pupil. This resulted in the vertical misalignmentof the right eye in relation to the left eye (Fig. 19b), making,therefore, the result of VDIF, of 3.14, indicating the presence ofstrabismus, which contradicts the specialist’s diagnosis.

In Fig. 19d, representing the left eye of patient 30 from Table 2,one can see that, similar to the last error, the region of the limbuswas not correctly located, leaving a small remainder of the limbusoutside the region located. This caused an increase in the verticalmisalignment of the left eye in relation to the right eye (Fig. 19c),making the result of VDIF, of 2.04, reveal the presence ofstrabismus, contradicting, in this way, the specialist’s diagnosis.

5. Conclusion

This work recommends the use of image processing techni-ques, geostatistical functions and support vector machines for theautomatic detection of strabismus on digital images by using theHirschberg method. Along with this study, some other contribu-tions can be verified. The first one may be seen to occur on the eyelocation stage, where an innovating combination of techniques tolocate the eyes on human faces is proposed, using homomorphicfiltering, the Hough Transform, geostatistical functions, stepwisediscriminant analysis, SVM and EMA similarity measurements.

Other contributions have to do with the features extraction stage,where geostatistical functions are used to extract texture informationfrom the eyes, allowing then the discrimination of eye regions fromthe other regions with a fair amount of precision. The third and maincontribution is the creation of a methodology that gives support tothe automatic identification of strabismus on digital images.

On the stage of automatic detection of eye region by usingprojections, the gradient magnitude exhibited a matching rate of100% for the patients’ images. On this stage, we concluded thatthe use of homomorphic filtering contributed to the overallmatching, since the technique of illumination adjustment madeit more uniform on the image.

On the second stage, the methodology obtained a matchingrate of 95.19%, with the combination of four geostatistical func-tions, such as texture descriptors in the extraction of features.Nevertheless, although such results may be seen as very promis-ing, it is still necessary to increase the diverseness of the samplefaces, so that a more robust and generic methodology can bedeveloped. However, we can conclude that the obtained resultsdo give the matter its due importance regarding the newapproaches based on geostatistical functions for describing thetexture of eye regions on digital pictures of faces. Still, on the

second stage, our methodology obtained 91.11% of matching forthe location of both eyes.

The location of the limbus and brightness, a stage on which weused Canny’s method and Hough’s transform, the achieved matchingrates were, respectively, 97.5% and 100%. We can conclude that theerrors that occurred in locating the limbus were mainly due to theluminous reflections generated during image capturing.

The result for the identification of strabismus achieved matchingrate capacity of 77.5% for the 40 images that passed on to the stage oflocation of limbus and brightness. This result had a direct influenceover the seven images for which the Hirschberg test was not used,due to its limitations in doing an aesthetic evaluation of strabismus.Nevertheless, disregarding the images for which the Hirschberg testwas not effective, it was possible to achieve a matching rate of 94%.

Even showing great potential as to its application in helpingthe specialist to diagnose strabismus, our methodology stillrequires the use of other techniques, since the Hirschberg test isless precise when compared to other methods, such as theKrimsky and Cover Test.

The number of patients’ images (30 healthy patients and 15strabic patients) and the disproportion (more healthy patients thanstrabic patients) does not allow us to carry out more preciseanalyses to ascertain the total efficiency of the proposed method.Thus, it is necessary to deepen our present analysis with larger andmore balanced image base of patients. Also, it is important to useother images, captured under different acquisition protocols in orderto better evaluate the behavior of the present methodology.

Conflict of interest statement

None declared.

References

[1] G.V. Noorden, E. Campos, Binocular Vision and Ocular Motility: Theory andManagement of Strabismus, Mosby Inc, 2001.

[2] J.P. Diaz, C.S. Dias, Strabismus, Butterworth Heinemann, Woburn, Massachu-setts, EUA, 2000.

[3] A.S. Jolson, H.R. Myler, A. Weeks, Apparatus for evaluating eye alignment, USPatent 5,094,521, 1992.

[4] E.M. Helveston, F.H. Orge, R. Naranjo, L. Hernandez, Telemedicine: strabismuse-consultation, J. Am. Assoc. Pediatr. Ophthalmol. Strabismus 5 (5) (2001)291–296.

[5] M.W. Quick, R.G. Boothe, A photographic technique for measuring horizontaland vertical eye alignment throughout the field of gaze, Invest. Ophthalmol.Vis. Sci. 33 (1) (1992) 234.

[6] I. Subharngkasen, Successful amblyopia therapy by using synoptophore,J. Med. Assoc. Thailand¼Chotmaihet thangphaet 86 (2003) S556.

[7] H.J. Kim, W. Kim, Eye detection in facial images using zernike moments withSVM, ETRI J. 30 (2) (2008) 335–337.

[8] J. de Almeida, A. Silva, A. Paiva, Automatic eye detection using semivariogramfunction and support vector machine, in: Seventeenth International Conferenceon Systems Signals and Image Processing IWSSIP 2010, 2010, pp. 174–177.

[9] O.S.S. Souza Jr., A.C. Silva, Z. Abdelouah, Personal identification based on iristexture analysis using semivariogram and correlogram functions, Int. J.Comput. Vis. Biomech. 2 (1) (2009) 121–129.

[10] A. Silva, P. Carvalho, M. Gattass, Analysis of spatial variability usinggeostatistical functions for diagnosis of lung nodule in computerized tomo-graphy images, Pattern Anal. Appl. 7 (3) (2004) 227–234.

[11] J.D. Sousa de Almeida, Metodologia Computacional para Deteco Automtica deEstrabismo em Imagens Digitais atravs do Teste de Hirschberg., http://www.tedebc.ufma.br//tde_busca/arquivo.php?codArquivo=430, 2009.

[12] L.C. Junqueira, J. Carneiro, Histologia basica. 8 Edic- ~ao, Guanabara Koogan.[13] H. Bicas, Estrabismos: da teoria �a pratica, dos conceitos �as suas operaciona-

lizac- oes, Arq. Bras. Oftalmol. 72 (5) (2009) 585–615.[14] J.L. Mims, R.C. Wood, Proportional (fractional) displacement of the Hirsch-

berg corneal light reflection (test): a new easily memorized aid for strabo-metry; photogrammetric standardization (calibration), Binocul. Vis.Strabismus Q. 17 (3) (2002) 192.

[15] K. Wright, P. Spiegel, L. Thompson, Handbook of Pediatric Strabismus andAmblyopia, Springer Verlag, 2006.

[16] P.E. Romano, Individual case photogrammetric calibration of the HirschbergRatio (HR) for corneal light reflection test strabometry, Binocul. Vis. Strabis-mus Q. 21 (1) (2006) 45.

J. Dallyson Sousa de Almeida et al. / Computers in Biology and Medicine 42 (2012) 135–146146

[17] S. Hasebe, H. Ohktsuki, R. Kono, Y. Nakahira, Biometric confirmation of theHirschberg ratio in strabismic children, Invest. Ophthalmol. Vis. Sci. 39 (13)(1998) 2782.

[18] K. Peng, L. Chen, S. Ruan, G. Kukharev, A robust algorithm for eye detectionon gray intensity face without spectacles, J. Comput. Sci. Technol. 5 (3) (2005)127–132.

[19] R.d. Melo, E.d. A. Vieira, A. Conci, in: A. Karras, S. Voliotos, M. Rangouse, A.Kokkosis (Eds.), A System to Enhance Details on Partially Shadowed Images,2005, 309–312.

[20] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Prentice-Hall, NewJersey, 2002.

[21] E.H. Isaacs, R.M. Srivastava, Applied geostatistics, 1990.[22] P.A. Lachenbruch, M. Goldstein, Discriminant analysis, Biometrics 35 (1)

(1979) 69–85.[23] C. Chang, C. Lin, LIBSVM—A Library for Support Vector Machines, available at

http://www.csie.ntu.edu.tw/�cjlin/libsvm/, 2001.[24] T. D’Orazio, M. Leo, C. Guaragnella, A. Distante, A visual approach for driver

inattention detection, Pattern Recogn. 40 (8) (2007) 2341–2355.

Anselmo Cardoso de Paiva received BSc in civil engineering from Maranh~ao StateUniversity—Brazil in 1990, a MSc in civil engineering-Structures and a PhD inInformatics from Pontiphical Catholic University of Rio de Janeiro—Brazil in 1993and 2002. He is currently a Professor at the Informatics department, FederalUniversity of Maranh~ao—Brazil. His current interests include medical imageprocessing, geographical information systems and scientific visualization.

Aristofanes Correa Silva received a PhD degree in Informatics from PontiphicalCatholic University of Rio de Janeiro—Brazil in 2004. Currently he is a Professor atthe Federal University of Maranh~ao (UFMA), Brazil. He teaches image processing,pattern recognition and programming language. His research interests includeimage processing, image understanding, medical image processing, machinevision, artificial intelligence, pattern recognition and, machine learning.

Jo ~ao Dallyson Sousa de Almeida received BSc in computer science and a MSc inElectric Engineering at the Federal University of Maranh ~ao (UFMA), Brazil in 2010.His major interest nowadays is obtaining a PhD degree. Currently he is a SystemsAnalyst at the UFMA. His research interests include signals and image processing,pattern recognition, machine learning and automation systems.

Jorge Antonio Meireles Teixeira received a BSc in Medicine from the FederalUniversity of Maranh~ao (UFMA), Brazil, in 2004 and PhD in Medicine (Ophthal-mology) at the Federal University of S~ao Paulo—Paulista Medical School (2004).Currently he is a Professor of Medicine at and head of the Course of Medicine ofUFMA. He has experience in Medicine with emphasis on ophthalmology, acting onthe following topics: strabismus and pediatric ophthalmologist.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.