Real-timeEyeGazeTracking

Alireza RahimpourFall2010

1

Eyetrackingsystem:Nowadayseyegazetrackinghaswiderangeofapplicationsinhumancomputerinteraction.Oneoftheseapplicationsisusingtrajectoryofeyegazeinsteadoffootorhandfordisabledpeopletoexecutesomecommands.

OtherApplications:ü Automaticmonitoringofdriversforaccidentavoidanceü Commercialusesü Psychologicaluses

2

DifferenttypesofEyetrackers

1-UsingEOGsignal:• Systemsbasedonelectrooculographyuseelectrodesplacedonthe

facetodetectthemovementsoftheeyesIntrusive

2-Usingactiveinfraredilluminationv Difficultcalibrationv Needhighresolutionandexpensivecamerasv Interference:ExternalsourcesofIR:Sun

3

Other types:

4

GOAL

Designinganeyetrackingsystemashumancomputerinterfacewhichavoids:

ü specializedhardwareü infraredlightsourcesü expensivecameras

andü Increasestheaccuracyandspeedofthesystemtobeusableinreal-

timeapplications.

5

Eyetrackingsystem

Recently,variousmethodshavebeenproposed,someofthesemethodscansuccessfullytracktheeyegaze.However,theyalwaysrequirespecificcircumstances,trainingorarenotcapableofreal-timeperformance.

Wewanttotracktheeyegazeinreal-timebyusingasimpleandlowcostwebcammountedonordinarylaptops.

Itshouldbeableto:controlthemotionsofmousecursorandclickonanonscreenkeyboardinrealtime.

6

Implementation

Severalmethodscanbeusedforfacialandeyefeatureextraction.

Oneofthemis:Templatematching

TemplateofFeature• Theregionofimagewithmaximumsimilaritywiththetemplate,

referstopositionofourfeatureinimage.

7

Templatematching• SimilaritybetweenimageandTemplate??• Minimizingtheerror:

• Parametrictemplates

ü Correlationfunction8

|),(),(|),(1

0

1

0∑∑−

=

−

=

++−≡B

j

B

it jviuIjiTvuSAD

Correlationfunctionasameasureofsimilarity

9

∁ 𝑢, 𝑣 =

Fortemplatematching,thetemplatetslidesoverf(image)andCiscalculatedforeachcoordinate.Then,thepointwhichexhibitsmaximumC isreferredtoasthematchpoint.

Templatespecifications

• Size

• Thesize ofthetemplateisnottheonlyissue,butmoreimportantly,trackingperformancedependsonthe“complexity”ofthetemplate.

• Complexity:haveenough brightnessvariations,e.g.,textureorlines,toberecognizedas distinctfeatures

10

Templates

• Templateswithminsizewhichcontainenoughdetailsarechosen.

• Facetemplate:• 106x74pix

• Eyetemplate:• 15x25Pix

• Pupiltemplate:

• Eyecorners:

11

Reducingthesearchwindowsize

ü Reducethesearchareaineachstepbycropping.

ü Usingstructureoffacetodeterminethesearchregionforeachfeature.

12

240X320

106X75

45X36

15X25

Accuracy• Usingweightedcorrelationcoefficientcanincreasetheaccuracy.

• Consideringseveralframesofimages,themaximumindexincorrelationcoefficientmatrixcanbedetermined.

• Forthenextframes,weconsidermoreweightforthewinnerindex(andafewneighbors).

13

Trackingsystem

14

Thesystemconsistsoftwomainmodules:

1)thefacedetectorandtracker

2)theeyeanalysismodule

Theoutputfromtheeyemodulecanbe theinputtoasimplecomputercontrolinterfacebysimulatingakeypress.

Also:Differentmodesaredefinedcorrespondingtoeyeclosuretime.

EYEGAZEESTIMATION

15

• Horizontaldirection1-usingscleraregionchanges2-usingcenterofpupil

• VerticalDirection1-usingeyecorners2-usinguppereyelids

Eyegazeinhorizontaldirectionusingscleraregion

16

Dividingthemonitorinhorizontaldirectionto16pointsanddefinealphaasfollow:

( )( ) ( )RALA

LA+

=α

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

02468101214161820222426283032

alpha

position

ScleraRegionextraction

17

Croppedeyeimage:

Contrastenhancement:

Converttobinaryimageusingathreshold:

Morphological operations (opening – closing):

Makingcontoursandcalculatethenumberofpixelsinthemandfinding thecoordinates.

Eyegazedetection

18

• Horizontaldirection2-Usingpupilcentercoordinatewithrespecttoeyecorners

• Verticaldirection:1-eyelids:

2-eyecorners

Experimentandresults

19

ü Framesize240x320pix-30framepersec

ü Distanceofusertomonitor(17”):60cm

ü Dataset:realtimevideotakenfromregularwebcamonlaptops.ü Nospecificilluminationcondition

ü Testedontwoonscreenkeyboard:ü 3x7and5x5keys.

Result(facetrackermodule)

20

Robustnessto:differentusersandbackgroundchangesandheadtilt

Fastmoving ofuserinfrontofthecamera

Results:

Eyegazedetection(withmethod2)Using3x7onscreenkeyboard:

21

DEMO

Detectionrate

Faceillumination

Keyboard VerticalPre.

Horizontalprecision

Method

76.6% sensitive 5x5 4deg. 6.8deg. Method1

90% Notsensitive

3x7 2.8deg 11.3deg. Method2

22

Results:

Method1:usingscleraregion forhorizontaltrackingandeyecornersforvertical.

Method2:usingpupil centerforhorizontalandeyecornersforverticaltracking.

Eyetrackingsystem:

ü Realtimeü Notexpensiveü Moreaccuratecomparedtosimilarsystemsü Noneedtotrainingandoverwhelming calibration

Any Questions?

23