Towards a stochastic theory of saccadic eye movement

GURU Oct 1, 2001

Jonathan Nelson

Thanks especially to Wendy Ark Gary Cottrel Carrie Joyce Javier Movellan Marty Sereno Ruth Williams

Outline Function of eye movement in vision Traditional analyses of eye

movement Three papers using stochastic

processes to model eye movement Research towards a better

generative model

Why model eye movement? for cognitive science and economics: how do a person’s

knowledge and goals influence their information-gathering in perception? (applications also to entertainment and interface design)

for medical diagnosis: how does a physician/technician read an x-ray? Can we automate or assist the physician?

for understanding and treating pathological conditions: autism, williams’ syndrome. Can training appropriate eye movement help autistics in social situations?

for artificial perception: are the visual data that humans acquire also informative for active, sequential artificial vision systems?

Saccadic eye movement Eccentricity and image resolution Saccades and fixations Relationship to task, applications

Traditional analyses Large number of papers Inspired by behavioral statistics:

lose dynamic character, or dichotomize/polychotomize time

Yet unexplained qualitative results: “highly individual scanpaths” “high individual consistency from trial

to trial”

Towards a stochastic theory Regions of interest in an image Timescale

Regions of interest:

Regions of interest:

Regions of interest:

Regions of interest:

Timescale Discrete or continuous? Every point sampled by the eye

tracker? Every nth millisecond? Every fixation?

Timescale Nearby fixations grouped together Saccade points rejected

What kind of SP? Discrete state, usually with regions

of interest that contain current point of gaze as state

Discrete time, with points of meaningful fixations forming the time step

Time-homogenous Markov chain

White et al.: mammograms How do experts read

mammograms? Can experts’ patterns in reading

mammograms suggest means of computer aided diagnosis?

from http://sprojects.mmi.mcgill.ca/mammography/imageanalysis.htm

White et al.’s modeling One model per subject, per image A sequential clustering algorithm

to determine regions of interest Exclude transitions to same state,

and fixations outside of regions of interest

“Not enough data”

Joyce: face perception Goal: augment or corroborate

eyewitness testimony, using eye movement EEG GSR

Task: view a face recognize it as not novel, after a

delay

Task: “Study this face”

Task:

“Have you seenthis face before?”

Joyce’s regions of interest 10 ROI:

hair right eye nose etc.

Normalized faces Nearest neighbor

Joyce’s model Markov States: each of 10 regions of

interest Time: from entering region till

leaving region is one discrete step

Joyce’s results More entropy in saccade

sequences when viewing a novel face

Matches qualitative findings

Eye-typing: Salvucci Goal: have users type a pre-

selected word, by looking at an on-screen keyboard

Models used: saccade or fixation HMM letter HMM grammar

Saccade-fixation HMM If previous time is fixation, current time is most likely

fixation If low velocity, current time most likely fixation “Standard HMM parameter estimation” (Rabiner, 1989) Viterbi to optimize

Fixations and centroids

“Centroid submodel HMM” States:

peaked bivariate distribution for (x, y)

diffuse bivariate distribution for (x, y)

Grammar for words Which word gives the highest

likelihood of the centroid data?

Results 92% accuracy with 1000 word vocabulary too computationally intensive for realistic-size

(e.g. 50,000 words) vocabulary, for next few years

… a nice proof of concept (previous systems required >750ms between

fixations, and 4 degrees between targets) (for simpler multiple-letter models, use a

string-edit distance algorithm to find the nearest vocabulary word)

Future work More temporal dependence: try

3rd+ order model Can a sophisticated string-edit

distance algorithm correct for bias in limited sampling?

Future work Systematic exploration of time index Improvement of saccade-fixation HMM Distribution of fixations within regions What is the nature of individual

differences? Relationship of knowledge and goals to

task

Sources Cited White, KP; Hutson, TL; Hutchinson, TE (1997). Modeling

human eye behavior during mammographic scanning: preliminary results. IEEE Transactions on Systems, Man and Cybernetics, A, 27, 494-505.

Salvucci, DD (1999). Inferring intent in eye-based interfaces: tracing eye movements with process models. Proceedings of the 1999 Computer Human Interaction Conference.

Joyce, CA (2000). Saving Faces: Using Eye Movement, ERP, and SCR Measures of Face Processing and Recognition to Investigate Eyewitness Identification. Ph.D. dissertation, University of California, San Diego.