Viewing task influences eye movement controlduring active scene perception

Department of Psychology, Queen’s University,Kingston, ON, CanadaMonica S. Castelhano

Vanderbilt University, Nashville, TN, USAMichael L. Mack

University of Edinburgh, Edinburgh, UKJohn M. Henderson

Expanding on the seminal work of G. Buswell (1935) and I. A. Yarbus (1967), we investigated how task instructioninfluences specific parameters of eye movement control. In the present study, 20 participants viewed color photographs ofnatural scenes under two instruction sets: visual search and memorization. Results showed that task influenced a numberof eye movement measures including the number of fixations and gaze duration on specific objects. Additional analysesrevealed that the areas fixated were qualitatively different between the two tasks. However, other measures such asaverage saccade amplitude and individual fixation durations remained constant across the viewing of the scene and acrosstasks. The present study demonstrates that viewing task biases the selection of scene regions and aggregate measures offixation time on those regions but does not influence other measures, such as the duration of individual fixations.

Keywords: eye movements, scene perception, task instruction, cognitive control, gaze control

Citation: Castelhano, M. S., Mack, M. L., & Henderson, J. M. (2009). Viewing task influences eye movement control duringactive scene perception. Journal of Vision, 9(3):6, 1–15, http://journalofvision.org/9/3/6/, doi:10.1167/9.3.6.

Introduction

Human vision is an active, dynamic process in which theviewer seeks out specific visual input as needed to supportongoing cognitive and behavioral activity (Findlay &Gilchrist, 2001; Henderson, 2003, 2007). A critical aspectof active vision is directing the eyes to task-relevantstimuli in the environment. Current theoretical treatmentsof human gaze control in active vision focus on twosources of input to the control system: stimulus-basedsources and cognitive sources. Cognitive control, thoughhighlighted in two classic eye movement studies (Buswell,1935; Yarbus, 1967) and a dominant theoretical drivingforce in experimental psychological studies in the 1960sand 1970s (e.g., Antes, 1974; Friedman, 1979; Loftus &Mackworth, 1978; Mackworth & Morandi, 1967), hastaken a less central role in recent theorizing about thecontrol of eye movements in scene perception (Itti &Koch, 2001; Koch & Ullman, 1985; Parkhurst, Law, &Niebur, 2002; Parkhurst & Niebur, 2003). In the presentstudy, we were concerned with a particular aspect ofcognitive control: the influence of viewing task on thecontrol of eye movements during scene viewing.Buswell (1935) and Yarbus (1967) both examined the

influence of viewing task on eye movements duringcomplex picture viewing. These studies began with theobservation that eye fixations are not randomly distributedin a scene but instead tend to cluster on some regions atthe expense of others. For example, in Chapter 6, Buswell

(1935) asked a group of participants to first look at apicture of a tower under free viewing instructions and thenlook to find a person in one of the windows in the tower.He noted that the fixation distribution changed dramati-cally (as would be expected): participants fixated forlonger and more often when examining the tower with thesearch instructions than with the free viewing instructions.He also asked them to look once at a scene with noparticular instructions and then a second time, afterreading a description of the picture. He found that thenumber of fixations made after the description was readincreased substantially (from 61 to 108 fixations). Hesurmised that these changes in fixation number acrossboth experiments were due to the instructions (ordescriptions) arousing the interest of participants tocertain parts of the picture. He referred to these effectsas changing the “mental set” of participants as theyviewed the pictures.In a classic comparison that has been widely described,

Yarbus (1967) asked a participant to look at the painting,“The Unexpected Visitor” by I.E. Repin under sevendifferent viewing instructions. Yarbus (1967, p. 174)showed that eye movement patterns differed dramaticallywith task instructions. For example, fixations were morelikely to land on the faces of the people in the image whenthe instructions were to estimate the age of the people inthe painting rather than estimate the material circum-stances of the family.Both the Buswell and Yarbus studies strongly suggest

that task affects eye movement behavior during scene

Journal of Vision (2009) 9(3):6, 1–15 http://journalofvision.org/9/3/6/ 1

doi: 10 .1167 /9 .3 .6 Received March 6, 2008; published March 13, 2009 ISSN 1534-7362 * ARVO

viewing. However, in both of these studies, viewing taskwas not manipulated in a controlled manner. For example,in both studies, task instruction was confounded withviewing order. In addition, the reported data were sparseand qualitative: both Buswell and Yarbus simply pre-sented the scan patterns generated for each task and bothdescribed the eye movement pattern according to whichparts of the picture seemed to have a greater cluster offixations. Buswell reported the number of fixations over-all, but these were confounded with the fact thatparticipants were allowed to view the picture for as longas they pleased, with no recording of those times.Furthermore, although it is clear from these studies thatcognitive processes influence viewing patterns, it isunclear which specific eye movement parameters areaffected by task. More fine-grained measures of eyemovement behavior were not presented. We are aware ofno published study that has attempted to directly examineeye movements as a function of viewing task in a fullycontrolled experimental design.The second goal of the present study was to examine the

nature of fixation duration control during scene viewing.In comparison to the reading literature, few researchstudies have investigated the control of fixation durationsduring scene perception (Castelhano & Rayner, 2008;Henderson, 2003; Henderson & Hollingworth, 1998;Rayner, 1998). The reason for this is quite simple: acrossmost studies, the task in reading is well understood to beto comprehend the text. Therefore, models and theories ofeye movements of reading have been focused on under-lying cognitive processes and have been able to accountfor such measures as fixation placement and fixationduration with great success (Engbert, Longtin, & Kliegl,2002; Engbert, Nuthmann, Richter, & Kliegl, 2005; Kliegl& Engbert, 2003; Reichle, Pollatsek, Fisher, & Rayner,1998; Reichle, Pollatsek, & Rayner, 2006; Reichle,Rayner, & Pollatsek, 2003). In contrast, with sceneperception the task is often varied and sometimes vague(e.g., free viewing instructions or viewing for a preferencerating). As a result, it is more difficult to pinpoint theunderlying processes affecting fixation durations (but seeHenderson & Pierce, 2008; Henderson & Smith, in press;van Diepen & d’Ydewalle, 2003).A number of studies on scene perception and eye

movement control have been concerned with the nature ofimage properties that are related to fixation location(Henderson, Brockmole, Castelhano, & Mack, 2007;Krieger, Rentschler, Hauske, Schill, & Zetzsche, 2000;Mannan, Ruddock, & Wooding, 1996, 1997; Parkhurstet al., 2002; Parkhurst & Niebur, 2003; Reinagel & Zador,1999; Tatler, Baddeley, & Gilchrist, 2005), and computa-tional models that have reflected this focus on theplacement of fixations (Itti & Koch, 2001; Torralba,Oliva, Castelhano, & Henderson, 2006). Though fixationposition is an important component of eye movementbehavior, an equally important component is the amountof time the eyes remain fixated in a particular scene

region. Models of eye movement control that focus onlocation but ignore duration lead to an incomplete accountand can even lead to misleading conclusions about thenature of eye movement control (Henderson, 2003). Theaverage fixation durations across the entire scene as well asthe summed durations of consecutive fixations on a specificregion of interest are related to the ongoing perceptual andcognitive activity (Castelhano & Henderson, 2008b;Henderson & Ferreira, 2004; Henderson, Weeks, &Hollingworth, 1999; Rayner, 1998).Intimately related to control of fixation placement and

duration is the question of cognitive control over saccadeamplitudes across the entire scene. Reviews of naturalscene studies report that the average saccade amplitude isbetween 4- and 5- on scenes that are on average 20-–30-wide (Antes, 1974; Henderson & Hollingworth, 1998;Rayner, 1998). It is clear from examination of Buswell’s(1935) and Yarbus’ (1967) studies that saccade amplitudeis also affected by task instruction and so naturally wouldbe subject to some degree of cognitive control. However,because saccade amplitude (and by proxy, fixation place-ment) is governed by information not being directlyfixated, it is unclear to what extent stimulus factors versuscognitive processes are influencing saccade patterns.Generally, the degree to which fixation duration andplacement is influenced by viewing task in sceneperception is largely unexplored (but see Henderson et al.,1999).A related issue concerns the manner in which fixation

durations change over the course of viewing a scene. Thisissue has been investigated in a number of studies but hasled to inconsistent results (Antes, 1974; De Graef,Christiaens, & d’Ydewalle, 1990; Friedman & Liebelt,1981; Henderson et al., 1999; Unema, Pannasch, Joos, &Velichkovsky, 2005). Antes (1974) had participants view10 different paintings for 20 s each under the instructionsthat they were to examine each one and choose theirpreferred painting at the end. With regard to fixationdurations across the course of viewing a single painting,Antes found that the early fixations on the paintingstended to be of shorter duration (È215 ms) than those inthe later part of the viewing (È310 ms) across allpaintings. Antes related this trend to the participants’initial scanning of the painting with further scrutiny lateron. However, he also noted that early fixations tended tofall on more informative areas initially and later fixationson less informative ones. It is not clear that the change infixations monitored over the course of viewing was due toa change in viewing strategy (i.e., the same task ofdeciding preference in mind), or reflected differences inthe task itself (i.e., the participant is initially scanning thepicture, followed by a bored perusal of the less interestingareas while waiting for the viewing period to end).Although some studies have reported this increase offixation durations over the initial viewing of a scene(Antes, 1974; Unema et al., 2005), other studies havefound the opposite pattern or no difference in fixation

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 2

durations over the course of viewing the scene (De Graefet al., 1990). Differences across stimuli and task instructionsacross studies make comparison of these results difficult.It is clear from the studies reviewed above that task has

a great influence on eye movements in scene viewing(Buswell, 1935; Henderson et al., 1999; Yarbus, 1967)just as it does in reading (e.g., reading vs. skimming,Masson, 1983). However, in scene viewing, it is unclearwhich specific eye movement parameters are affected bytask. In the present study, we investigate the nature offixation durations during scene viewing, how durationsmay differ given the task instructions, and how fixationdurations fluctuate across the period spent viewing thescene. We compared the eye movement patterns ofparticipants while they performed one of two sceneviewing tasks: visual search and memorization.

Methods

Participants

Twenty members of the Michigan State Universityundergraduate participant pool participated in this experi-ment. All had normal, uncorrected vision and receivedcourse credit or /7 for participation.

Stimuli

Thirty-five photographs of complex real-world indoorand outdoor scenes were digitized for the experiment.Thirty of the scenes were the critical stimuli used in theanalysis, and five were used as target-present fillers in thesearch condition. The scenes were displayed at a reso-lution of 800 � 600 pixels � 32,768 colors and subtended15- of visual angle horizontally and 12- vertically at theviewing distance of 1.13 m. In the visual search condition,a word naming the search target was presented prior to thescene for that trial. The word was printed in a Times NewRoman font in black against a gray background andsubtended 4- by 1- on average.

Apparatus

The stimuli were displayed on an NEC Multisync XE15-inch monitor driven by a Hercules Dynamite Pro supervideo graphics adapter (SVGA) card with a refresh rate of100 Hz. Eye movements were monitored using a Gen-eration 5.5 Stanford Research Institute Dual PurkinjeImage Eye Tracker (Crane, 1994), which has a resolutionof 1 min of arc and a linear output over the range of thevisual display used. A bite-bar and forehead rest main-tained the participant’s viewing position and distance. The

right eye was tracked, though viewing was binocular.Signals were sampled from the eye tracker using thepolling mode of the Data Translations DT2803 analog-to-digital converter, producing a sampling rate slightlygreater than 1000 Hz. The display system and eye trackerwere interfaced with a computer running a 90-MHzPentium processor. The computer controlled the experi-ment and kept a complete record of eye position and timevalues over the course of each trial.

Procedure

Participants first read a description of the experimentalong with a set of instructions. The instructions indicatedthat participants were taking part in two experiments, oneinvolving memorizing scenes, and the other involvingsearching for particular objects in scenes. Participantswere told that their eye movements would be monitoredduring these tasks. Task was blocked and the order of taskblock was counterbalanced across participants. Eachviewing task was explained separately just before it wasto be performed, allowing each participant to receive ashort break between the two task blocks.For the memorization task, participants were instructed

to view each scene in preparation for a later memory testthat would be administered at the end of the session. Theywere told that the memory test would examine memoryfor specific objects in the scenes. Following the instruc-tions, the eye tracker was calibrated and three practicetrials were given, after which the eye tracker wasrecalibrated and the participant viewed 15 scenes for 10 seach. For the search task, participants were instructed tolocate within each scene the target object specified for thattrial. They were further instructed that when they hadfound the target to continue to look at the target and pressthe response button. Prior to each trial, a word naming thetarget for that trial was presented in the center of thedisplay for 2 s. The scene was then displayed for 10 s oruntil the participant pressed the response button indicatingthat the target was or was not present. To encourageparticipants to search the critical scenes exhaustively, thesearch target was never present in the 15 critical scenes.The search target was however present in the five fillerscenes.In both conditions, eye tracker calibration consisted of

having the participant fixate 4 calibration markers at thetop, bottom, left, and right sides of the display area.Calibration was checked by displaying a calibrationscreen consisting of 5 test positions (center, top, bottom,and left and right sides) and a fixation marker thatindicated the computer’s estimate of the current fixationposition. The participant fixated the test positions, and ifthe fixation marker was approximately within T8 min arcof each (this is equivalent to 8 pixels, which was definedon the screen as a box surrounding the fixation marker),calibration was considered accurate.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 3

A trial consisted of the following events. First, thecalibration screen was shown and calibration waschecked. The eye tracker was recalibrated whenevercalibration was deemed inaccurate. Following the calibra-tion check, the participant fixated the center fixation crosson a gray background to indicate that he or she was readyfor the trial to begin. The experimenter then started thetrial: The fixation display was replaced by the trial scenein the memorization condition and by the search targetword for 2 s followed by the search scene in the searchcondition. The scene remained visible for 10 s in bothconditions unless the participant pressed the responsebutton in the search condition. Following scene offset, thecalibration screen reappeared.Each participant saw all 30 critical scenes plus 5 search

filler scenes in a within-subject design. The 30 criticalscenes were divided into two groups and the assignmentof scene group to task condition was counterbalancedacross participants so that each scene appeared in eachtask condition an equal number of times, but eachparticipant saw each scene only once. The order of scenepresentation in each task block was determined randomlyfor each participant. The entire experiment lasted approx-imately 45 minutes.

Results

Although participants were presented with 35 scenes intotal, only the 30 critical scenes were included in the analysis.The visual search scenes analyzed did not have a targetobject in order to maximize the amount of time participantsspent viewing the scene and to better approximate the timespent viewing the scene in the memorization task.

Task performance

Participants reported approaching the two blocks asseparate tasks. Memory performance on a later difficultobject memory test in the memorization condition wasover 80% correct. The memory test was a two-alternativeforced-choice (2AFC) in which participants had todiscriminate between a single object cropped from apreviously viewed scene (in either of the tasks) and adistractor matched at its basic-level category, making ita relatively difficult memory test. A report focusing onthe performance data for the memorization and searchtasks can be found in Experiment 1 of Castelhano andHenderson (2005). Total scene viewing time was 8.7 s onaverage in the search condition, and 69% of the searchscenes were viewed for the entire 10 s maximum.Therefore, the Search task was on average 1.3 s shorterthan the Memorization task. However, this difference didnot have any consequences for the measurements of

interest, as shown below. Therefore, the primary eyemovement analyses were based on all the data.

Eye movement data analysis

Our primary concern in the present study was the natureof eye movement behavior as a function of viewing task.Raw eye movement data files consisted of time and positionvalues. Saccades were defined as changes in eye positiongreater than 8 pixels (about 8.8 arcmins) in 15 ms or less.Manual inspection of the raw data files confirmed that thiscriterion effectively eliminated saccades while preservingslow drifts. Once saccades had been identified, fixationpositions and durations were computed over the remainingdata. The duration of a fixation was the elapsed timebetween two consecutive saccades. During a fixation, theeyes often drift. The scored position for a given fixation wasthe Euclidean average of the position samples (in pixelvalues) taken during that fixation weighted by the durationsof each of those position samples. Thus, if the eye positiondrifted over two pixels before coming to rest on a third, thescored location of that fixation was the average value ofthe three pixel positions weighted by the amount of timethe eyes were directed at each of the three pixels. Theduration of that fixation was then taken as the sum of timeon those three pixels (see Henderson, McClure, Pierce, &Schrock, 1997). Following this procedure, fixation dura-tions less than 90 ms were removed to eliminate fixationartifacts resulting from signal overshoot in the dual-Purkinje Image eye tracker and fixation durations greaterthan 2000 ms were excluded as outliers, most likelyresulting from false locks in tracking. This procedureeliminated 5.6% in the Visual Search task and 7.1% in theMemorization task. All data reduction and analysis wereconducted using automated analysis software.Figure 1 shows the typical viewing patterns for two

participants looking at a scene in the memorization andsearch instruction conditions, respectively. The circlesrepresent fixations and the straight lines represent sac-cades. Consistent with the results of prior studies, viewersgenerally distributed their fixations across a large part ofeach scene, with the majority of fixations landing on ornear objects (Buswell, 1935; Yarbus, 1967; see Henderson& Ferreira, 2004, for review).Several measures were calculated to quantify viewers’

eye movement patterns on the scenes and on specificobjects as a function of viewing task. These measures arereported below. Eye movement measures for the entirescene will first be reported, followed by eye movementmeasures related to specific object processing in the scene.

Eye movement behavior on whole scene

For the analyses of the whole scene, eye movementmeasures were analyzed from scene onset to the button

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 4

press or timer expiration that terminated the trial. Theseanalyses were conducted to provide a global overview ofeye movements during the task and to allow comparisonagainst eye movement behavior in other types of sceneviewing studies.We examined ten eye movement measures to address

the degree to which fixation patterns in complex real-world scenes change as a function of viewing task. Thedata from these measures are summarized in Table 1 andFigures 2–5.

Spatial distribution of fixations

The first issue we examined was the degree to whicheye movements during scene viewing are distributeddifferently as a function of viewing task. As discussed inthe Introduction section, eye movements are typicallydistributed over informative scene regions at the expenseof uniform and uninformative regions. Whether or not agiven region is informative should be at least partlydetermined by its task relevance. Therefore, to the extentthat cognitive factors influence the global distribution offixations over a scene, the prediction is that the spatialdistributions of fixations will change as a function ofviewing task.Figure 2 shows the distribution of fixations in a scene

for all participants over the same scene performing eachof the task conditions. The figure was created byconvolving a fixation map with a Gaussian filter. Eachfixation was weighted according to the fixation’s durationand the Gaussian filter was defined by a standard deviationof 1 degree to approximate the size of the fovea (forsimilar analyses of fixation distributions, see Henderson,2003; Pomplun, Ritter, & Velichkovsky, 1996). Toquantify the spatial distribution of fixations shown inFigure 2, we computed the percentage of each scene thatwas foveated across participants. To calculate the areafixated, a circular filter (1- radius) was placed centered oneach fixation for each scene under each viewing condition;thus for each scene, the summed area occupied by thecircular filters represented the total area fixated. By thismeasure, on average, participants fixated 48% of the areaof the scenes in the memorization condition and 37% inthe visual search condition across scenes, t(29) = 4.89,p G 0.05. These data thus support the qualitativeobservation that participants distributed their fixationsmore widely in the memorization condition and concen-trated fixations to certain areas in the visual searchcondition that presumably would most likely contain thetarget object.

Total scan path length

As an additional measure of the dispersion of fixationsin each of the tasks, we also calculated the total scan path

Figure 1. Typical viewing patterns for two participants looking at ascene in the (A) Memorization and (B) Visual Search instructionconditions, respectively. The participants were asked to look for abucket in the visual search task.

Memorization task Visual search task

DifferenceMean SE Mean SE

Percentage of scene area fixated 48% 0.08% 37% 0.09% 11%*Total scan path length 82- 2.2- 74- 2.4- 8-**Total number of fixations 27.6 0.09 24.2 0.08 3.4**Average fixation duration (ms) 287 0.29 292 0.24 j5Average saccade amplitude (deg) 3.0- 0.03 3.1- 0.03 j0.1Elapsed time to first saccade execution (ms) 317 9.73 269 7.14 48**

Table 1. Global measures of eye movement behavior in complex real world as a function of viewing task. Note: *p G 0.05; **p G 0.01.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 5

length.1 In degrees of visual angle, we found that onaverage the total length of the scan path was significantlygreater in the Memorization task (82-) than in the Visualsearch task (74-), t(19) = 2.77, p G 0.01.

Total number of fixations

Related to the measure of total scan path length, we alsoexamined the total number of fixations. We found that the

total number of fixations was greater in the Memorizationtask (È28) than the Visual Search task (È24), t(19) =5.21, p G 0.001.Although we found significant differences in the total

scan path length and the total number of fixations as afunction of task, it is possible that these differences weredue to differences in viewing time. Specifically, for theVisual Search task, the total number of fixations wasÈ87% of the Memorization task and the total scan path

Figure 2. The distribution of all fixations of all participants over the same scene. (Top) The original image viewed by participants. (Bottomleft) The image showing placement of all fixation in the Memorization task condition. (Bottom right) The image showing fixation placementin the Visual Search task condition. As can be seen qualitatively in this figure, fixations tended to be more distributed in the memorycondition and more focused on search-relevant regions in the search condition. In this image, participants were asked to look for a bucketand fixations were concentrated in the windows of the hardware store (see text for more information on how these images were created).

Figure 3. The fixation duration distribution for each task.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 6

length was È90% of the Memorization task. This isequivalent to the difference in viewing times (viewingtimes for the Visual Search task is È86% of theMemorization task).2 We conducted a second analysis toequate viewing times for these two measures by includingonly fixations occurring within the first 8.7 s for each tasktype. We found that the task had a marginal effect on total

scan path length (Memorization: 72- and Visual Search:66-, t(19) = 1.89, p = 0.075) and had a significant effect onfixation count (Memorization: 24 and Visual Search: 22,t(19) = 4.19, p G 0.01). Taken together, the effect of taskon these two measures seems to be due to more thansimply the increased time participants spent viewing thescene in the Memorization task.

Figure 5. The saccade amplitude distribution for each task.

Figure 4. (A) The fixation duration by ordinal fixation number for each task. (B) The saccade amplitude by ordinal fixation number for eachtask. In order to quantify the differences, the first five saccades were analyzed for the ordinal fixation durations (see text for more details).

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 7

Fixation duration distribution

The fixation duration distribution for each task isshown in Figure 3. The shapes of the distributions foreach task were similar. To examine the distributionsmore closely, we analyzed the mean and ordinal initialfixation durations for each task.

Average fixation duration

Fixation duration has been shown to be a sensitivemeasure of ongoing cognitive processing in reading(Rayner, 1998) and similarly reflects ongoing processingin scene viewing (Henderson & Pierce, 2008; Henderson& Smith, in press). To determine whether overall fixationdurations were on average different in the two taskconditions, we computed each participant’s averagefixation duration across all scenes and compared theparticipant means as a function of viewing task. Theaverage fixation duration was a non-significant 5 msshorter in the memorization than the visual searchcondition, t(19) = 0.799, ns.

Fixation duration by ordinal fixation number

It has sometimes been reported that fixation durationchanges over the course of scene viewing (Antes, 1974;Friedman & Liebelt, 1981; Unema et al., 2005). Wetherefore conducted an analysis of fixation durations as afunction of ordinal fixation number in the scene andviewing task to determine whether durations change overthe time course of scene viewing, and if so, whetherdifferences in durations as a function of task might berevealed at different ordinal time points (Figure 4A).Analysis of the first five fixations on the scene revealed aneffect of ordinal fixation number, F(19) = 18.03, p G 0.01,but no effect of task, F G 1. Further analyses over the firstfive fixations showed a significant linear trend present forthe Visual Search task, F(98) = 42.86, p G 0.01, andMemorization task, F(98) = 8.93, p G 0.01. There were noeffects for the last 5 fixations (all F’s G 1).3 The increasein fixation durations over the first few fixations during theinitial viewing of the scene in both tasks suggests a quickinitial scene scan.

Elapsed time to first saccade execution

At the onset of the scene, the participants’ fixation isdirected at the center of the scene. The initial perception ofthe scene involves identifying the scene being presented(Castelhano & Henderson, 2007, 2008a; Potter, 1976;Schyns & Oliva, 1994) but also deciding and planningwhere to direct the next fixation (Castelhano & Henderson,2007). Holding all else equal, it seems reasonable toassume that differences across tasks in the time to executethe first saccade after scene onset is due to a combinationof these processes. We analyzed the elapsed time to the

first fixation and found that there was an effect of task,t(19) = 4.71, p G 0.01. Time to the execution of the firstsaccade upon scene onset was 48 ms longer in Memo-rization than in Visual Search, suggesting there was agreater time spent committing the information to memoryand possibly on planning the first eye movement.

Saccade amplitude distribution

Figure 5 shows the frequency distribution of saccadeamplitudes for each task condition. The saccade distri-butions for each task appear similar. To look at possibledifferences in saccade amplitude across task conditions,we analyzed the average saccade amplitude and theinitial ordinal saccade amplitudes.

Average saccade amplitude

The average saccade amplitude was only 0.06- longer inthe Visual Search task than the Memorization task and didnot differ significantly, t(19) = j0.62, ns. To more closelyinspect a possible effect of task on saccade amplitude, wealso analyzed the initial saccades made after scene onset.

Saccade amplitude by ordinal fixation number

Figure 4B shows the average saccade amplitude byordinal fixation number in each task. To analyze thesedistribution differences, we looked at the average lengthof the first 5 saccades made on the scene for each task in awithin-subject ANOVA. There was an effect of task,F(1,19) = 9.73, p G 0.01, in which initial saccades werelonger on average during Visual Search than Memoriza-tion. There was also an effect of ordinal saccadeamplitude, F(1,19) = 5.93, p G 0.01. Further analysesshowed significant linear trends for both the VisualSearch task, F(1,98) = 8.87, p G 0.01 and Memorizationtask, F(1,98) = 4.81, p G 0.05. There was no significantinteraction.

Eye movement behavior on objects duringscene viewing

For the object-level analyses reported below, eyemovement behavior on discrete objects in the scenes wasof interest. Scoring regions for these objects were definedby a rectangular box that was just large enough toencompass that object (see Figure 6). The pixel coor-dinates of the box were then taken as the position of theobject. The same objects and scoring boxes were used inthe analysis of the two viewing instructions. For the objectanalyses, each fixation in a scene was determined to bewithin or outside of the scoring box based on its pixelposition value as defined above. Assignment of fixation

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 8

position to object regions was independent of the initialgeneration of fixation positions.We examined eight measures of eye movement behav-

ior to address the degree to which eye movements onobjects in complex real-world scenes changes as afunction of viewing task. With the exception of theproportion of objects fixated, each measure was condi-tional on fixation of the target object, so non-fixations didnot contribute to the computed means. The measures weresubdivided into two groups:

a. measures that reflect properties of single fixationsand

b. measures that reflect aggregate fixations on objects.

The means for these measures are reported in Table 2.

Proportion of objects fixated

Of the three objects defined in each scene, wecalculated the proportion of objects that were fixated at

least once as a function of task. We found that theproportion of objects fixated was 13% greater in theMemorization than Visual Search task, t(19) = j4.23,p G 0.001. The result is consistent with the spatialdistribution of fixation measure described above showinga greater distribution of fixations in the Memorization thanVisual Search task.

Average saccade amplitude to object

In order to look at possible effects of task on saccadeamplitude, we also calculated the average saccadeamplitude that preceded the first fixation on an object.We found that the 0.17- difference between tasks was notsignificant, t(19) = 1.07, ns.

Average fixation duration and first fixation duration

The average fixation duration was only 11 ms longer inthe Memorization than Visual Search task and notstatistically significant, t(19) = j1.25, ns. We also foundfor the first fixation durations that the 14-ms differencebetween the tasks was not significantly different, t(19) =j1.43, ns.

First gaze duration

Initial processing of the objects was also measured bylooking at the first gaze duration. First gaze duration isdefined as the sum of all fixations made within thedefined object region before the eyes fixate anotherlocation outside the region. We found that the averagefirst gaze duration on objects was 91 ms longer in theMemorization than Visual Search task, t(19) = j7.27,p G 0.001.

First gaze fixation count

In addition to duration, initial fixation density onindividual objects during the initial processing wasmeasured by looking at the first gaze fixation count. First

Memorization task Visual search task

DifferenceMean SE Mean SE

Proportion of objects fixated 0.66 0.020 0.53 0.024 0.13**Average saccade amplitude to object (deg) 3.63 0.10 3.80 0.07 j0.17Average fixation duration (ms) 290 9.08 279 6.09 11Average first fixation duration (ms) 286 8.82 271 6.13 14First gaze duration (ms) 439 11.40 348 8.90 91**First gaze fixation count 1.6 0.05 1.3 0.04 0.3*Total time (ms) 830 30.40 644 18.50 185**Total number of fixations 2.7 0.07 2.2 0.05 0.53**

Table 2. Measures of eye movement behavior on discrete objects in complex real-world scenes as a function of viewing task. Note:*p G 0.01; **p G 0.001.

Figure 6. A sample scene in which three isolated objects weredefined in order to analyze differences in eye movement behavioron discrete objects. The only criterion for each scene was that thethree chosen objects were not occluded.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 9

gaze fixation count is defined as the initial number offixations made within the defined object region before theeyes fixate another region. We found that the average firstgaze fixation count was 1.6 fixation for the Memorizationtask and 1.3 for the Visual Search task, t(19) = j5.73,p G 0.01.

Total time on objects

The Total Time on objects was calculated by summingall fixations on an object. We found that total time spenton objects was 185 ms longer when participants wereperforming the Memorization than when performing theVisual Search task, t(19) = j4.77, p G 0.001.

Total number of fixations on objects

Related to the Total Time, we also looked at the averagetotal number of fixations on an object over the wholeviewing period. We found that the number of totalfixations was greater for the Memorization than VisualSearch task, t(19) = j7.05, p G 0.001, which is consistentwith the other aggregate eye movement measures reportedabove.

General discussion

The present study investigated how task instructionaffects eye movement patterns during the viewing of ascene. The effect of task on eye movement patterns hasbeen long established by pioneers of research into eyemovement patterns (Buswell, 1935; Yarbus, 1967). How-ever, as convincing as they are, the accounts of theseeffects are descriptive, fixation data are depicted asimages, and the results lack quantification. We sought toprovide quantitative analyses, with a specific emphasis oninvestigating the nature of fixation durations in addition totheir placement. We found that task effects are observed atboth the scene and object level of analysis. We also foundthat task affected both the placement and fixation durationpatterns during scene viewing.

Task effects on eye movements across thewhole scene

At the level of the whole scene, fixations were moredistributed in the memorization condition and morefocused on search-relevant regions in the search condition,which directly replicate the findings of Buswell (1935)and Yarbus (1967). This is not a surprising finding whenwe consider the strategies involved for each task. In theMemorization task, participants were told that they would

be tested on specific objects within the scene, and so toimprove encoding of the different objects, it makes sensethat they would try to fixate as many different objects aspossible. This pattern of spreading fixations over manydifferent objects was also reflected to some extent in thetotal scan pattern and to a greater extent in the greatertotal number of fixations; that is, there was a numericallylonger scan pattern and a higher count of fixations in theMemorization than in Visual Search task. In the VisualSearch task, fixations were more narrowly focused withinthe scene, and we can assume that participants limitedtheir fixations to areas that most likely contained thetarget. This finding is consistent with other studiesshowing that context information leads to more efficientsearches (Brockmole, Castelhano, & Henderson, 2006;Brockmole & Henderson, 2006, 2008; Castelhano &Henderson, 2007; Chun & Jiang, 1998; Neider &Zelinsky, 2006). Furthermore, this effect of context hasbeen implemented in a recent computational model byTorralba et al. (2006), which showed that participants’fixations largely remained within scene areas that werestatistically most likely to contain the target object.We found that for both tasks average fixation durations

increased as viewing time increased (for the first 5fixations) and then remained stable in the later viewingperiod. This finding is consistent with earlier studies thatreported similar patterns (Antes, 1974; Friedman &Liebelt, 1981; Unema et al., 2005). We found that thefixation durations stopped increasing after only È2 s. Thissteep increase during the first seconds of viewing is alsofound in other studies in which the task demands rely onthe quality of the initial performance (e.g., È3.4 s inUnema et al., 2005). One could conclude based on thefixation duration data that the initial scanning of thescenes did not differ between tasks. However, the saccadeamplitude measure across the whole scene seems to pointto a different pattern, which we will turn to now.When we examined average saccade amplitude, there

were no systematic differences between tasks; however,there were differences in the saccade amplitude during theinitial viewing of the scene. We found that participantsmade longer saccades during initial viewing in VisualSearch versus Memorization, while later saccades did notdiffer across tasks. Again, this difference can be attributedto the strategies that participants are implementing as theyexamine the scene in each task condition. However, it isnot clear whether the difference in saccade amplitude isdue to the participants staying closer to the center duringthe initial encoding for the Memorization task, or whetherthey are simply scanning the whole scene more thor-oughly in the Visual Search task. It is not clear what aproper baseline for these tasks would be, but in apreference rating task, Antes (1974) found that theaverage saccade amplitudes seem to decrease withincreased viewing time. If we see this as the default (aninitial wide scanning of the scene with the first fewseconds of viewing), then it may be that when a

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 10

memorization strategy is implemented the system canimmediately start to examine details without the need foran initial wide scanning of the scene. This finding is indirect contrast to other studies that report that the first fewfixations are controlled by stimulus factors alone (DeGraef et al., 1990; Mannan, Ruddock, & Wooding, 1995).For instance, Mannan et al. (1995) measured eye move-ments while viewers examined grayscale photographs for3 s each. The photographs were either high-pass filtered,low-pass filtered, or unfiltered. Results showed thatfixation positions were similar on the unfiltered and low-pass filtered scenes during the first 1.5 s of viewing.However, as noted by Henderson and Ferreira (2004),even if eye movement control is largely determined bystimulus factors during the initial scanning of the scene,this does not prevent fixations from being influenced bytask. In the present study, the immediate implementationof the memorization strategy is also seen with the elapsedtime to the first saccade and is discussed further below.The elapsed time to the execution of the first saccade

was much longer for the memorization task than the visualsearch task. This elapsed time until the first fixation (orthe initial fixation at scene onset) is theoretically differentfrom other fixations made on the scene because it involvesidentifying the scene being presented (Castelhano &Henderson, 2007, 2008a; Potter, 1976; Schyns & Oliva,1994), as well as deciding and planning where to targetthe next fixation (Castelhano & Henderson, 2007). Theadditional 48 ms in the Memorization task, in addition tothe shorter initial saccade amplitudes discussed above,suggest that the effect of task was immediate. That is, thelargest differences between the tasks were seen within thefirst few seconds of viewing with both saccade amplitudeand fixation duration becoming similar in the latter part ofthe viewing period. This immediate effect is interesting inlight of other top-down influences, such as the effect ofscene context on the examination of objects within thescene (De Graef et al., 1990; Henderson et al., 1999;Rayner, Castelhano, & Yang, 2009), which seem to onlyemerge in later viewing. Top-down effects due to scenesemantics seem to take a while to onset (relative to thewhole viewing period), while top-down effects of taskare seen immediately and seem to be more pronouncedin the first few seconds of viewing.

Task effects on eye movements on objects

To better understand the effect of tasks on theexamination of objects within the scenes, we also lookedat fixation patterns on objects. As would be expected fromthe effect of task on the distribution of fixations, we alsofound that participants tended to examine more objects inthe memorization task condition. However, theoreticallymore interesting is the failure to observe an effect of taskon the average fixation duration. The reason this isinteresting or even surprising is that the lack of an effect

of the task goes against the findings in the readingliterature, in which effects of task, context, word difficulty,and word length are seen at the level of the averagefixation duration (Rayner, 1998). Instead, we found thattask affected gaze duration by modifying the number offixations within a gaze on a given object. The samepattern was also seen across other aggregate measures ofeye movements on the objects viewed in the scenes. Thisfinding is consistent with the failure to observe effects ofother factors on individual fixation durations during sceneviewing (see Henderson & Ferreira, 2004; Henderson &Hollingworth, 1998, for review; but see Henderson &Pierce, 2008; Henderson & Smith, in press).In general, participants tended to spend more time

fixating objects in the Memorization task than in theVisual Search task. However, this was seen in the numberof times that the objects were fixated, not in the averagefixation duration. This finding is consistent with an earlierstudy by Loftus (1972) that reported memory for sceneregions was not related to the average fixation durationbut rather to the number of fixations made on the region.The finding that the number of fixations is greater formemory than the visual search task can be easilyattributed as a system level strategy by which visualinformation in the memory task is encoded more thor-oughly. However, because an equivalent effect is notfound at the level of the fixation duration may indicate alimit in the architecture governing the decision of when tomove the eyes. Rather than influencing when the eyesmove, the effect of the task on scenes was observed inregards to where the eyes move.

When to move the eyes during sceneperception

Based on Morrison’s (1984) reading model, researchers(Henderson, 1992; Rayner, 1998) have suggested that thedecision of when to move the eyes during scene viewingis based on the processing of currently fixated visualinformation to a certain level. In reading, that level isthought to be lexical access of the word (Reichle et al.,2003), whereas in scene viewing, it is proposed to be therecognition of the object at fixation (Henderson, 1992). Ina recent set of studies, Henderson and colleagues(Henderson & Pierce, 2008; Henderson & Smith, in press)investigated the degree to which the currently fixatedvisual information affects fixation durations on scenes. Bymasking the stimulus at the end of a saccade, theavailability of the scene information was delayed. Therationale (based on Rayner & Pollatsek, 1981) was that iffixation durations depend on the information currentlybeing encoded, then the fixation durations should increasein proportion to the delay of the stimulus onset. Resultsshowed that although there was a subpopulation offixations whose durations were not affected by the delay,for a second population of fixations there was a substantial

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 11

link between the availability of the fixated informationand the duration of the fixations. The authors concludethat fixation duration in scene viewing is partiallycontrolled by the immediately available information fromthe scene.The effect of the information at fixation on duration has

also been reported in other scene viewing studies. Forexample, researchers have found that stimulus factorssuch as semantically inconsistent objects result in longerfixation durations (Friedman, 1979; Henderson et al.,1999). These longer fixation durations have been attrib-uted to the requirement of more visual analysis of localdetails for inconsistent objects and to consolidation of theinconsistent information into the familiar scene gist orschema. So, for semantic anomalies, fixation durations arethought to be directly affected by higher order cognitiveprocesses (Henderson & Hollingworth, 1999).However, until now no study (that we are aware) has

directly investigated the effects of task on individualfixations. In a study by Henderson et al. (1999), taskinstructions were manipulated across experiments, andon average, fixation durations were numerically longer(38 ms) in the Memorization experiment than in theVisual Search experiment. In the present study, we foundan 11-ms difference that was not significantly different.The earlier study was not designed to directly compare theeffects of task on fixation duration and no gaze durationmeasures were reported (due to the fact that whenfixations landed on or near the target in the visual searchtask, the participant would terminate the trial). So, fromthis previous study, it is difficult to determine what roletask instruction played in controlling fixation and gazedurations during scene viewing.In the current study, we found that task influences gaze

duration but not average fixation duration. This finding oflonger gaze durations is similar to higher order cognitiveeffects mentioned above (i.e., semantic inconsistency);time spent examining objects in a memorization task isgreater than the time spent during a visual search, just astime fixating a semantically inconsistent object is greaterthan a semantically consistent one. Based on the notionthat fixation durations are determined by the amount ofprocessing completed (Henderson, 1992; Rayner, 1998), itis possible to posit why there is a qualitative differencebetween the increase in examining individual objectswithin scenes for inconsistent semantic information vs.task instruction information. For the inconsistent informa-tion, fixations durations could be longer either becauseobject recognition processes for unexpected objectsrequire more time, or because memory consolidation forunexpected objects takes longer (Henderson, 1992). Slow-ing down of object recognition does not apply to thepresent study. However, despite being easily recognized, asuccessful implementation of a memorization task instruc-tion requires that object information be properly con-solidated to ensure that the information is available for thememory test. Thus, to increase the time spent on the

currently fixated information, the system simply plans arefixation of the object when the criteria for saccadeinitiation (e.g., object recognition) are met.

Conclusions

We have attempted to outline in this paper the specificeffects of task on eye movement control during examina-tion of real-world scenes. This research has highlightedsome differences from reading that are perhaps unex-pected. Namely, the influence of task is found at the levelof aggregate eye movement measures and not at the levelof individual fixations. As discussed above, although thisfinding is consistent with some other studies on sceneperception, it is inconsistent with other studies examininghigher order processing. For instance, individual fixationdurations are thought to be affected by semantic anoma-lies. Here, we found that for a memorization task, in whichperformance would benefit from longer fixation times, theindividual fixation durations did not differ from a visualsearch task. This is the most intriguing finding of thepresent study and more research is needed to differentiatebetween these different effects of higher order cognitiveprocessing on the control of gaze. For instance, in the caseof some ambiguous visual information where the task is tomemorize versus search, will fixation durations then reflectdifferences in task? Previous studies suggest that longerfixation durations reflect the need for more visual analysisof local details or for consolidation of inconsistentinformation into a schema. Building on the present study,we believe that manipulating task and stimulus propertieswould give rise to valuable information on the control ofeye movements and the architecture of the visual system.

Acknowledgments

This work was supported by grants from the NationalScience Foundation (BCS-0094433 and ECS-9873531),NSF IGERT Program (DGE0114378), the Army ResearchOffice (W911NF-04-1-0078), and the Economic andSocial Research Council of the UK (RES-062-23-1092)to JMH and by a grant from the Natural Sciences andEngineering Research Council of Canada to MSC. Wethank George Malcolm for comments on an earlier draftof this article, and Eyal Reingold and another anonymousreviewer for helpful comments.

Commercial relationships: none.Corresponding author: Monica Castelhano.Email: monica.castelhano@queensu.ca.Address: Department of Psychology, Queen’s University,Kingston, ON, K7L 3N6, Canada.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 12

Footnotes

1We would like to thank Eyal Reingold for this

suggestion.2We thank an anonymous reviewer for making this

suggestion.3For both the fixation duration and saccade amplitude

ordinal analyses, fixations 20–24 were analyzed as the last5 fixations (ordinal fixations greater than 24 were muchless frequent for the Visual Search task than theMemorization task).

References

Antes, J. R. (1974). The time course of picture viewing.Journal of Experimental Psychology, 103, 62–70.[PubMed]

Brockmole, J. R., Castelhano, M. S., & Henderson, J. M.(2006). Contextual cueing in naturalistic scenes:Global and local contexts. Journal of ExperimentalPsychology: Learning, Memory, and Cognition, 32,699–706. [PubMed]

Brockmole, J. R., & Henderson, J. M. (2006). Using real-world scenes as contextual cues for search. VisualCognition, 13, 99–108.

Brockmole, J. R., & Henderson, J. M. (2008). Prioritizingnew objects for eye fixation in real-world scenes:Effects of object-scene consistency. Visual Cognition,16, 375–390.

Buswell, G. (1935). How people look at pictures. Oxford,England: University of Chicago Press.

Castelhano, M. S., & Henderson, J. M. (2005). Incidentalvisual memory for objects in scenes. Visual Cogni-tion, 12, 1017–1040.

Castelhano, M. S., & Henderson, J. M. (2007). Initialscene representations facilitate eye movement guid-ance in visual search. Journal of ExperimentalPsychology: Human Perception and Performance,33, 753–763. [PubMed]

Castelhano, M. S., & Henderson, J. M. (2008a). Stableindividual differences across images in humansaccadic eye movements. Canadian Journal ofExperimental Psychology, 62, 1–14. [PubMed]

Castelhano, M. S., & Henderson, J. M. (2008b). Theinfluence of color on the perception of scene gist.Journal of Experimental Psychology: Human Percep-tion and Performance, 34, 660–675. [PubMed]

Castelhano, M. S., & Rayner, K. (2008). Eye movementsduring reading, visual search, scene perception: Anoverview. In K. Rayner, D. Shem, X. Bai, & G. Yan(Eds.), Cognitive and cultural influences on eye

movements (pp. 175–195). Tianjin: Tianjin People’sPublishing House / Hove: Psychology Press.

Chun, M. M., & Jiang, Y. (1998). Contextual cueing:Implicit learning and memory of visual context guidesspatial attention. Cognitive Psychology, 36, 28–71.[PubMed]

Crane, H. D. (1994). The Purkinje image eyetracker,image stabilization and related forms of stimulusmanipulation. In D. H. Kelley (Ed.), Visual scienceand engineering: Models and applications (pp. 15–89).New York: Marcel Dekker.

De Graef, P., Christiaens, D., & d’Ydewalle, G. (1990).Perceptual effects of scene context on object identi-fication. Psychological Research, 52, 317–329.[PubMed]

Engbert, R., Longtin, A., & Kliegl, R. (2002). Adynamical model of saccade generation in readingbased on spatially distributed lexical processing.Vision Research, 42, 621–636. [PubMed]

Engbert, R., Nuthmann, A., Richter, E. M., & Kliegl, R.(2005). SWIFT: A dynamical model of saccadegeneration during reading. Psychological Review,112, 777–813. [PubMed]

Findlay, J. M., & Gilchrist, I. D. (2001). Visual attention:The active vision perspective. In M. Jenkins &L. Harris (Eds.), Vision and attention (pp. 83–103).New York: Springer-Verlag.

Friedman, A. (1979). Framing pictures: The role ofknowledge in automatized encoding and memory forgist. Journal of Experimental Psychology: General,108, 316–355. [PubMed]

Friedman, A., & Liebelt, L. S. (1981). On the time courseof viewing pictures with a view towards remember-ing. In D. F. Fisher, R. A. Monty, & J. W. Senders(Eds.), Eye movements: Cognition and visual percep-tion (pp. 137–155). Hillsdale, NJ: Erlbaum.

Henderson, J. M. (1992). Visual attention and eye move-ment control during reading and picture viewing. InK. Rayner (Ed.), Eye movements and visual cogni-tion: Scene perception and reading (pp. 260–283).New York: Springer-Verlag.

Henderson, J. M. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences,7, 498–504. [PubMed]

Henderson, J. M. (2007). Regarding scenes. CurrentDirections in Psychological Science, 16, 219–222.

Henderson, J. M., Brockmole, J. R., Castelhano, M. S., &Mack, M. (2007). Visual saliency does not accountfor eye movements during visual search in real-worldscenes. In R. van Gompel, M. Fischer, W. Murray, &R. Hill (Eds.), Eye movements: A window on mindand brain (pp. 537–562). Oxford, UK: Elsevier.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 13

Henderson, J. M., & Ferreira, F. (2004). Scene perceptionfor psycholinguists. In J. M. Henderson & F. Ferreira(Eds.), The interface of language, vision, and action:Eye movements and the visual world (pp. 1–58). NewYork: Psychology Press.

Henderson, J. M., & Hollingworth, A. (1998). Eyemovements during scene viewing: An overview. InG. Underwood (Ed.), Eye guidance while reading andwhile watching dynamic scenes (pp. 269–293).Oxford, UK: Elsevier.

Henderson, J. M., & Hollingworth, A. (1999). High-levelscene perception. Annual Review of Psychology, 50,243–271. [PubMed]

Henderson, J. M., McClure, K. K., Pierce, S., & Schrock, G.(1997). Object identification without foveal vision:Evidence from an artificial scotoma paradigm. Percep-tion & Psychophysics, 59, 323–346. [PubMed]

Henderson, J. M., & Pierce, G. L. (2008). Eye movementsduring scene viewing: Evidence for mixed control offixation durations. Psychonomic Bulletin & Review,15, 566–573. [PubMed]

Henderson, J. M., & Smith, T. J. (in press). How are eyefixation durations controlled during scene viewing?Evidence from a scene onset delay paradigm. VisualCognition.

Henderson, J. M., Weeks, P. A., Jr., & Hollingworth, A.(1999). Effects of semantic consistency on eyemovements during scene viewing. Journal of Exper-imental Psychology: Human Perception and Perfor-mance, 25, 210–288.

Itti, L., & Koch, C. (2001). Computational modelling ofvisual attention. Nature Reviews Neuroscience, 2,194–203. [PubMed]

Kliegl, R., & Engbert, R. (2003). SWIFT explorations. InJ. Hyona, R. Radach, & H. Deubel (Eds.), The mind’seye: Cognitive and applied aspects of eye movementresearch (pp. 391–411). Amsterdam: Elsevier Science.

Koch, C., & Ullman, S. (1985). Shifts in selective visualattention: Towards the underlying neural circuitry.Human Neurobiology, 4, 219–227. [PubMed]

Krieger, G., Rentschler, I., Hauske, G., Schill, K., &Zetzsche, C. (2000). Object and scene analysis bysaccadic eye-movements: An investigation with higher-order statistics. Spatial Vision, 13, 201–214. [PubMed]

Loftus, G. R. (1972). Eye fixations and recognitionmemory for pictures. Cognitive Psychology, 3,525–551.

Loftus, G. R., & Mackworth, N. H. (1978). Cognitivedeterminants of fixation location during picture view-ing. Journal of Experimental Psychology: HumanPerception and Performance, 4, 565–572. [PubMed]

Mackworth, N. H., & Morandi, A. J. (1967). The gazeselects informative details within pictures. Perception& Psychophysics, 2, 547–552.

Mannan, S., Ruddock, K. H., & Wooding, D. S. (1995).Automatic control of saccadic eye movements madein visual inspection of briefly presented 2-D images.Spatial Vision, 9, 363–386. [PubMed]

Mannan, S. K., Ruddock, K. H., & Wooding, D. S. (1996).The relationship between the locations of spatialfeatures and those of fixations made during visualexamination of briefly presented images. SpatialVision, 10, 165–188. [PubMed]

Mannan, S. K., Ruddock, K. H., & Wooding, D. S. (1997).Fixation sequences made during visual examinationof briefly presented 2D images. Spatial Vision, 11,157–178. [PubMed]

Masson, M. E. (1983). Conceptual processing of textduring skimming and rapid sequential reading.Memory & Cognition, 11, 262–274. [PubMed]

Morrison, R. E. (1984). Manipulation of stimulus onsetdelay in reading: Evidence for parallel programmingof saccades. Journal of Experimental Psychology:Human Perception & Performance, 10, 667–682.[PubMed]

Neider, M. B., & Zelinsky, G. J. (2006). Scene contextguides eye movements during visual search. VisionResearch, 46, 614–621. [PubMed]

Parkhurst, D., Law, K., & Niebur, E. (2002). Modeling therole of salience in the allocation of overt visualattention. Vision Research, 42, 107–123. [PubMed]

Parkhurst, D. J., & Niebur, E. (2003). Scene contentselected by active vision. Spatial Vision, 6, 125–154.[PubMed]

Pomplun, M., Ritter, H., & Velichkovsky, B. (1996).Disambiguating complex visual information: Towardscommunication of personal views of a scene. Percep-tion, 25, 931–948. [PubMed]

Potter, M. C. (1976). Short-term conceptual memory forpictures. Journal of Experimental Psychology:Human Learning and Memory, 2, 509–522. [PubMed]

Rayner, K. (1998). Eye movements in reading andinformation processing: 20 years of research. Psycho-logical Bulletin, 124, 372–422. [PubMed]

Rayner, K., Castelhano, M. S., & Yang, J. (2009). Eyemovements when looking at unusual/weird scenes:Are there cultural differences? Journal of Experimen-tal Psychology: Learning, Memory & Cognition, 35,254–259.

Rayner, K., & Pollatsek, A. (1981). Eye movementcontrol during reading: Evidence for direct control.Quarterly Journal of Experimental Psychology A:

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 14

Human Experimental Psychology, 33, 351–373.[PubMed]

Reichle, E. D., Pollatsek, A., Fisher, D. L., & Rayner, K.(1998). Toward a model of eye movement control inreading. Psychological Review, 105, 125–157.[PubMed]

Reichle, E. D., Pollatsek, A., & Rayner, K. (2006).Modeling the effects of lexical ambiguity on eyemovements during reading. In R. P. G. Van Gompel,M. F. Fischer, W. S. Murray, & R. L. Hill (Eds.), Eyemovements: Awindow onmind and brain (pp. 271–292).Oxford, England: Elsevier.

Reichle, E. D., Rayner, K., & Pollatsek, A. (2003). TheE-Z reader model of eye<movement control in read-ing: Comparisons to other models. Behavioral andBrain Sciences, 26, 445–476. [PubMed]

Reinagel, P., & Zador, A. M. (1999). Natural scenestatistics at the centre of gaze. Network, 10, 341–350.[PubMed]

Schyns, P. G., & Oliva, A. (1994). From blobs toboundary edges: Evidence for time and spatial scale

dependent scene recognition. Psychological Science,5, 195–200.

Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005).Visual correlates of fixation selection: Effects of scaleand time. Vision Research, 45, 643–659. [PubMed]

Torralba, A., Oliva, A., Castelhano, M. S., & Henderson,J. M. (2006). Contextual guidance of eye movementsand attention in real-world scenes: The role of globalfeatures in object search. Psychological Review, 113,766–786. [PubMed]

Unema, P. J. A., Pannasch, S., Joos, M., & Velichkovsky,B. M. (2005). Time course of information processingduring scene perception: The relationship betweensaccade amplitude and fixation duration. VisualCognition, 12, 473–494.

van Diepen, E. M. J., & d’Ydewalle, G. (2003). Earlyperipheral and foveal processing in fixations duringscene perception. Visual Cognition, 10, 79–100.

Yarbus, I. A. (1967). Eye movements and vision. NewYork, Plenum Press.

Journal of Vision (2009) 9(3):6, 1–15 Castelhano, Mack, & Henderson 15