Eye Tracking Within the Packaging Design Workflow:Interaction with Physical and Virtual Shelves

Chip TonkinClemson University

tonkin@clemson.edu

Andrew D. OuztsClemson University

aouzts@g.clemson.edu

Andrew T. DuchowskiClemson University

duchowski@clemson.edu

ABSTRACTMeasuring consumers’ overt visual attention through eyetracking is a useful method of assessing a package design’simpact on likely buyer purchase patterns. To preserve eco-logical validity, subjects should remain immersed in a shop-

ping context throughout the entire study. Immersion can beachieved through proper priming, environmental cues, andvisual stimuli. While a complete physical store offers themost realistic environment, the use of projectors in creatinga virtual environment is desirable for efficiency, cost, andflexibility reasons. Results are presented from a study com-paring consumers’ visual behavior in the presence of eithervirtual or physical shelving through eye movement perfor-mance and process metrics and their subjective impressions.Analysis suggests a difference in visual search performancebetween environments even though the perceived differenceis negligible.

Categories and Subject DescriptorsJ.4 [Computer Applications]: Social and Behavioral Sci.

1. INTRODUCTIONThe shift in consumer behavior over the past two decades

is forcing dramatic changes in the way products are designed,packaged and marketed. Presently 70% of consumer pur-chase choices are made at the shelf, 85% are made withouteven picking up a competitive item, and 90% are made af-ter looking at just the front face of the packaging [4]. Thuspackage design clearly plays a significant role in determin-ing the success of a product, but as Clement points out,currently accepted design methodology understates this im-pact and does not include any objective method of assess-ing the product’s visual impact on buying decisions. Eyetracking is a means of quantifying an observer’s overt vi-sual attention, and can be used to evaluate and comparevisual search patterns of individuals in a variety of situa-tions. This approach is readily applicable to the measure-ment of consumer behaviour in a retail environment when

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.NGCA ’11 Karlskrona, SwedenCopyright 2011 ACM 978-1-4503-0680-5/11/05 ...$10.00.

searching for a product on a store shelf. Companies such asKraft Foods, PepsiCo, and Unilever regularly employ thistechnology in the development of new packaging and retailstrategies, however, their methodologies and results are notavailable to the public [17]. For this approach to be usefulto and to be widely adopted by industry, it needs to be anintegral part of the design process that is incorporated as afeedback loop attached to the creative step in the workflow(as opposed to a research study performed post facto). Thegoal is similar to performance simulation developed for eval-uation of package design (similar to shelf life, sustainability,and shipping capability), and it follows that one would alsowant to accurately predict its consumer response.

The challenge is developing the workflow, equipment pa-rameters, and environment that provides a realistic approx-imation to the retail shopping experience. It must be usablefor a wide variety of products, remain cost effective, andprovide meaningful results. Most of the eye tracking equip-ment and the bulk of published research suggest use of anall-in-one monitor system. This is sufficient for testing webusability and responses to printed ads and promotions be-cause the size of the test stimuli readily fit the screens andthe seating position of the subjects is not unnatural. How-ever, this is not a realistic shopping simulation.

Alternatives to the all-in-one desktop system include self-standing eye tracking modules that can be used in front ofphysical samples or a projector screen. These have beenused for retail studies, but the requirement of standing per-fectly still in front of the shelf or projection screen makes thissetup less than ideal for simulating the retail shopping ex-perience. To better simulate the shopping experience, con-sumers should be allowed to“wander up and down the aisles”as they would do in reality.

Obviously the most realistic consumer experience studyought to be performed in an actual retail store, and insome instances this may be possible, but the logistics ofregularly rearranging a store to meet experimental condi-tions, and controlling many other real-world variables makethis prospect infeasible for the majority of typical controlledstudies.

The CUshop consumer experience lab (depicted in Fig-ure 1) will simulate browsing freedom within a realistic en-vironment. The lab is designed as a fully self-containedenvironment with sliding glass doors, re-configurable shelv-ing, a refrigerated section, and appropriate signage and win-dow treatments to create a realistic consumer shelf simu-lation. While it is understood that the physical environ-ment is ideal, the lab will contain equipment to run studies

Figure 1: Architectural renderings of the CUshop consumer experience laboratory (to be completed June

2011). The winning design was developed as part of the Spring 2011 Creative Inquiry class led by A. Hurley.

within projected virtual environments because projection of-fers cheaper and faster setup while preserving a high levelof stimulus control. The purpose of this study is to evalu-ate consumers’ visual behavior and subjective impressionsas they perform a product search task when encounteringeither a virtual projected or physical shelf unit.

The use of projectors to simulate an otherwise expensiveor difficult environment is not unusual. Flight and drivingsimulators have been used successfully for training and re-search purposes, and while they are not able to achieve com-plete physical or photo-realism, they have served as viablepredictors of behavior in various situations [15]. This papercompares and contrasts the impact of a virtual store shelfon consumer behavior to gauge the level of realism afforded.Although patterns of information acquisition when viewingan image are expected to be similar to those when viewingthe real environment, the general level of performance in ob-ject memory tests has been shown to be better in the latter[8]. In this paper visual is search is examined between thereal and virtual to test for performance differences, if any.

2. BACKGROUNDRusso and LeClerc describe the consumer’s product se-

lection process in three stages [13]. The first, dubbed “Ori-entation”, relies on the ability to evaluate overall patterns,colors, and shapes in the scene. Once an interesting areais identified, the consumer transitions to “Evaluation”, inwhich the focus is on a small number of items. At this stageinformation is likely processed much more intently and in aserial fashion [4]. The last phase of the process defined byRusso and LeClerc is the “Verification” stage. This is thepoint at which the consumer verifies that the product meetstheir needs, makes pricing comparisons, and garners assur-ance that it was the right choice. Currently, the consumer’srationale for making product selection decisions are stud-ied using standard interview techniques, focus groups, andobservation, but being able to objectively determine when,how long, and what attracts attention could give much moreprecise and actionable information than the softer, subjec-tive responses typically garnered from a focus group [21].

Assuming recorded visual attention follows the fovea [7],eye tracking can be a valuable tool for assessing consumerattention in shopping environments [17]. In cases wherethere is clear brand recognition, attention-catching packag-ing (imagery, colors, etc.) have less impact, but when brandis not a major consideration, packaging that captures atten-

tion is key in the buying decision [16, 12]. Buying decisionsare based on a combination of brand recognition and whatChandon et al. coined “visual equity” [2]. This term refersto the incremental consideration given to items that attracta buyer’s attention so that while a consumer enters with acertain amount of “memory equity” related to their needsand understanding of brand value, this can be changed atthe point of decision by what catches their attention. Eyetracking is a good tool for measuring this effect, as Johansenand Hansen discovered during webpage navigation, notingthat individual recollection of what attracted attention andwhat order things were seen in were not nearly as accurateas a record of recorded eye movements [6].

Because shoppers may not remember what they saw orperhaps because they may not be willing to honestly di-vulge their reasons for their decisions, the practice of mea-suring “brand recall” as typically done in marketing stud-ies is largely meaningless. According to Chandon et al.,brand recall is overwhelmingly driven by brand familiarityand, oddly, eye fixations on products within a given mar-ket segment can enhance brand recall for the target productwhether it is present in the study or not [2]. They foundthat major brands tended to inhibit the recollection of mi-nor brands while conversely the viewing of minor brandstended to enhance the recall of major brands.

Young makes the point that the most important factorin achieving applicable results is that the consumer mustbe kept in a shopping context [22]. He stated that “when ashopper is removed from this context, she often leaves be-hind the shopping mindset and, instead, takes on an art di-rector’s aesthetic mentality.” He compares this to a “beautycontest” in which the most aesthetically pleasing packagetends to win (this is not typically the attribute that actu-ally decides purchase decisions at the shelf). This lack ofrealism has been a significant problem (noted or otherwise)in practically all of the consumer shelf studies thus far.

Russo and LeClerc noted that the mean decision time intheir experiment was well above industry norm (30 secondsvs. 12 seconds) and gave several likely experimental setupreasons for their subjects’ slower behavior [13]. In review-ing these findings along with those of other experiments,Clement found that they presented serious validity problemsbecause they were in laboratory experiments that poorlysimulated real-world conditions [4]. Subjects were sitting inchairs looking at pictures of packages or viewing relativelysmall projected images that were not accurate for size orvisual angle.

0.75 m

2.5 m

4.0 m

1.25 m

Figure 2: Physical and virtual viewing dimensions with example participant searching for target item.

Even if subjects are shown an accurate picture with ob-jects taking up the same amount of visual angle as a reallife product, performance on spatial tasks significantly im-proves as the image becomes larger [14]. This supports thenotion that peripheral vision is used to improve search capa-bility. When searching, the consumer visually perceives theenvironment (i.e., the store shelf) in toto and in parallel, ori-enting selective attention [4]. This can be compared to one’sability to hear and “feel” the overall surrounding sounds orselectively listening to a particular voice in a crowd, but notdo both at the same time [3]. The size of the visual field istherefore likely to influence visual search, e.g., a small field islikely to restrict parafoveal preview benefit. Construction ofa visually realistic shopping environment is likely to matterto any studies of shoppers’ visual behavior.

Factors such as visual realism and the visual field size needto be considered in the CUshop design and methodology. Toachieve ecological validity, we believe it is necessary for thelaboratory to look and feel like a shopping environment, theeye tracking equipment must be unobtrusive and flexible,and the task be structured and primed in such a way thatthe subject carries the mindset of a consumer during thelength of the study.

To our knowledge, thus far all consumer-related eye track-ing studies have been restricted to environments where thevisual display was projected on a screen. Studies most sim-ilar to ours include those of Lundberg, Whitney et al., andChandon et al. [9, 18, 3]. Lundberg proposed developmentof the Packaging Media Lab, in which an eye tracker wouldbe used while a shopper viewed a shelf of products pro-jected on a screen. The lab was eventually designed by ThePackaging Arena, Ltd., and built within the Bergvik shop-ping centre in Karlstad, Sweden. Whitney et al. constructedthe Balance NAVE Automatic Virtual Environment consist-ing of three back-projected screens providing a wide field ofview projection-based system. Their purpose was not totest shopping decisions per se, rather, it was to test the ef-fect of navigation through the VR grocery environment onparticipants with and without vestibular dysfunction (no eyetracker was used). Chandon et al. used an eye tracker whenlooking at planograms (shown on a single 4�×5� screen, 80��

away from the viewer) to test the influence of number ofshelf facings and position.

In the present study, we measure the differences in vi-sual behavior between a virtual environment and its physicalcounterpart from which the virtual is derived.

3. METHODOLOGYThe effect of physical or virtual environment was mea-

sured on performance (visual search), process (eye move-ments), and subjective measures (i.e., the feeling of presence

within each environment and preference). The main taskwas search for a target item, with the main experimentalfactor consisting of environment type.

Stimulus. Two shelving environments were created forthe experiment. The physical shelf was a 3.6 m (141��) Aislemade with a Gondola 0.6 m (23��) base system, constitutinga 2 m (78��) tall shelving system with four 0.4 m (16��) deepupper shelves (this was used store shelving removed froma major US retailer). The shelf was populated with realphysical cereal boxes with two fabricated cereal brands usedas search targets.

The virtual environment was a snapshot of the physicalshelf projected on a wall. The image was captured by aCanon EOS Rebel T1i 500D camera mounted on a tripodapproximating the eye-level of an average-height US adult(1.7 m (67��) [11]). The image was then corrected for geo-metrical distortion caused by the lens, cropped, and resam-pled to achieve pixel dimensions of 2560×800, and displayedacross two Epson BrightLink 450 WI projectors, chosen fortheir brightness and short throw distance which eliminatedshadow interference when standing in front of the display.

In both physical and virtual presentations of the cerealshelf, care was taken to present the participant with thesame apparent view. In both instances the environmentmeasured 4.0×1.25 m (160��×49��) at an elevation of 0.75 m(30��) off the ground, as sketched in Figure 2. In both phys-ical and virtual search tasks, participants stood centered ata distance of 2.5 m (98��) from either display.

The stimuli (see Figure 3) used as search targets werecereal boxes made especially for this study to preclude fa-miliarity with the products. Artificial cereal boxes were cre-ated to ensure that they could not have been known a priori

to any of the participants. Each box measured 22×28 cm(8.5��×11��) and matched the dimensions of a box on the pro-jector wall. Figure 4 shows one of the physical cereal boxesmatching the dimensions of its projected counterpart.

Yellow and black price tags, visible in Figure 4, were alsoartificially created for this study and displayed below ev-ery distinct cereal box. Tobii’s infra-red (IR) markers wereplaced atop the darker portions of the price tags in an effortto blend their appearance.

Figure 3: Artificial cereal boxes designed and con-

structed specifically for the experiment.

Apparatus. Eye movements were captured using TobiiGlasses, a head-mounted eye tracking system resembling apair of glasses (see Figure 5(a)). The tracker is monocular(right eye only), sampling at 30 Hz with 56◦ × 40◦ record-ing visual angle. The Tobii Glasses were used in conjunctionwith two other pieces of hardware: the Recording Assistantand IR markers. The Recording Assistant is a small de-vice (4.7��×3.1��×1.1��) that attaches to the glasses and isused to both calibrate the eye tracker and store recordedeye movement and video data on a mini-SD card. IR mark-ers (see Figure 5(b)) are used to delineate an Area of Anal-ysis (AOA), a plane determined by the placement of 4 ormore IR markers, similar in concept to an Area/Region ofInterest (A/ROI) commonly used in eye tracking research todelineate sections of stimulus within which filtered eye move-ments, i.e., fixations, are counted. The difference between anAOA and an AOI is that an AOA exists in physical space andis required for data aggregation when the glasses are used.An IR marker serves this function only when attached to anIR marker holder; otherwise, it works in calibration modeand emits a visible (green) light for calibration.

Calibration. Calibration using the Tobii Glasses is some-what different from traditional calibration procedures em-ployed with table-mounted, fixed, or more commonly knownas “remote” eye trackers. To calibrate the glasses, an IRmarker is used in calibration mode. The experimenter firstasks the participant to stand at a distance of 1 m from a flat,vertical surface (e.g., a wall) and begins the calibration pro-cess using the Recording Assistant. The Recording Assistantthen displays a 3×3 grid of points to the experimenter, whomust position the IR marker at each corresponding point onthe wall. During this process, the participant is instructedto hold their head steady and follow the green light emittedby the IR marker with their eyes.

Experimental Design. The experiment consisted of a 2(environment) × 2 (box type) × 2 (box placement) design.The environment was either the physical or virtual cerealshelf, the box type included two versions of a cereal box(Figure 3), and box placement featured the target box atone of two locations (left vs. right). A center target positionwas avoided it is likely to be fixated first [20].

Each participant performed two trials, with environmentand box type reversed in the second trial, counterbalancingtrial combinations.

Figure 4: Physical cereal box held against its coun-

terpart projected in the virtual environment.

Participants. The study recruited 42 participants re-cruited from Packaging Science and Computer Science clas-ses. Ten participants were excluded from analysis due to cal-ibration issues (specifically we found that calibration pointson the left side of the grid were difficult for these participantsto fixate; a possible consequence of the monocular nature ofthe Tobii Glasses). Four additional participants were ex-cluded for incorrectly performing the task on at least onetrial—data showed post facto that these participants neverfixated the target box, their data could thus be consideredoff-target or erroneous. Analysis therefore considered onlysuccessful trials, consisting of data captured from 28 partici-pants (18 male, 14 female). These participants’ ages rangedfrom 20 to 42 (median 22).

Procedure. Before starting the experiment, participantswere asked to fill out a basic demographic questionnaire(gender, age, use and type of corrective lenses, etc.). Theywere then walked to an unmarked, white wall for the cal-ibration process. Participants stood 1 m (39��) from thewall and underwent the 9-point calibration procedure. Tenparticipants could not achieve a satisfactory calibration andwere thanked for their participation and dismissed.

Next, participants were given instructions for their first

(a) (b)

Figure 5: (a) Tobii Glasses, Recording Assistant,

and (b) IR marker. Courtesy of Tobii Technology.

task. If their task was the physical space task, the researchershowed the participant one of the two target boxes. The par-ticipant was told that their task would be to find this boxon a physical shelf and verbally announce its price. Theywere given as much time as desired to examine their targetbox in as much detail as they wished (no participant spentmore than 30 s). The participant was also shown examplesof the price tags’ appearance. They were then told the lo-cation of the physical shelf, and asked to walk directly to amarker on the ground (2.5 m (98��) from the stimulus) be-fore looking up at the shelf. When ready, they were askedto look straight ahead so the glasses could auto-adjust forrecording to begin. Finally, the experimenter walked withthe participant to the shelf and recorded eye movements un-til the participant announced the price of the object. Thephysical shelving area was concealed from the participantprior to this task, to avoid preview benefit.

For the virtual space task, a similar procedure was fol-lowed, with the only difference being that the participantwas walked to a marker 2.5 m (98��) from a projector wall,and the image on the projector was changed from a blankimage to the stimulus image when the participant was ready.

After the first task, the participant was given a custom-tailored Witmer-Singer presence questionnaire [19]. Theparticipant was given the option to remove the glasses whilethey took the questionnaire if they felt uncomfortable wear-ing them. Those who chose to remove them had to repeatthe calibration procedure before the second task; however,only one participant elected to do so. Participants werethen given their second task, with the same instructions.After completion of the second task, they were again giventhe presence questionnaire, but told that it referred only totheir experience in the second task (be it physical or vir-tual). Finally, the participant was given a post-experimentquestionnaire to collect subjective information (e.g., com-fort) and any comments related to the study.

Search in the environments was counterbalanced such thathalf the participants searched within the physical environ-ment first and half first searched in the virtual. Position ofthe target box was also counterbalanced so that one quarterof the trials contained the target at left, another quarter atright and vice versa (corresponding images of the physicalenvironment were used in the virtual projection).

3.1 Dependent MeasuresEye Tracking Metrics. The primary metric of interest

was time to first fixation on the target box. This metric ef-fectively measures time to task completion, or performanceof the task. Additionally, we measured the number of fix-ations prior to the first fixation on target. We considered,but rejected, other eye tracking metrics such as fixation du-ration. In this type of visual search task, a participant’seye movements typically consist mostly of saccades until thetarget is found. After the target is found, the number or du-ration of fixations on it give us no further information—wewere mainly interested if the time to location of the targetdiffered between environment types.

Presence Questionnaire. A presence questionnaire,based on Witmer and Singer’s version 3.0, tailored to thepresent experiment, was used to gauge participants’ sub-jective impressions of both environments, specifically alongfour subscales: immersion, involvement, sensory fidelity, andinterface quality. Four questions were chosen from the im-

mersion and involvement subscales and three from the sen-sory fidelity and interface quality subscales. All questionswere administered along a 7-point Likert scale. Questionsrelating to non-visual senses were omitted.

4. RESULTSEye movement data in the form of numbers of fixations

and time to first fixation of the target AOI were exportedfrom Tobii Studio for analysis with R [1].

A repeated-measures three-way ANOVA of time to firstfixation revealed significance of the main effect of environ-ment (F(1,27) = 22.77, p < 0.01). No other significanteffects (of box type or placement) were detected (see Fig-ures 6(a) and 6(b)).

A repeated-measures three-way ANOVA of the numberof fixations prior to the first fixation on the target also re-vealed significance of the main effect of environment (F(1,27)= 16.56, p < 0.01) but not of box type or placement (seeFigures 6(c) and 6(d)).

Results from the modified Witmer-Singer Presence Ques-tionnaire were analyzed following Madathil and Greenstein’sanalytical approach, by first computing the mean responsesof questions related to each of the four subscales used andthen comparing differences between each of these means (ofmeans) via a Welch two-sample t-test between physical andvirtual trials [10]. No significant differences were observedbetween the means of any of the four subscales tested (seeTable 1 and Figure 9). A trend toward higher perceivedfidelity appears to point toward the physical environment,but, on average, the effect is negligible. Furthermore, modalresponses to the subjective post-experiment questionnaireshow neutral preferential attitudes to either of the physicalor virtual (projector) tasks (see Table 2).

5. DISCUSSIONResults indicate that the physical environment afforded

significantly faster search performance than the virtual pro-jected image. The eye tracking data provides clear evidenceof the discrepancy in performance: because the number offixations generally coincides with time taken to complete vi-sual search, it is clear that participants took longer in thevirtual environment because they had to issue a larger num-ber of fixations. This is visualized in Figure 8 and shows thereason for the difference in time to task completion whichmight not have been evident had this been measured with astopwatch (eye tracking data provides clear evidence of ac-tive visual search—participants were not simply daydream-ing or staring at a fixed point).

Eye movement data also suggests that individuals mayhave approached the search task in a fundamentally differ-ent way over the projected image. Heatmap visualizationsof aggregated scanpaths are shown in Figure 7. Note thatthe heavily fixated regions in the four corners represent thepossible locations of the boxes—the image chosen for thevisualization is one of the layouts used in the experiment,it is used in Figure 7 as a representative for visualizationof aggregate data from all trials. In the virtual environ-ment, it appears that most viewers may have begun theirsearch near the center, but there is no such obvious trend inthe physical environment. What is particularly interestingabout this result is that Chandon et al. found that objectslocated near the center of the “shelf” can be seen more often

Table 1: Mean responses to the tailored Witmer-Singer presence questionnaire, marked on a 7-point Likert

scale with 1 indicating most negative agreement and 7 indicating most positive agreement to the given

question regarding experiences in either virtual or physical environment.

# Question Physicalenvironment

Virtualenvironment

Involvement

1. My interactions with the shelving environment seemed natural. 6.1 4.63. The visual aspects of the environment involved me. 5.8 5.18. I was able to completely survey or search the environment using vision. 6.4 6.211. I felt involved in the search task. 6.0 5.8

group means (means of means) 6.1 5.4Immersion

2. All my senses were completely engaged. 4.6 3.84. I was completely aware of events occurring in the real world around me. 5.9 5.16. The information coming from my visual sense felt inconsistent or disconnected. 2.5 3.212. I was distracted by display devices. 2.9 3.2

group means (means of means) 4.0 3.8Sensory Fidelity

7. My experiences with the shelving system seemed consistent with my real-world experience. 5.9 4.69. I felt that I was able to examine objects closely. 5.4 4.910. I felt that I was able to examine objects from multiple viewpoints. 4.5 3.9

group means (means of means) 5.3 4.5Interface Quality

5. I was completely aware of any display and control devices. 5.9 5.113. Visual display quality interfered or distracted me from completing my task. 2.2 3.014. I was able to concentrate on the search task and not on the devices used to perform the task. 6.1 5.2

group means (means of means) 4.5 4.6

Table 2: Modal responses to subjective post-experiment questions, marked on a 7-point Likert scale with 1

indicating strong disagreement and 7 indicating strong agreement.

# Question mode1. The eye tracking glasses felt comfortable. 62. The eye tracking glasses distracted me and hindered my ability to perform my tasks. 13. I preferred the projector search task to the physical search task. 44. I understood what was expected of me in each task. 75. I preferred the physical search task to the projector search task. 4

but not actually considered (for purchase) in correspondingpercentages. Their finding did not fit with other data thatsuggested that attention correlates fairly well with consider-ation. Since they did not use an actual shelf in their study(only a projected image), they speculated that this occurredbecause people might tend to orient their attention to thecenter of an image during a transition increasing the numberof fixations in the area (as is seen in Figures 7(b) and 8(b)).Our findings suggest that this might not occur as consis-tently in physical environments.

A key reason for the observed difference in visual searchperformance may be the fidelity of the projected scene. Al-though we were careful to control for apparent image size,the projected image clearly differs from its projected coun-terpart. The projectors offer relatively poor brightness andcontrast reproduction of the physical scene. The physicalscene is much richer in terms of visual elements (color gamut,contrast, and visual depth). The human eye can perceivea very high dynamic range contrast ratio, e.g., 100,000:1,with static perception of about 10,000:1 at any given time.The projectors’ lumens rating of 2,500 and contrast ratioof 2,000:11 may have impeded visual search in comparisonto what was seen in the physical environment. Projectorsare available with greater contrast ratios and spatial resolu-tion (e.g., 12,000:1, 1080p high-definition of the PowerLite

1http://www.epson.com/brightlink

home cinema projector), but these projectors are usually“long-throw” projectors and would cause shadow interfer-ence problems in the CUshop virtual shopping experiencebeing constructed.

What is curious in our study is the lack of perceived differ-ences in response to post-task presence and post-experimentpreference questionnaires. Figure 9 summarizes the datafound in Table 1 and shows that while the physical envi-ronment appears to have been rated slightly higher in termsof the presence subscales, the differences, along with modalresponses to preference, are negligible. The projected imagemay have failed to provide either physical realism (in whichthe image provides the same visual stimulation as scene) orphoto-realism (in which the image produces the same visualresponse as the scene), but the image may have containedsufficient functional realism (in which the images providesthe same visual information) [5] to perform the task, albeitconsistently more slowly (note that our data analysis per-tains to all successful trials).

6. CONCLUSIONResults were presented from a study comparing consumers’

visual behavior when searching for an item located on a vir-tual or physical shelf. These indicate that the physical en-vironment afforded significantly faster search performancethan the virtual projected image. Eye tracking data corrob-

0

2

4

6

8

10

Physical Virtual

Tim

e (

in s

econds; w

ith S

E)

Environment

Time to 1st

fixation on target

(a)

0

2

4

6

8

10

Left

Physical

Right Left

Virtual

Right

Tim

e (

in s

econds; w

ith S

E)

Target Placement × Environment

Time to 1st

fixation on target

(b)

0

50

100

150

200

Physical Virtual

Fix

ation C

ount (w

ith S

E)

Environment

No. of fixations prior to 1st

fixation on target

(c)

0

50

100

150

200

Left

Physical

Right Left

Virtual

Right

Fix

ation C

ount (w

ith S

E)

Target Placement × Environment

No. of fixations prior to 1st

fixation on target

(d)

Figure 6: Results: performance and process metrics.

(a) Physical environment

(b) Virtual environment

Figure 7: Heatmaps (all participants) in either env.

(a) Physical environment

(b) Virtual environment

Figure 8: Scanpaths (all participants) in either env.

0

1

2

3

4

5

6

7

Involvement Immersion Sensory

Fidelity

Interface

Quality

Lik

ert

score

means o

f m

eans (

with S

E)

Presence metrics

physical

virtual

Figure 9: Results: presence questionnaire.

orates this finding by indicating a significantly larger numberof fixations made over the virtual shelf.

One reason for the observed difference in visual search per-formance may be due to the poor fidelity of the projectedscene in comparison to the physical shelf. It is possible thatthe projectors’ relatively low contrast ratio impeded visualsearch. Better projectors and more photo-realistic simula-tions may improve congruence of eye movement metrics, butone must also consider the overall environment in which theparticipant is immersed. Advancements in other forms ofsimulation (automotive and flight for instance) have comenot from improvements in visual quality (e.g., resolution,contrast), but from an expanded field of view, realistic mo-tion, and sound. Although visual fidelity will continue toplay a significant role in the shopping simulation, the re-maining senses must also be addressed. We believe construc-tion of a physical space filled with tactile objects, rich visualelements, and sounds, through which participants navigate,will go a long way toward mitigating the sense of standingin front of a projection screen.

Physical shelves offer a step closer towards physical real-ism, but they are costly to set up and to stock. If there aresufficient resources, such shelves offer better ecological valid-ity. However, the lack of a perceived difference between theenvironments suggests that projected replicas may be suf-ficient for consumer testing (e.g., visual search) since theyprovide as much visual information. Our findings suggestthat virtual presentation of the stimulus offers a viable al-ternative to a physical mock-up so long as one maintainsawareness of the potential effect on performance in relationto performance in the field. If the effect is consistent, how-ever, then relative measurements of performance within thevirtual environments are still likely to be valid.

AcknowledgmentsWe would like to thank Andrew Hurley and his Spring 2011creative inquiry team for their help in designing and brand-ing the CUshop consumer experience lab. We are also grate-ful to Harris A. Smith for his generous support of the lab.

7. REFERENCES[1] J. Baron and Y. Li. Notes on the use of R for

psychology experiments and questionnaires. OnlineNotes, 09 November 2007. URL: http://www.psych.upenn.edu/~baron/rpsych/rpsych.html.

[2] P. Chandon, J. W. Hutchinson, E. T. Bradlow, andS. Young. Visual Marketing: From Attention to

Action. Lawrence Erlbaum Assoc., Mahwah, NJ, 2007.[3] P. Chandon, J. W. Hutchinson, E. T. Bradlow, and

S. Young. Does in-store marketing work? Effects ofthe number and position of shelf facings on brandattention and evaluation at the point of purchase.Journal of Marketing, 73:1–17, 2009.

[4] J. Clement. Visual influence on in-store buyingdecisions: an eye-track experiment on the visualinfluence of packaging design. Journal of Marketing

Management, 23(9):917 — 928, 2007.[5] J. A. Ferwerda. Three Varieties of Realism in

Computer Graphics. In Human Vision and Electronic

Imaging, pages 290–297, Bellingham, WA, 2003. SPIE.[6] S. A. Johansen and J. P. Hansen. Do we need eye

trackers to tell where people look? In CHI’06

Extended abstracts on human factors in computing

systems, pages 923–928, New York, NY, 2006. ACM.[7] A. F. Kramer and J. S. McCarley. Oculomotor

Behaviour as a Reflection of Attention and MemoryProcesses: Neural Mechanisms and Applications toHuman Factors. Theoretical Issues in Ergonomics

Science, 4(1–2):21–55, 2003.[8] M. F. Land and B. W. Tatler. Looking and Acting:

Vision and Eye Movements in Natural Behavior.Oxford University Press, New York, NY, 2009.

[9] E. Lundberg. Packaging Media Lab: A proposal to aPackaging Evaluation Environment for ConductingConsumer Studies. Master’s thesis, UppsalaUniversity, Uppsala, Sweden, Sep. 2004.

[10] K. C. Madathil and J. S. Greenstein. SynchronousRemote Usability Testing – A New ApproachFacilitated by Virtual Worlds. In CHI ’11: Proceedings

of the SIGCHI Conference on Human Factors in

Computing Systems, New York, 2011. ACM Press.[11] M. A. McDowell, C. D. Fryar, C. L. Ogden, and K. M.

Flegal. Anthropometric Reference Data for Childrenand Adults: United States, 2003–2006. Technicalreport, National Health Statistics, 2008.

[12] U. Orth and K. Malkewitz. Holistic package designand consumer brand impressions. Journal of

Marketing, 72:64–81, 2008.[13] E. J. Russo and F. LeClerc. An eye-fixation analysis of

choice processes for consumer nondurables. Journal of

Consumer Research, 21(2):274, 1994.[14] D. S. Tan, D. Gergle, P. G. Scupelli, and R. Pausch.

Physically large displays improve performance onspatial tasks. In ACM Transactions on Computer -

Human Interaction, volume 13, pages 71–99, NewYork, NY, 2006. ACM.

[15] J. Tornros. Driving behavior in a real and simulatedroad tunnel – A validation study. Accident Analysis

and Prevention, 30(4):497–503, 1998.[16] R. Underwood, N. Klein, and R. Burke. Packaging

communication: Attentional effects of productimagery. Journal of Product and Brand Packaging,10(7):403–422(20), 2001.

[17] M. Wedel and R. Pieters. A review of eye-trackingresearch in marketing. In Review of Marketing

Research. Emerald Group, Bingley, UK, 2008.[18] S. L. Whitney, P. J. Sparto, L. F. Hodges, S. V. Babu,

J. M. Furman, and M. S. Redfern. Responses to aVirtual Reality Grocery Store in Persons with andwithout Vestibular Dysfunction. CyberPsychology &

Behavior, 9(2), 2006.[19] B. G. Witmer and M. J. Singer. Measuring Presence

in Virtual Environments: A Presence Questionnaire.Presence, 7(3):225–240, June 1998.

[20] D. S. Wooding. Fixation Maps: QuantifyingEye-Movement Traces. In ETRA ’02: Proceedings of

the 2002 Symposium on Eye Tracking Research &

Applications, pages 31–36, New York, NY, 2002. ACM.[21] S. Young. Packaging design, consumer research, and

business strategy: The march toward accountability.Design Management Journal, 10(3):10–14, 2002.

[22] S. Young. Five principles for effective paging research.Brand Packaging, 18(1):24–26, 2005.