What Do Exploratory Searchers Look at in a Faceted Search Interface?

Bill Kules and Matthew Banta

The Catholic University of AmericaSchool of Library and Information Science

Outline

• 60-second demo

• Our goals

• Research questions

• Experimental design

• Results

• Future work

Our Goals

• Improve understanding of how faceted interfaces affect searcher actions and tactics

• Develop and validate a methodology for creating exploratory search tasks for evaluations of search systems

Research Questions

• How long do searchers look at the major elements (facets, results, query box, breadcrumbs, etc.) of the interface?

• In what order do searchers look at the major elements in a faceted search interface?

Experimental Design

• N=18 successful sessions– From 21 subjects recruited

• 1x2 within-subjects design

• Task types– Exploratory tasks (n=4)– Known item tasks (n=2)

• Counterbalanced within task type

Procedure

• Introduction• 90 second training video• Calibrate eye tracker• Conduct 6 searches• Questionnaire after each search• Retrospective verbal report

– Video of two searches with gaze data overlaid

• Final questionnaire

Six Tasks

• ExploratoryA. Feminism in the United StatesB. Textile industry on three continentsC. Great Britain and the colonies in the 20th

centuryD. History of the Olympic games

• Known item: find a bookE. Firefly Encyclopedia of TreesF. Harry Potter and the Goblet of Fire

Exploratory Task

Imagine you are taking a class called “Feminism in the United States”. For this class you need to write a research paper on some aspect of the U.S. feminist movement, but have yet to decide on a topic. Use the catalog to find two possible topics for your paper. Then use the catalog to find three books for each topic so that you might make a decision as to which topic to write about.

Equipment Configuration

• NCSU Catalog Research Testbed• Tobii 2150 remote eye tracker

– 21” monitor– 50 Hz sampling rate– Resolution 1024x768

• Tobii Clearview v2.7.1• Gaze fixations

– Minimum 100 ms– 30 pixel radius

• Manually segmented AOIs

Query

Facets

Breadcrumbs

Results

Interface with Areas of Interest (AOIs)

How long did searchers look at the major elements of the interface?

Exploratory Search Tasks Known ItemSearch Tasks

For exploratory search tasks

In what order did searchers look at the major elements?

Transitions in Attention Between AOIs

Query

Facets

Breadcrumbs

Results

Interface with Areas of Interest (AOIs)

Post-search Interviews

• “The subject thing worked. I don’t normally do subject searches.”

• “I needed a subject and I didn’t want to look through 2000 books.”

• “shopping around” - selecting facets and then looking to see what was available for a particular subject

Limitations

• Training was provided

• Researcher-provided tasks in lab setting

• One high-level scenario

• Tasks constructed were focused on this study

Future Work• Additional factors

– Training – Next study supported by OCLC/ALISE grant and CUA purchase of an eye tracker

– Number and size of facets– Domain, search knowledge

• Additional measures– Gaze behavior – e.g. fixation counts– “Traditional” measures – e.g. clicks

• Refine procedure for exploratory search task generation

Conclusions• Facets played a major role in exploratory

searches– Fixation time about ½ as much as on results– On first page about equal– Facet-result & result-breadcrumb ~ ½ of all transitions

Acknowledgements

• Rob Capra, Tito Sierra, Jason Casden, and Joe Ryan

• Doug Oard and members of the UMD HCIL – for the use of their facilities and eye-tracker

• This research was supported by a grant from Catholic University and in part by a grant from the NSF/Library of Congress (IIS 0455970).