Human Vision and Information Visualization: A Collaboration

Ruth Rosenholtz & Allison WoodruffSIMS-Information Visualization and Presentation

November 2000

Outline

• The benefits of collaboration• Collaboration 1: Understanding clutter • Collaboration 2: Using thumbnails to

search the web

The benefits of collaboration

• Knowledge of human vision useful for design.– What determines how easy it is to find and

accurately interpret visual information?• What is processed preattentively?• How to best represent what kinds of information?• What affects how accurately one comprehends a

display?

• UI/infovis a potentially fruitful source of ideas for human vision research.

Barriers to collaboration

• Much current human vision research is aimed at uncovering mechanisms of the visual system.– This is not typically that useful for people who

want predictive models of visual processing.

• Much infovis research (as a result?) makes no reference to human vision, or uses results inappropriate to the application.

Outline

• The benefits of collaboration• Collaboration 1: Understanding clutter• Collaboration 2: Using thumbnails to

search the web

Clutter

• Makes a user interface less aesthetically pleasing.

• Makes it more difficult or more tiring to use.

• Makes it more difficult to understand a display and interact with it in a timely fashion.

• But what is it? Can we measure it? How do you reduce clutter?

Hypothesis: one component of clutter

• Clutter is when adding information to a display interferes with a user’s quick and accurate interpretation of the information in the display.

• Investigate: – What kinds of display manipulations interfere

with the extraction of information from the display?

VIDA

• Manipulates the density in visualizations• <<VIDEO>>• Manipulating density is good because

you fit more information in the display• … but what affect does this have on

perception of distribution?

Example: Replacing glyphs

• Added information:– Points in a scatter plot

frequently replaced with symbols representing other information about those data points.

• Does this added information affect judgments about the scatterplot?– E.G. trend of plot,

outliers, distribution of density of points in plot.

Example: Replacing glyphs

• Added information:– Points in a scatter plot

frequently replaced with symbols representing other information about those data points.

• Does this added information affect judgments about the scatterplot?– E.G. trend of plot,

outliers, distribution of density of points in plot.

Sample questions

• Does changing element:– Size– Shape– Color– ...

• Affect judgments of:– Number, density,

trend, location, or distribution of the data points

by making the judgments either slower or inaccurate/more difficult?

Effects of size on judgments of number

?>

?>

?>

fixed ## varies

Effect of size on judgments of number

• How many medium-sized disks do you need for it to appear that # small = # medium?

# large = # medium?

# mixed-size = # medium?

• Looking at quick judgments, here, not counting.

Results with small/large dots

20 40 60

20

40

60

80

100

# m

ed

ium

dots

# small dots20 40 60

20

40

60

80

100

# m

ed

ium

dots

# large dots

Results with mixed-size dots

20 40 60

20

40

60

80

100

# m

ed

ium

dots

# small+medium+large dots

Preliminary results

• Users tend to overestimate the number of small dots relative to medium dots.

• Users tend to underestimate the number of large dots relative to medium dots.

• No bias in estimating the number of mixed-size dots, but users’ performance is more variable.

Outline

• The benefits of collaboration• Collaboration 1: Understanding clutter• Collaboration 2: Using thumbnails to

search the web

Using Thumbnails to Search the Web

Allison Woodruff, Andrew Faulring, Ruth Rosenholtz, Julie Morrison, and Peter Pirolli

Outline

• Introduction• Related Work• Thumbnail Generation• Study• Conclusions

Comparing our enhanced thumbnails to plain thumbnails and text summaries

Motivation

• People spend a lot of time searching the Web

• We want to make search faster• Options

– Improve precision Improve summaries

13 million queries/day

Make them more accurate and make them easier to read quickly

Probably need to clarify our model fairly early (search terms)Probably need to clarify our model fairly early (search terms)

Text and Image Summaries

• Text summaries– Lots of abstract, semantic information

• Image summaries (plain thumbnails)– Layout, genre information– Gist extraction faster than with text

• Benefits are complementary• We create textually-enhanced thumbnails that

leverage the advantages of both text summaries and plain thumbnails

URL, textual context for search term

Text summaries don’t give you layout information, and they take longer to scan.

Can’t usually read text in thumbnails.

Outline

• Introduction• Related Work• Thumbnail Generation• Study• Conclusions

Related Work

• Other thumbnails– Reduced image plus “dogears” (Cockburn et

al.)– “Caricatures” showing selected features

(Wynblatt and Benson)

• Use of thumbnails– Data Mountain, Web Forager, Pad, etc.

• Thumbnails for recall– Czerwinski

Outline

• Introduction• Related Work• Thumbnail Generation• Study• Conclusions

System

• Preprocessor modifies HTML– e.g., increase size of

text, modify color of text

• Renderer creates scaled image of page

• Postprocessor transforms image– e.g., apply color wash,

add text callouts

System

• Preprocessor modifies HTML– e.g., increase size of

text, modify color of text

• Renderer creates scaled image of page

• Postprocessor transforms image– e.g., apply color wash,

add text callouts

System

• Preprocessor modifies HTML– e.g., increase size of

text, modify color of text

• Renderer creates scaled image of page

• Postprocessor transforms image– e.g., apply color wash,

add text callouts

Design Goals

• Enhance features that help the user decide whether document is relevant to their query– Emphasize text that is relevant to query

• Text callouts

– Enlarge (make readable) text that might be helpful in assessing page

• Enlarge headers

Design Issues: Visual Layering

• Problem: Callouts did not stand out – appeared to be part of original document

• Solution: Put them in a separate visual layer

Hard to do by modifying HTML because HTML documents so diverse

Putting Callouts in a Separate Visual Layer

• Transparency• Occlusion

Junctions indicate the occurrence of these events.

Design Issues:Color Management

• Problems: Callouts need to be both readable and draw attention

• Solution: Desaturate the background image, and use a visual search model to choose appropriate colors– Colors look like those in highlighter pens

Could show figures 1a/1c

Design Issues:Resizing of Text

• Problem: We want to make certain text elements readable, but not necessarily draw attention to them

• Solution: Modify the HTML before rendering the thumbnail

Assuming text in original page is readable

Assuming text in original page is readable

Examples

Outline

• Introduction• Related Work• Thumbnail Generation• Study• Conclusions

Tasks

• Criteria: tasks that…– Are representative of common queries– Have result sets with different characteristics– Vary in the number of correct answers

• 4 types of tasksPicture: “Find a picture of a giraffe in the wild.”Homepage: “Find Kern Holoman’s homepage.”Side-effects: “Find at least three side effects of halcion.”E-commerce: “Find an e-commerce site where you can

buy a DVD player. Identify the price in dollars.”

Each of the 4 tasks had a different number of correct answersRange: 1/100 correct to 20/100 correct

Some required genre classification. In others, semantic information in the text was especially useful. Some of the result sets were more textual than others.

Tasks were chosen so that the result sets would have different characteristics.

Study is based on search tasks we had the users perform.

Conditions

• Text summary– Page title– Extracted text with query

terms in bold– URL

• Plain thumbnail• Enhanced thumbnail

– Readable H1, H2 tags– Highlighted callouts of

query terms– Reduced contrast level

in thumbnail

For each of the questions we developed, we issued a query to Google. For each of the URLs Google returned as a search result, we created the following three types of summaries.

Thumbnails get same number of pixels as text summaries.

Collections of Summaries

• 100 results in random order

Approximately same number of each summary type on a page

Method

• Procedure– 6 practice tasks– 3 questions for each of the 4 task types

• e.g., each participant would do one E-commerce question using text, one E-commerce question using plain thumbnails, and one E-commerce question using enhanced thumbnails

– Questions blocked by type of summary– WebLogger recorded user actions during browsing– Semi-structured interview

• Participants– 12 members of the PARC community

Entire process took about 75 minutes

18 questions, with 100 query results each

12 men and women from Xerox PARC

All appropriate randomization was done so that different subjects did different questions with different conditions in different orders

Data Analysis

• Preliminary analysis:– Measure total search time on each trial– Normalize the data

• Some questions (“find …”) are slower than others• For each question, subtract the mean search time

for that question from the total search time

– Average normalized data across subjects• For each task type (picture/homepage/…) and

each condition (text/plain/enhanced), find mean normalized search time

Results

• Average total search times, by task:– Picture: 61 secs– Homepage: 80 secs– E-commerce: 64 secs– Side effects: 128 secs

Results

-60

-40

-20

0

20

40

60

80

100

Picture Homepage E-commerce Side-effects

Norm

ali

zed

tota

l se

arc

h t

ime (

s)

textplainenhanced

Nor

mal

ized

tota

l sea

rch

tim

e (s

)

Results Pooled Across All Tasks

• Subjects searched 20 seconds faster with enhanced thumbnails than with plain

• Subjects searched 30 seconds faster with enhanced thumbnails than with text summaries

• Mean search time overall was 83 seconds

Results: User Responses

• Participants preferred enhanced thumbnails– 7/12 preferred overall– 5/12 preferred for certain task types

• Enhanced thumbnails are intuitive and less work than text or plain thumbnails– One subject said searching for information with text

summaries did not seem hard until he used the enhanced thumbnails.

• Many participants reported using genre information, cues from the callouts, the relationship between search terms, etc.

Outline

• Introduction• Related Work• Thumbnail Generation• Study• Conclusions

Conclusions

• Relative performance of the summary types depends upon task

• In each task, enhanced thumbnails performed best, or tied for best

• Incorporating benefits of both text summaries and plain thumbnails led to consistently good performance for the enhanced thumbnails