<Insert Picture Here>

Comparing Information Graphics: A Critical Look at Eye Tracking

Joe Goldberg and Jon Helfman

BELIV ‘10 – 4/10/10

Introduction

• How to select the most appropriate graph type?

• Consider three graphs, showing identical data. Can eye tracking methods help determine which graph type(s) • Are easiest to comprehend?• Are most appropriate for tasks?• Can be compared efficiently?

• Eye tracking may potentially reveal trends that are not easily observed by traditional usability metrics...but, eye tracking brings its own set of challenges

Eye Tracking Challenges: Practice-Related

• When is Eye Tracking Appropriate, and What Can it Tell Us?

• Designing and Conducting Studies• Length/number of tasks?• Allow mouse? Voice?• Recruitment profile? Number?• Allow dynamic stimuli?

• Analyzing and Reporting Data• How much time required?• Use quantitative metrics?• Spatial (heatmaps) vs. temporal (scanpaths)?• How to aggregate eye tracking scanpaths?• How to extract scanning strategy from data?• When are results significant enough to be meaningful?

Eye Tracking Challenges: Technical

• How to Define Fixations?• Several methods

• How to Define AOIs?• Padding• Overlap, hierarchy

• Assign Fixations to AOIs?• Spatial location + path?

• Gaze Location Error• Calibration• Foveal Area/Resolution• Attentional Dissociation

• Manage Scanpath Interruptions?• How to Define Metrics?

• No best practices or standards

Comparison of Information Graphics:An Evaluation Study

• Objective• Which graph type(s) best support relative comparison along

multiple dimensions, within and between graphs?• Compare bar, line, and spider graphs

• Participants: 5 graph users (Oracle colleagues)• Apparatus: Tobii T60 Eye Tracking system• 6 Tasks (same order)

• 3 Easy (2 dimensions, 1 graph)• “Should Candidate 2 be hired, considering only job dimensions a and b?”

• 3 Difficult (8 dimensions, 2 graphs)• “Which job candidate exceeded the greatest number of job requirements: 2 or 4?”

Graph Stimuli

• Each 4-graph set:

• 15x15 cm, Center of screen

1 job candidate per graph, indicated by number

Job dimensions indicated by a-h

Job requirements

Candidate abilities

Decision making, validation, uncertainty

Completion vs. First Fixation Times• Completion Times Include Scanning, Decision Making, and

Validation

Scanning Strategies• Two Different Comparison Strategies

• Bar Graphs: Most of first bar graph dimensions viewed before second bar graph viewed

• Spider Graphs: Much more reciprocal scanning between dimensions, compared to bar graphs

Discussion Questions

• How can eye tracking methods help evaluate information visualizations?

• What are the most appropriate metrics?• Can eye tracking methods evaluate dynamic,

interactive visualization usability?• How to aggregate and represent group scanning

strategies on visualizations?• Is there a ‘standard practice’ for eye tracking

methods?