Proposed Working Memory Measures for Evaluating Information Visualization Tools
Laura Matzen, Laura McNamara, Kerstan Cole,
Alisa Bandlow, Courtney Dornburg & Travis Bauer
Sandia National Laboratories
Albuquerque, NM 87185
This work was funded by Sandia’s Laboratory Research and Development Program as part of the Networks Grand Challenge (10-119351).
Sandia National Laboratories is a multi-program laboratory operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin company, for the U.S. Department of Energy’s National Nuclear Security
Administration under contract DE-AC04-94AL85000.
Evaluation of information visualization tools
Evaluations typically developed for a single, specific task and tool Time consuming, expensive, results can’t be
generalized
We propose using measures of cognitive resources to create standardized evaluation metrics
Why assess cognitive resources?
All analysis tasks are cognitively demanding
Human cognitive resources are finite
Well-designed interfaces should free cognitive resources for making sense of data Reduce cognitive burden of searching and
manipulating data
Working Memory
Mental workspace underlying all complex cognition
Has a limited, measurable capacity
Often used as a performance metric in other domains
Proposed methodology
Evaluate visual analytics interfaces using a dual-task methodology: Primary task: Interaction with interface Secondary task: Test of working memory capacity
Performance on the secondary task should correspond to cognitive resources that would be available for sensemaking in a real-world analysis task
Example working memory task Sternberg task (Sternberg, 1969)
Memory set delay probe items
Low-load memory set: M G J High-load memory set: D K H Y R Q
M G J FM
W
Comparisons of different interface designs
FM
W
Later, Compare Different Visualizations of Same Dataset
FM
W
By this time next year…Pilot Study 1: Compare two interface designs for a simple video player with tagging feature
Pilot Study 2: Compare two different interface designs for a visual text analytics application developed at Sandia
Use NASA TLX to develop convergent evidence
Both studies should provide insight into the use of working memory based metrics for interface assessment
…we should have data!
Comments, please!
Top Related