Visualizing Information in Global Networks in Real Time Design, Implementation, Usability Study.
Usability & Evaluation in Visualizing Biological Data Chris North, Virginia Tech VizBi.
-
Upload
dustin-bradford -
Category
Documents
-
view
216 -
download
0
Transcript of Usability & Evaluation in Visualizing Biological Data Chris North, Virginia Tech VizBi.
Usability & Evaluationin Visualizing Biological Data
Chris North, Virginia Tech
VizBi
Usomics & Evaluationin Visualizing Biological Data
Chris North, Virginia Tech
VizBi
Myths about Usability
Usability = Voodoo
Science of Usability
Measurement
Modeling
Engineering
Science
Phenomenon
…analogy to biology
Usability Engineering
User-centric
Iterative
Engineering = process to ensure usability goals are met
1. Analyze Requirements
2. Design
3. Develop
4. Evaluate
Myths about Usability
Usability = Voodoo
Usability = Learnability
Myths about Usability
Usability = Voodoo
Usability = Learnability
Usability = Simple task performance
Impact on Cognition
Spotfire
66
40
0
10
20
30
40
50
60
70
80
90
GeneSpring
Insight gained:
Myths about Usability
Usability = Voodoo
Usability = Learnability
Usability = Simple task performance
Usability = Expensivehttp://www.upassoc.org/usability_resources/usability_in_the_real_world/roi_of_usability.html
Usability Engineering
1. Analyze Requirements
2. Design
3. Develop
4. Evaluate
Requirements Analysis
Goal = understand the user & tasksMethods: Ethnographic observation, interviews, cognitive task analysis
Challenge: Find the hidden problem behind the apparent problem
Analysts’ Process
Pirolli & Card, PARC
Systems Biology Analysis
Beyond read-offs -> Model-based reasoning
Mirel, U. Michigan
Usability Engineering
1. Analyze Requirements
2. Design
3. Develop
4. Evaluate
Why Emphasize Evaluation?
Many useful guidelines, but…
Quantity of evidence
Exploit domain knowledgeHunter, Tipney, UC-Denver
Science of Usability
Measurement
Modeling
Phenomenon
Measuring Usability in Visualization
system,algorithm
Measurements
• frame-rate• capacity• …
• realism• data/ink• …
• market• ?
• ?
2 kinds of holes
visualperception,interaction
inference,insight
goal,problemsolving
Phenomena
• task time• accuracy• …
Time & Accuracy
Controlled Experiments Benchmark tasks
Results
Performance Time
0
0.5
1
1.5
2
2.5
3
T1* T2 T3 T4* T5* T6 T7*Tasks
Tim
e (
in m
in)
1 Tpt M Tpts. M. Graphs
Accuracy
0
2
4
6
8
10
T1 T2 T3 T4* T5 T6* T7Tasks
Co
un
t
1 Tpt M Tpts. M. Graphs
+ Consistent overall
+ Fast for single node analysis- Slow and inaccurate for expression across graph
+ Accurate for comparing timepoints
p<0.05
Cerebral Munzner, UBC
Insight-based Evaluation
Problem: Current measurements focus on low-level task performance and accuracy
What about Insight?
Idea: Treat tasks as dependent variableWhat do users learn from this Visualization?Realistic scenario, open-ended, think aloudInsight codingInformation-rich results
Insight?
Spotfire
GeneSpring
Cluster/Treeview
TimeSearcher
HCE
Gene expression visualizations
Cluster- Time- Gene- View Searcher HCE Spotfire Spring
4.6
7
14
8
16
0
2
4
6
8
10
12
14
16
18
Av
g T
im
e to
F
irs
t In
sig
ht
48 51
34
66
40
0
10
20
30
40
50
60
70
80
90V
alu
e
18
21
14
25
20
0
5
10
15
20
25
30
Co
un
t
Count of insights
Total value of insights
Average timeto first insight(minutes)
Results
Insight Summary
Time series Viralconditions
Lupusscreening
Clusterview
TimeSearcher
HCE
Spotfire
GeneSpring
Users’ Estimation
41
4842
67
52
0
10
20
30
40
50
60
70
80
Av
g F
ina
l A
mo
un
t48 51
34
66
40
0
10
20
30
40
50
60
70
80
90
Va
lue
Total value of insights
Users’ estimated insight percentage
Cluster- Time- Gene- View Searcher HCE Spotfire Spring
Insight Methodology
Difficulties:Labor intensive
Requires domain expert
Requires motivated subjects
Short training and trial time
Opportunities:Self reporting data capture
Insight trails over long-term usage – Insight Provenance
Trend towards Longitudinal Evaluation
Multidimensional in-depth long-term case studies (MILCS)Qualitative, ethnographic
GRID: Study graphics, find features, ranking guides insight, statistics confirm
But: Not replicable, Not comparative
Shneiderman, U. Maryland
Onward…
VAST ChallengeAnalytic dataset with ground truth
E.g. Goerg, Stasko – JigSaw study
BELIV Workshop – BEyond time and errors: novel evaLuation methods for Information Visualization
Visual Analytics
Visualization Visual Analytics
Perception, Interaction
Cognition, Sensemaking
Visualization tasks Whole analytic process
Visual representations, interaction techniques
Connection to data mining, statistics, …
Datatype scenarios Real usage scenarios, Analysts
Embodied Interaction
GigaPixel Display Lab, Virginia Tech Carpendale, U. Calgary
1) Cognition is situated. 2) Cognition is time-pressured. 3) We off-load cognitive work onto the environment. 4) The environment is part of the cognitive system. 5) Cognition is for action. 6) Off-line cognition is body-based.
-- Margaret Wilson, UCSC