28/03/14 pag. 1

Information visualization lecture 6

case studies

Katrien Verbert Department of Computer Science

Faculty of Science Vrije Universiteit Brussel

katrien.verbert@vub.ac.be

28/03/14 pag. 2

Case study 1: small interactive calendars

28/03/14 pag. 3

DateLens

28/03/14 pag. 4

design philosophy

… much of the groundwork for this design was laid by earlier work

… while individual features of FishCal [=DateLens] represent only variations of existing approaches, the primary contribution here is the integration of a host of techniques to create a novel application that is both usable and useful in an important domain.

28/03/14 pag. 5

11Sun

12 Mon

13 Tue

14 Wed

15 Thur

16 Fri

17Sat

Fly LAKathy to airport Model Maker

Check slides, notes.Family barbeque

Fly LHR Kathy to collectChapter 2/ see Dave March

JulyJuneMayAprilMar Aug Sept Oct

Flight to SFOTutorial set-upTutorialUnited flight Heathrow

PointerColor OHsJane+John

Call Kathy

Background: the first bifocal calendar (1980)

28/03/14 pag. 6

9 - 10

1 - 4

15Thu

15Thu

16Fri

8 - 99 - 10

12 - 1

3 - 4

Conference trip/who ?Lecture EE 2.23Rudzinski visitPromotions discussClinicOptimisation

prt alarm

20

21

22

23

24

25

26

27

28

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

Mon Mon Mon Mon

Tue Tue Tue Tue

Wed Wed Wed Wed

Thu Thu Thu Thu

Fri Fri Fri Fri

Sat Sat Sat Sat

Sun Sun Sun Sun

Tue

Wed

Thu

Fri

Sat

Sun

19Mon

Sun 25 Sun 4 Sun 11 Sun 18 Sun 25Sat 17 Sun 18

Wed 14 Tue 13 Mon 12

FEBRUARY MARCH

Wed

Thu

Fri

Sat

Sun

Mon

Align

ICON

Tue

16Fri

8 - 99 - 10

12 - 1

3 - 4

Conference trip/who ?Lecture EE 2.23Rudzinski visitPromotions discussClinicOptimisation

prt alarm

15Thu

Meet RCA group/mergePh.JB/RA + BTG/sg

Accounts MoDSection meeting

Check finance DD Tutorial

9 - 10

1 - 4

alarmprt

(a) (b)

The tectonic calendar. (a) Successive suppression of detail by masking; (b) the resulting tectonic calendar

28/03/14 pag. 7

maximise!maximise!maximise!

minimise! minimise! minimise!

tap!

Tiny view! Agenda view! Full Day view! Appointment! detail!

The four views offered, and some of the interactions involved in transitions between them

28/03/14 pag. 8

Movement of the lower scrollbar thumb controls the range of days displayed

Use of the scrollbar thumb control to adjust the visible time span

h"p://www.cs.umd.edu/hcil/datelens/    

28/03/14 pag. 9

Search

28/03/14 pag. 10

28/03/14 pag. 11

Usability study

•  6 male and 5 female subjects •  Each subject performed 11 tasks with the interface •  Limit of two minutes to complete each task •  Typical tasks:

–  Find  the  date  of  a  specific  calendar  event  –  Find  how  many  Mondays  a  par<cular  month  contains  –  View  all  birthdays  for  the  next  three  months  –  Find  free  <me  to  schedule  an  event  

28/03/14 pag. 12

Observations

•  performance measures: –   <me  needed  to  complete  task  –  Success  in  comple<ng  a  task  

•  User satisfaction and preference – by quantitative value (1=very difficult, 5 = very easy)

28/03/14 pag. 13

Average task completion times for the two calendars

28/03/14 pag. 14

Task success

The percentage of tasks completed by participants for each task (blue = datelens, red = Pocket PC) avg datelens: 88.2%, avg pocket pc: 76.3%

100

80

60

40

20

0 1 2 3 4 5 6 7 8 9 10 11Task

Average PercentCompleted

28/03/14 pag. 15

Usability

•  Several usability issues: –  Default  day  view  from  9am  to  5pm  –  Strong  concerns  about  readability  of  text  –  Desirability  of  seFng  own  default  views  

•  6 out of 11 users preferred traditional calendar •  1 subject abstained •  4 subjects preferred datelens

28/03/14 pag. 16

“Achieving positive results for first-time users of novel visualization systems is rare” (Kent Wittenburg)

28/03/14 pag. 17

references

•  Bederson, B., Clamage, A., Czerwinski, M., Robertson, G. (May 2002) DateLens: A Fisheye Calendar Interface for PDAs. Transactions on Computer-Human Interaction [Published Version]

•  Bederson, B. B., Clamage, A., Czerwinski, M. P., & Robertson, G. G. (2003, April). A fisheye calendar interface for PDAs: Providing overviews for small displays. In CHI'03 extended abstracts on Human factors in computing systems (pp. 618-619). ACM.

28/03/14 pag. 18

case study 2: web browsing through a keyhole

28/03/14 pag. 19

Seeking news

28/03/14 pag. 20 Source: Courtesy Oscar de Bruijn and Chieh Hao Tong

removal of graphic content to provide a complete menu in the display area, without the need for scrolling

28/03/14 pag. 21

The sequential presentation of link previews, each occupying the full available display area

28/03/14 pag. 22

The sequential presentation of link previews, each occupying the full available display area

28/03/14 pag. 23

Schematic diagram of the operation of the RSVP Browser

28/03/14 pag. 24 Source: Courtesy Oscar de Bruijn and Chieh Hao Tong

Link preview and associated news item

28/03/14 pag. 25 Source: Courtesy Oscar de Bruijn and Chieh Hao Tong

RSVP-Browser

itv

NEWS

History-bar

Link-bar

Main viewing area

Info-bar

1

System design

28/03/14 pag. 26

Navigational controls of the RSVP Browser

28/03/14 pag. 27

Evaluation

•  Comparative study with pocket version of Internet Explorer •  30 subjects:

–  15  subjects  used  RSVP  browser  –  15  subjects  used  Pocket  IE  

•  Tasks: each subject answered 8 questions •  Example:

 The  cross-­‐border  train  between  Belfast  and  Dublin  has  been  closed  aPer  several  explora<ons  were  heard  near  the  line.  Has  the  cause  of  these  explosions  been  iden<fied?    

• Data acquisition –   video  recordings    –   preferences  expressed  by  users  

28/03/14 pag. 28

Advice available to subjects taking part in the experimental evaluation of the RSVP Browser

Q1   Q2 to Q6   Q7 and Q8  

No instruction   given  

Subject could ask  experimenter for advice  

No advice available to  subject  

Procedure

28/03/14 pag. 29

The average times needed to answer questions 1, 7 and 8 for subjects using the RSVP Browser compared with Pocket IE

100

24 2522 24 25

020406080

100120

1 7 8

Question

Tim

e (s

econ

ds)

RSVP-Browser

Pocket IE

Time to solution

28/03/14 pag. 30

The mean number of extra (unnecessary) steps taken by subjects in finding the answers to questions 1, 7 and 8 using either the RSVP Browser or Pocket IE

1.7

0.4 0.40.3

0.80.4

0

0.5

1

1.5

2

1 7 8

Question

Ext

ra s

teps

RSVP-Browser

Pocket IE

Number of steps

28/03/14 pag. 31 Satisfactory Fast Too fastSlowToo slow

19

9

2

Subjects’ perception of the speed of the RSVP presentation

28/03/14 pag. 32

Discussion

•  Initially disadvantage because of unfamiliarity •  Initial disadvantage disappeared •  Advantages:

–  Reduc<on  of  efforts  to  scroll    –  Li"le  training  required  

•  Debate about best approach •  Many open questions •  But viable alternative

28/03/14 pag. 33

Reference

de Bruijn, O., & Tong, C. H. (2004). M-RSVP: Mobile Web browsing on a PDA. In People and Computers XVII—Designing for Society (pp. 297-311). Springer London.

28/03/14 pag. 34

Case study 3: archival galaxies

28/03/14 pag. 35

InfoSky

•  Designed to explore or search large collections of documents –  personal  collec<ons    –  collec<ons  maintained  by  news  organiza<ons  

•  Attention to value of algorithms to organize data spatially •  Case study: 109.000 news articles – organized in hierarchy •  Requirements:

–  Scalability  –  Hierarchy  plus  similarity  –  Focus+context  –  Stability  –  Explora<on  

28/03/14 pag. 36

A landscape representation of data about a collection of documents

Queries

Haptic

Navigation

Video Interaction

Cognition

Earlier work: BEAD

28/03/14 pag. 37

A themescape representation of 700 articles related to the financial industry

Earlier work: SPIRE

28/03/14 pag. 38

Earlier work: hyperbolic browser

28/03/14 pag. 39

h"p://portal.mace-­‐project.eu    

28/03/14 pag. 40

The cone tree, tilted to allow the text associated with each node to be readable.

Selective distortion could be applied to allow focus on any part

Earlier work: cone tree

28/03/14 pag. 41

View of the entire Galaxy, showing collection boundaries and titles at the top level

Design decisions

Metaphors  •  Galaxy  of  stars    

 …  to  represent  repository  •  Telescope    

 …  to  support  seman<c  zoom  

Design decisions

28/03/14 pag. 43

View of the sub-collection Bundeslander Deutschlands

Interaction and search

28/03/14 pag. 44

The result of clicking on the title Bayern. The mouse now hovers over Wirtschaftsraum Bayern

Interaction

28/03/14 pag. 45

The result of selecting Wirtschaftsraum Bayern

Interaction

28/03/14 pag. 46

At the lowest level titles are visible

Interaction

28/03/14 pag. 47

Highlighting of both relevant regions and documents follows the entry of a keyword

Search

28/03/14 pag. 48

Representation of collections at two levels of the hierarchy and, for the lower level, the layout of the collections (A, B, C, etc.) and their centroids (C1, C2, C3)

Layout

28/03/14 pag. 49

Evaluation

•  Evaluated in comparison with conventional tree viewer

•  Eight subjects divided in two groups –  First  group  used  InfoSky  first,  then  tree  viewer  –  Second  group  vice  versa  

•  Five tasks •  Video recording and interview •  Time recorded •  On average tree performed sign. better •  Potential reasons:

–  Many    hours  of  experience  with  tree  –  Early  stage  of  development  

28/03/14 pag. 50

Reference

Andrews, K., Kienreich, W., Sabol, V., Becker, J., Droschl, G., Kappe, F., ... & Tochtermann, K. (2002). The infosky visual explorer: exploiting hierarchical structure and document similarities. Information Visualization, 1(3-4), 166-181.

28/03/14 pag. 51

Case study 4: Student Activity Meter (SAM)

28/03/14 pag. 52

Student activity meter

28/03/14 pag. 53

Design Based Research Methodology

•  Rapid prototyping •  Evaluate Ideas in short iteration cycles of design,

implementation & evaluation •  Focus on usefulness & usability

–  Think-aloud evaluations – User satisfaction (SUS, desirability toolkit)

28/03/14 pag. 54

System Usability Scale h"p://www.measuringusability.com/sus.php    

28/03/14 pag. 55

Desirability toolkit

Benedek,  J.,  &  Miner,  T.  (2002).  Measuring  Desirability:  New  methods  for  evalua<ng  desirability  in  a  usability  lab  seFng.  Proceedings  of  Usability  Professionals  Associa<on,  2003,  8-­‐12.      

28/03/14 pag. 56

Iteration one

•  usability and user satisfaction evaluation •  12 CS students, using a twitter-based time tracker •  2 evaluation sessions:

–  task based interview with think aloud (after 1 week of tracking)

–  user satisfaction (after 1 month)

28/03/14 pag. 57

Learnability, errors & efficiency

28/03/14 pag. 58

User satisfaction

•  average SUS score: 73%

28/03/14 pag. 59

User satisfaction

demo-graphics

evaluation goal

design changes

negative positive

I. 12 CS

students

usability, satisfaction, preliminary usefulness

1st iteration small usability

issues

• ↑learnability • ↓errors • good satisfaction • usefulness positive

II. 19

teachers & TA’s

assessing teacher needs, use & usefulness

help function resource

recomm. not useful

• provides awareness • all vis. useful • many uses • 90% want it

III. 12

participants

assessing teacher needs, expert

feedback, use & usefulness

re-orderable PC with

histograms

most addressed needs are indecisive

• provides awareness and feedback • many uses • 66% want it • recomm. can be useful

IV. 11

teachers & TA’s

use, usefulness & satisfaction

filter & search, icons, zooming in line chart,

editing PC axes

conflicting visions of

students doing well or at risk

• provides time overview • provides course overview • PC assist with detecting problems • many uses & insights • 100% want it

28/03/14 pag. 61

References

Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012, May). The student activity meter for awareness and self-reflection. In CHI'12 Extended Abstracts on Human Factors in Computing Systems (pp. 869-884). ACM.

28/03/14 pag. 62

Case study 5: TalkExplorer

28/03/14 pag. 63

Recommender systems

63  

28/03/14 pag. 64

User-based CF

Sam  

Ian  

Neil  

A  

B  

C  

high  correla+on  

28/03/14 pag. 65

TalkExplorer

•  Purpose: visualizing recommendations to support –  explora<on  –  transparency    –  controllability  

•  Context: academic conferences

28/03/14 pag. 66

Problem statement

•  Complexity prevents users from comprehending results –  Trust issues when recommendations fail –  Aggravated with contextual recommendation

•  The black box nature of RS prevents users from providing feedback

•  Algorithms typically hard-wired in the system code –  generate a list of top-N recommendations –  little research has been done to study more flexible

approaches

66  

28/03/14 pag. 67

Approach

•  Using set relevance visualization –  One dimension of relevance = one set

•  Agent metaphor to mix user- tag- and engine-based relevance –  recommender systems are shown as agents –  in parallel to real users collecting talks –  tags are also agents collecting talks –  users can interrelate entities to find items

28/03/14 pag. 68

Conference Navigator

68

28/03/14 pag. 69

TalkExplorer

69  

28/03/14 pag. 70

John  O'Donovan,  Barry  Smyth,  Brynjar  Gretarsson,  Svetlin  Bostandjiev,  and  Tobias  Höllerer.  2008.  PeerChooser:  visual  interac<ve  recommenda<on.  CHI  '08  

Related work: PeerChooser

28/03/14 pag. 71

Related work: Smallworlds

Gretarsson, B., O'Donovan, J., Bostandjiev, S., Hall, C. and Höllerer, T. SmallWorlds: Visualizing Social Recommendations. Comput. Graph. Forum, 29, 3 (2010), 833-842.

28/03/14 pag. 72

Related work: TasteWeights

Bostandjiev,  S.,  O'Donovan,  J.  and  Höllerer,  T.  TasteWeights:  a  visual  interac<ve  hybrid  recommender  system.  In  Proceedings  of  the  sixth  ACM  conference  on  Recommender  systems  (RecSys  '12).  ACM,  New  York,  NY,  USA  (2012),  35-­‐42.      

28/03/14 pag. 73

TalkExplorer

28/03/14 pag. 74

Evaluation

•  Setup –  supervised user study –  21 participants at UMAP 2012 and ACM Hypertext 2012 conferences

•  Procedure –  Tasks

•  interact with users and their bookmarks •  interact with agents •  interact with tags

–  Post-questionnaire

74

28/03/14 pag. 75

Evaluation

•  Data collection –  recordings of voice and screen using camtasia studio –  system logs

•  Measurements –  effectiveness: number of explorations/number of selections

28/03/14 pag. 76

Effectiveness

76

28/03/14 pag. 77

Summary results

28/03/14 pag. 78

Post-questionnaire

28/03/14 pag. 79

Post-questionnaire

28/03/14 pag. 80

Reference

Verbert, K., Parra, D., Brusilovsky, P. and Duval, E. Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 17th International Conference on Intelligent User Interfaces (IUI’13), IUI’13, pages 1-12, New York, NY, USA, 2013. ACM

28/03/14 pag. 81

Questions?

28/03/14 pag. 82

Readings

•  Chapter 6