Library Website Usability Test Results

Fall 2012

Why?

Identify confusing/problematic areas of the site and areas for further work and development

Ensure Library website meets needs of students at all COCC campuses

Make data-driven changes

Ultimately: Support student academic success via the websiteFlickr, @Doug88888

What is usability?

ISO 9241 (Ergonomics of Human System Interaction) defines usability quite well: Effectiveness Efficiency Satisfaction

BRLESC I, Early Electronic Computer. Photography. Encyclopædia Britannica Image Quest. Web. 10 Dec 2012.

Library Websites & Usability “Known problems” with

Library websites: Library jargon/terminology

(databases, periodicals, peer review)

Users’ lack of information literacy training

Lack of clarity in differences between catalogs, databases, online e-book collections, etc. and what can be found in each

Too many options on Library websites, especially for novice users

Cluttered homepages

District Of Columbia. Photography. Encyclopædia Britannica Image Quest. Web. 11 Dec 2012.

What Library Websites Can Feel Like for Novice Users

•13 scheduled volunteers•10 completed usability tests•18-20 additional volunteers turned away

Participants

21-24

25-29

30-34

35-39

40-44

0

4

8

Age Range

# of Partic-ipants

1-4 terms 9-12 terms

0

4

8

Time at COCC

# of Par-ticipants

Aver

age

Adva

nced

06

Self-Assessment of Computer Skills

# of Par-ticipants

Students Majoring in:•CIS (2)•Automotive•Undecided•AAOT (2)•Aviation

Students Majoring in:•Community Health Ed•HIT•Criminal Justice

Participants, Con’t

Yes No0

2

4

6

8

10

Have Taken a Library Class

# of Partic-ipants

0

4

8

Frequency of Visits to Library or Library

Website

# of Partic-ipants

Testing

13 common tasks to try to complete on Library’s website

Screen recorded using Blackboard (Bb) Collaborate & notes taken during testing

After completing task portion, students answered two short surveys in Bb Demographic-type information System Usability Scale (SUS)

1 2 3 4 5 6 7 8 9 10

16:04 16:50 16:57

20:53

4:235:37

11:03

14:58

19:51

23:02

Amount of Time Users Took to Complete 13 Tasks

Average (Mean): 29:21:08Range: 30:56

Task #1: Find the Library Hours.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #2: Find out if the Library owns a copy of College Study Skills: Becoming a Strategic Learner.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #3: Find out how long you can keep a checked out book.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #4: Find how to get immediate help with your research.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #5: Find out what’s on reserve for Math 111.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #6: Find a book on global warming.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #7: Find an article on depression.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #8: Find out if the Library has study rooms.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #9: Find the book Unquenchable: America’s Water Crisis and What to Do about It (not available at COCC).

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #10: Find the most recent online issue of the Journal of Ecology.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #11: Find a list of resources specifically for psychology.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #12: Find information about citing your sources.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

Task #13: Find an academic encyclopedia article about Mexico.

CE CP CU IEE IW IUT IS0

1

2

3

4

5

6

7

8

9

10

CE32%

CP22%

CU15%

IEE10%

IW18%

IUT3%

IS1%

Success Rate, All Tasks

SUS (System Usability Scale) Results

0 1 2 3 4 5 6 7 8 9 100

102030405060708090

100

SUS Scores

SUS Scores

Google Analytics Use Data

Jul-12* Aug-12 Sep-12 Oct-12 Nov-12 -

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

Library Website Pageviews

Icons or Links?

Top 10 Library Pages

defa

ult

data

base

s

Resea

rch-

Tool

s

cata

logs

artic

les

refe

renc

e

hour

s

subj

ects

acad

emicen

cylo

pedi

ashe

lp -

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

default38%

databases18%

Research-Tools16%

catalogs9%

articles6%

reference4%

hours3%

subjects2%

academicencylopedias2%

help1%

Suggestions from Participants No clear definition when searching COCC

web site library whether the information you are looking for is located in the COCC library. I couldn’t tell if I was still on the COCC website or was directed towards OSU's website till it loaded the screen. 

Improvements in consistency in the interface and reduction of options per screen would help me.

A more obvious link that would take me back to the homepage. 

Suggestions, Con’t

Making a little less complex and having the links on the left side a little more prominent.

Had I not taken [Lib 127] I think I would have found the site much more difficult to navigate.

I had trouble figuring out that I had to click on the arrow on the menu on the left on the homepage in order to get a dropdown menu with further choices. 

Once I got down a path to find resources I had to use the back button to get to the home page.

What Participants Liked

Lots of resources available Clean interface Icons on home page Well-labeled links Availability of help throughout the

site

Suggestions from Library Staff Improvements to “Subjects by Resources” page – both

in organization/presentation and location of link Make video tutorials easier to find, more obviously

linked Needs to be more engaging/welcoming; too boring –

more color, pictures Live chat widget instead of “Get a Live Answer” icon Return of rotating images to home page rather than

static icons Improve Articles & More page to give students better

idea of what to do when they land there Things in left menus are too hidden Too much scrolling necessary on articles,

encyclopedias, subjects by resources pages

Less is better, but how do we get there?

“Since they lack sufficient skills for searching and evaluating resources, it is difficult for undergraduates to navigate successfully in an environment that provides an overwhelming amount of information.” (Lee, 2008)

Ryoanji Temple, Dry Stone Garden And Blossom, UNESCO World Heritage Site, Kyoto City, Honshu Island, Japan, Asia. Photography. Encyclopædia Britannica Image Quest. Web. 11 Dec 2012.

On the horizon…

Winter term project focusing on Redmond, Madras, and Prinevillle students and how the Library website can best serve their needs.

Migrating to Ex Libris’ Alma/Primo will significantly change how people search for library resources…in ways both foreseen and unforeseen. This is also likely to have a significant

impact on our website. Sun Coming Up Over Horizon. Photography. Encyclopædia Britannica Image Quest. Web. 11 Dec 2012.

References

Brooke, J. (1996). "SUS: a "quick and dirty" usability scale". In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland. Usability Evaluation in Industry. London: Taylor and Francis.

Dixon, L. , et al. (2010). “Finding articles and journals via Google Scholar, journal portals, and link resolvers: Usability study results.” Reference & User Services Quarterly 50.2 (170-181).

Lee, H. (2008). “Information Structures and Undergraduate Students.” Journal of Academic Librarianship 34.3: 211-219.

Letnikova, G. (2008). “Developing a standardized list of questions for the usability testing of an academic library web site.” Journal of Web Librarianship 2.2 (381-415).

Lown, C., Sierra, T. & Boyer, J. (anticipated 2013). “How Users Search the Library from a Single Search Box.” College and Research Libraries. Pre-print.

McHale, N. (2008). “Toward a user-centered academic library home page.” Journal of Web Librarianship 2.2 (139-176).

Swanson, T. & Green, J. (2011). “Why We Are Not Google: Lessons from a Library Website Usability Study.” Journal of Academic Librarianship 37.3 (222-229).

Vakkari, P. & Huuskonen, S. (2012). “Search Effort Degrades Search Output but Improves Task Outcome.” Journal of the American Society for Information Science and Technology 63.4 (657-670).