WVU Libraries Website Study

USABILITY TESTINGWVU Libraries Website

WVU LIBRARIES WEBSITE STUDYProblem:

We used TechSmith Morae to conduct usability testing of the new WVU Libraries website. The first round of usability testing was internal (undergraduate

student employees), from which we compiled and implemented suggested changes based on test results. This is the results of the second round of

usability testing, with randomly selected undergraduate students, and focusing on testing the changes that were implemented.

Study Description:A study to explore the usability issue with the WVU Libraries website.

Study Instructions:Thanks for participating in this study!

You will be asked to do tasks to help WVU Libraries understand where we need to improve our website https://lib.wvu.edu.

As you perform each task, please talk out loud about what you are doing and thinking.

1ST ROUND OF TESTINGTask 1: Go to the WVU Libraries website.Task 2: Find today’s hours for Eliza’s Coffee Shop.Task 3: Where can you find information about the kinds of cameras you can check out from the library?Task 4: Find the database Project Muse.Task 5: What is the loan period for books for undergraduate students?Task 6: On which floor of the Downtown Library can you find a book with the call number D810.S7L4 1954a?Task 7: Find out how to renew a book onlineTask 8: See if study room 1028 is available at 5pm tomorrow.Task 9: Find a peer-reviewed article on global warming.Task 10: Does the library have a book written by Mark Twain?Task 11: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library.Task 12: Where can you find information about how to print in color?Task 13: I need help writing citations in APA.Task 14: Find directions to the Health Sciences Center Library.Task 15: Is there a way to ask a question about the library or your research from the library website?Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the

course. Where should you go on the website to find it?Task 17: Can you find Martin Dunlap’s phone number on the WVU Libraries website?

ROUND 1 IMPLEMENTSSearch Box• Add color tabs• Possibly add icons• Talk to research services about indexing the webpages• Use 'Site Search' instead of 'Website'• Add 'Call Numbers' or 'Call#' next to MountainLynx

WVU Header• Eliminate 'Contact Us' Looked At Analytics• Make 'Library Hours' instead 'WVU Home'• Change 'Directory' to go to WVU Directory• 'Get Help' becomes 'Contact Us' page containing:

– Contact Us form– Ask A Librarian– Research Consultation form– Suggest A Purchase form– Hours possibly?

Design• Refining Print CSS• Make Ask A Librarian tab more dominant• Look at adding color to homepage quicklinks• Increase contrast of backpage search container

Services• Add/Restructure to list the following:

– Printing, Copying, Scanning– Computers and Laptops– Study Room Reservations

• Reorganize Borrowing page:– Make navigation reflect priority of undergraduate students– Redesign to make information more accessible

• Reduce extraneous information• Identify standardized links that should live on every page

Web Application• Include Google Analytics code into applications that end with

.lib.wvu.edu (eReserves, rooms, and hydra)• Have Call Number/Floor Maps link directly from the catalog• Change 'Directory' to 'Faculty & Staff'• Redesign/Rework Interface

– Room Reservations– Directory– Hours

Not implemented

2ND ROUND OF TESTINGTask 1: Go to the WVU Libraries website.Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library?Task 3: Find today’s hours for Eliza’s Coffee Shop.Task 4: Where can you find information about the kinds of cameras you can check out from the library?Task 5: Find the database Project Muse.Task 6: What is the loan period for books for undergraduate students?Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? Task 8: Find out how to renew a book onlineTask 9: See if study room 1028 is available at 5pm tomorrow.Task 10: Find a peer-reviewed article on global warming.

Does the library have a book written by Mark Twain?Task 11: How would you submit a request for an appointment for with a librarian for research assistance? Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library.Task 13: Where can you find information about how to print in color?Task 14: I need help writing citations in APA.

Find directions to the Health Sciences Center Library.Task 15: Find the business database “MRI Mediamark Reporter.”

Is there a way to ask a question about the library or your research from the library website?Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the

course. Where should you go on the website to find it?Task 17: Can you use the website to find the journal of Aircraft?Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website?Task 19: Find a call number for a copy of “Gone With the Wind” on video.

Additions Removed

TEST PARTICIPANTS

SamElectrical Eng., 2 yrs.

StephanieAnimal/Nutrition, 1 yrs.

TaylorBiochemistry, 4 yrs.

VanessaIndustrial Rel., 5+ yrs.

ZelaIndustrial Eng., 1 yrs.

AndrewHistory, 5+ yrs.

HartleySecondary Ed., 2 yrs.

JesseSpeech Path., 4 yrs.

MorganMathematics, 3 yrs.

RobertFinance & Accnt., 3 yrs.

1ST OVERALL TASK:AVERAGE TIME

* Average time includes delivery of tasks.

2ND OVERALL TASK:AVERAGE TIME


AVERAGE TIME: COMPARISON * Average time includes delivery of tasks.

ORDER TASK Test 1 - Overall Time Test 2 - Overall TimeTask 1: Go to the WVU Libraries website. 0.59 0.44Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? 1.4Task 3: Find today’s hours for Eliza’s Coffee Shop. 1.47 0.9Task 4: Where can you find information about the kinds of cameras you can check out from the library? 1.58 1.24Task 5: Find the database Project Muse. 1.39 0.57Task 6: What is the loan period for books for undergraduate students? 1.33 0.94Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? 2.6 1.42Task 8: Find out how to renew a book online 0.81 0.87Task 9: See if study room 1028 is available at 5pm tomorrow. 1.21 0.9Task 10: Find a peer-reviewed article on global warming. 1.3 0.71Eliminated 1: Does the library have a book written by Mark Twain? 0.75Task 11: How would you submit a request for an appointment for with a librarian for research assistance? 0.94Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library. 2.11 0.58Task 13: Where can you find information about how to print in color? 1.33 0.64Task 14: I need help writing citations in APA. 1.26 0.9Eliminated 2: Find directions to the Health Sciences Center Library. 1.04Task 15: Find the business database “MRI Mediamark Reporter.” 0.87Eliminated 3: Is there a way to ask a question about the library or your research from the library website? 0.56Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? 1.31 1.68Task 17: Can you use the website to find the journal of Aircraft? 0.67Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website? 1.28 0.78Task 19: Find a call number for a copy of “Gone With the Wind” on video. 1.4

0 0.5 1 1.5 2 2.5 3

Go to the WVU Libraries website.

Find today’s hours for Eliza’s Coffee Shop.

Where can you find information about the kinds of cameras you can check…

Find the database Project Muse.

What is the loan period for books for undergraduate students?

Where would you find a book at the Downtown Campus Library with a call…

See if study room 1028 is available at 5pm tomorrow.

Find a peer-reviewed article on global warming.

Find information about borrowing a laptop from the Downtown Library and…

Where can you find information about how to print in color?

I need help writing citations in APA.

Can you find Martin Dunlap’s phone number on the WVU Libraries website?Usability Test 2 - Overall TimeUsability Test 1 - Overall Time

CROSSOVER TASKS: TIME COMPARISON (-) * Average time includes delivery of tasks.

CROSSOVER TASKS: TIME COMPARISON (+/=)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Find out how to renew a book online

You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it?

Usability Test 2 - Overall Time Usability Test 1 - Overall Time


NEW TASKS: AVERAGE TIME

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library?

How would you submit a request for an appointment for with a librarian forresearch assistance?

Find the business database “MRI Mediamark Reporter.”

Can you use the website to find the journal of Aircraft?

Find a call number for a copy of “Gone With the Wind” on video.


1ST OVERALL TASK:SUCCESS

Completed

Completed with difficulty

Completed with ease

Almost Didn’t Completed

* Task 1 is not included on this graph. Individual solutions to Tasks do not qualify as failure. See individual tasks for more details.

2ND OVERALL TASK:SUCCESS

* Task 1 is not included on this graph. Individual solutions to Tasks do not qualify as failure. See individual tasks for more details.

Completed

Completed with difficulty

Completed with ease

Almost Didn’t Completed

TASK 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library?

Other Destinations:- 30% went to Collections

Student Observations: - To students Collections is vague and encompasses what the

library has. Exposition on this page as to what the library has, and what to do if we don’t have something is expected.

- A large percentage of students wouldn’t borrow from another library, but would simply get the book from Amazon.

Video:https://testing.lib.wvu.edu/usability2/Task2.wmv

https://testing.lib.wvu.edu/usability2/Task2.wmv

TASK 3: Find today’s hours for Eliza’s Coffee Shop.

Other Destination:- Services

Student Observations: - A lot of students skipped over Hours in About Us because there

are too many items. Seven items would be more appropriate.



TASK 4: Where can you find information about the kinds of cameras you can check out from the library?

Other Destinations:- About Us- Instruction

Student Observations: - Multimedia Equipment may be too vague. Framing this as ‘what you

can check out’ , or ‘equipment rental’ might be more appropriate.- 40% of weren’t aware that you could check out a camera.



TASK 6: What is the loan period for books for undergraduate students?

Other Destinations:- About Us

Student Observations: - 20% of students expect to find loan information under

Policies. Perhaps cross-listing would be beneficial.



TASK 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D?

Other Destinations:- 20% went to Services- 40% went to Collections

Student Observations: - A lot of students don’t know what Call# are. The way maps and

Call#s are rendered could be better framed.- 20% of students expect the Call#s link on the search box to be call

number search.



TASK 14: I need help writing citations in APA.

Other Destinations:- 60% went to Instruction

Student Observations: - Students are expecting to find citation help on the first page of

Instruction. I would suggest doing so, and restructuring Instruction.



TASK 15: Find the business database “MRI Mediamark Reporter.”

Student Observations: - Database side navigation is very confusing. The web application

needs to be reworked.- Misspellings for databases yield no suggestions. Showing

responses as the user searches is more preferable.



TASK 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it?

Other Destinations:- 30% went to Instruction- 30% went to Collections

Student Observations: - Students don’t know the difference between Course Reserves, LibGuides,

and Study Guides… this causes a great deal of confusion.- There doesn’t seem to be a clear and present way to get to Course Reserves,

LibGuides, and Study Guides when the student isn’t on the homepage.In general, Instruction as a primary navigation item needs to be rethought with a focus on what is beneficial to the primary target audience.



SYSTEM USABILITY SCALE (SUS) QUESTIONS1. I think that I would like to use this system frequently2. I found the system unnecessarily complex 3. I thought that the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system

1ST SUS: RESULTS

4

4

5

1

5

4

4

3

1

6

3

2

1

2

1

1

2

2

2

1

7

7

8

1

3

1

6

1

3

2

1

4

2

0 1 2 3 4 5 6 7 8 9 10

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able to use this system

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system very quickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with this system

Strongly Disagree Disagree N/A Agree Strongly Agree

2ND SUS: RESULTS

1

4

2

6

1

5

2

5

2

5

1

5

2

2

2

1

3

2

1

1

2

1

1

2

3

5

1

3

1

1

4

1

4

1

3

1

3

5

2

3

0 2 4 6 8 10

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able to use this system

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system very quickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with this systemStrongly Disagree Disagree N/A Agree Strongly Agree

USABILITY SUGGESTIONS

2ND TESTING RESULTS:

Common Usability Mistakes:- Students by default still treat the search box as a site search

only.

- Several students treat the large search box as a page filtering device. This is mainly due to its size and being on every page.Redesigning this to be more like a search box, or with ‘site search’ first might be more beneficial. Also, a large search box on the homepage with a condensed search box and the back page is preferable.

- Journal spelling suggestions or persistent search suggestions would be beneficial.

Usability Suggestions:- Collections - Exposition on this page as to what the library has,

and what to do if we don’t have something is expected.

- About Us - Compartmentalize; seven items would be more appropriate

- Multimedia Equipment - Framing this as ‘what you can check out’ , or ‘equipment rental’ might be more appropriate.

- Policies/Loan Periods - Cross-listing would be beneficial.

- Maps/Call#s - The way maps and Call#s are rendered could be better framed.

- Restructuring Instruction: Students don’t know the difference between Course Reserves, LibGuides, and Study Guides.

- Databases/Journals - Showing responses as the user searches is more preferable.

FIXABLE/ IN PROGRESS (BASE CAMP):

Usability Changes 1- Web Application - Have Call Number/Floor Maps link

directly from the catalog. - Templates for apps (directory, hours, etc.) still have old

WVU header links.

Library Website 8.1- Floating library header/navigation- Search website within WVU Library website- Homepage screen autofits to monitor size / tablet and

mobile navigation more clean- Redesign Collections

WebApps- Hours- Directory- Collections

Fixes- Edit 404 errors or dead ends to show the search query and

provide options- Add loading wheels to applications or media that require

significant bandwidth.

Future Testing- Experiment with remote un-moderated usability testing- Conduct mobile website usability testing (Mr Tappy)

UX ENHANCEMENTS

With more people using mobile android devices than desktop computers, we must consider:- How web applications, versus websites, versus libraries, versus

digital collections are all streamlined on the web through:- New Branding- Navigational Hierarchy- Responsivity- Sub-Branding- Universal Iconography

Guides / Policies- Content Style Guide- Media/Event Promotion Guide- Website Policies- WVU Library Pattern Lab

Mobile/Tablet Experiences - Condense center headers to one-block accordions.- Facets and filters on mobile should not disappear/move, require

instructions, or use icons only; they need to be apparent, and there need to be several indicators that ‘taps’ have been successful (color blocking works too).

- Explore and use hamburger-left, logo center, and search-right, navigation.

- WebApp tab bars- Test character-limits on facets.- Refining focus and functionality

UX ENHANCEMENTS THOUGHTS:

WVU Libraries Website Study

Education

Transcript of WVU Libraries Website Study