DESKTOP USABILITY TESTINGWest Virginia & Regional History Center

WVRHC DESKTOP USABILITY TEST

Problem:

We used TechSmith Morae to conduct usability testing of the West Virginia & Regional

History Center’s desktop website on Windows 7 laptop in Internet Explorer 11. This round of

usability testing was internal to WVU Libraries, utilizing undergraduate student employees.

TESTING TASKSA. Go to the WVRHC website.1. Is the WVRHC open tomorrow?2. What historical newspapers from Clarksburg are available to view online?3. Find pictures of coal miners.4. Can I order a copy of a photo I found at the WVRHC?5. Where can I find Ohio County court records?6. Does the WVRHC have historical maps from the 1600s?7. I need an obituary from a WV newspaper that the WVRHC has copies of, what do I do?8. Can I take pictures of manuscripts collections with my camera?9. Does the WVRHC have folk music recordings of Edden Hammons?10. Find a database that has documents about Civil War history.11. Find the volunteer application form.12. Does the WVRHC offer tours?

TEST PARTICIPANTS

Brandon Evan JP Rebecca Evan

OVERALL TASK:AVERAGE TIME

* Average time includes delivery of tasks.

OVERALL TASK:SUCCESS

Failed to complete

Completed with difficulty

Completed

Completed with ease

SYSTEM USABILITY SCALE (SUS) QUESTIONS1. I think that I would like to use this system frequently2. I found the system unnecessarily complex 3. I thought that the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system

SUS: RESULTS

1

1

1

1

3

4

2

1

2

1

2

2

2

1

1

1

2

2

1

1

3

4

1

2

1

3

2

1

1

0 1 2 3 4 5

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able to use this system

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system very quickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with this system

Strongly Disagree Disagree N/A Agree Strongly Agree

PROPOSED UX ENHANCEMENTS- The WVRHC needs to better market itself and increase search engine

optimization. This can be done by improving their existing Wikipedia articles (including the name in the article), developing a stronger brand identity, updating Wikipedia collection entries, developing a Pinterest page, and collaborating to have other entities link to the WVRHC.

- The WVRHC hours and WVRHC-relevant databases should dynamically render in-page, and not jump to the WVU Libraries website.

- The terms 'collections', 'core collections', and 'exhibits' confuse visitors; what needs to be used it what makes sense to users, should all be the same, and should vary based on online versus in-person.

- The Collections page requires a lot of reading and the user get's confused and lost. Users are wanting the left-navigation to be facets. Redesign.

- Remove fancy titles and sub-sub-sub-branding because users are searching for media type. Use 'Newspapers' instead of 'West Virginia Newspapers', nobody knows what 'Printed Ephemera' is, and use 'Photos' or 'Photographs' instead of 'West Virginia History OnView: Historical Photographs'.

- Users want to use the search box, but the search box doesn't work for users. They expect to see things that make sense to them, they don't, and then they either don't use the search box or want to use the WVU Libraries search box instead (because it's simpler). This is more useful for WVRHC employees. Redesign.

- On Homepage feature should take up less space, and people should see a signifier that they can scroll down, because a lot of users aren't scrolling down.

- Policies, Volunteer Form (which some people didn't find), and Tours need to be under contact us and visit; there is confusion between tours, visit, and contact us are all disparate, and don't seem connected.

- Add a Contact Us in-page form, because people are less likely to send an email to an address.

- All PDFs should be converted to text/HTML content, for accessibility purposes, and so users don't have to jump to another system.

- The Services page needs to be unpacked, and better cross-referenced within the WVRHC website.

USER VIDEOS

https://testing.lib.wvu.edu/wvrhc_usability/brandon.wmv

https://testing.lib.wvu.edu/wvrhc_usability/evan.wmv

https://testing.lib.wvu.edu/wvrhc_usability/jp.wmv

https://testing.lib.wvu.edu/wvrhc_usability/rebecca.wmv

https://testing.lib.wvu.edu/wvrhc_usability/shawn.wmv

OBSERVER COMMENTSTask A: Go to the WVRHC website.• Went to WVU Libraries site first.• Clicked on hours

Task 1: Is the WVRHC open tomorrow?• Clicked on hours immediately to get to calendar• Hours should be on WVRHC page• Went back to libraries page for hours• No issues finding hours.• Went to library hours not center

Task 2: What historical newspapers from Clarksburg are available to view online?• Went to collections, but was dissuaded from the names of

core collections, exhibits, etc.• Looked at Collections page, didn't see anything that looked

like it included newspapers• Went to databases first• Went to collections

• Searched in box/Mountainlynx• Looking for the letter N• It is one additional step to go to page• Went to search box• Search box first; then databases; searched book and

periodicals• Used Search box-- no newspapers, used books and user is

confused and lost• Would try databases after search boxes• Found Newspapers quick link, after he said he'd google.• Didn't scroll down to see newspaper link• Collections page requires a lot of reading. • Maybe just title newspapers• Didn't find papers on page• Found Newspapers page. Paused at N, before scrolling

down.• Used map.• Ended up in Montainlynx• User wants to use search

Task 3: Find pictures of coal miners.• Looking for something that says pictures• NUI started using photos once scrolled down on homepage• Used quick links again• Went to printed ephemera• Seems to be confused on what collections versus core

collections are• Photos isn't in the description for either Core Collections or

Digital Exhibits.• Not using search box• Collections first; digital exhibits; searched in page for coal

miners• User is confused between exhibits and core collections• Used in page search to get through collections... none matched• Used collections left nav as facets.• Didn't see link to collections on library collection information

page• At the end went to search in Summon• A&M found rather than webpage• Searched summon; limited to photograph; found photo• Went pass photos

Task 4: Can I order a copy of a photo I found at the WVRHC?• Went straight to On View to find photo ordering link. Could be

partially due to order of questions.• Used on view ordering• Navigated through WV History On View, rather than website.• Skipped over Photographs and Reproductions sections; Photos

vs. Reproductions is confusing. • Not going to about us or contact us for policies• I would go out to Mountainlynx• Click to order image in WV History OnView is very clear.• Used click to order image in OnView

Task 5: Where can I find Ohio County court records?• Looked at A&M Guide• Searched A&M for court records• Didn't see anything in dropdown• Thought court records might be in printed ephemera• Didn't find records.• Went to genealogy - then to county• Found Ohio County Court Records easily. Clear path.• Went to databases first• Databases first for court records; then search box-- searching

site-- success!• Searched and found in PDF

Task 6: Does the WVRHC have historical maps from the 1600s?• Looked for maps under bottom categories• Looked in links at bottom of page, then search box.• Searched under Archives and Manuscripts• Didn't go past collections page f/core /exhibits• Didn't find; Collections page is dead to him.• Would do a google search• Not scrolling down on homepage• Not using the search box• Not reading text-- skimming over large text blocks• Remove PDF on maps• Barely scrolls down on homepage until later• Looking for maps under featured resources.• Would use main page search because it's easier• Using summon to search for maps.• Would use summon to find maps, but not seeing if results are

WVRHC-specific• Remove PDF from site, just put in site

Task 7: I need an obituary from a WV newspaper that the WVRHC has copies of, what do I do?• Possibly unpack services page, or present information in a

better way• Does the newspapers page include information about

requesting obituaries?• Went to mountainlynx• Searched library website• Misunderstood task-- looking for obituaries rather than

requesting.• This question may not have been understood-- looking for

obits, rather than how to request them.• Would request under services• Want to get himself instead of requesting a copy something

similar to request a copy' (I think he means the photo)• Would go to contact us• Went to copies an reproductions• Question issue-- we didn't specify that you couldn't come in

and ask for a copy of the obituary.• We need to specify if it physical or not• They would still try to find it online, even if it is only in person• Searching Summon• Jumped out to WVU Libraries site, then back.• Another person looking for obituary info on Newspapers page.

Task 8: Can I take pictures of manuscripts collections with my camera?• Searched in A&M• Clicked on A&M Guide looking for information about

photographing manuscripts.• Contact about photos of manuscripts-- didn't look on site.• Fixing branding will help eliminate About Us confusion• User wants to search library since the WVRHC is part of the

webpage• About us policies. Searched in page.• Looked under policies• Didn’t think to go to policies

Task 9: Does the WVRHC have folk music recordings of Edden Hammons?• Used links at bottom of homepage.• Need a contact form on the WVRHC website.• Used Featured Resources list on home page.• Used leftnav on Collections page to get to core collections

Task 10: Find a database that has documents about Civil War history.• Jumped to databases app• Antebellum • Antebellum• Found Databases easily.• Civil War Antebellum• Jumped out to WVU Libraries database page; then back

before finding it.• Antebellum

Task 11: Find the volunteer application form.• Went to contact us for the volunteer form• I would move a sub navigation there of volunteering• Went to contact us• Went to Contact Us for Volunteer Form first, but About us

was second choice.• I would put the volunteer info on contact/cross-list• Hovered over About us, but then clicked on Teaching and

Tours• Went to teaching and tours• Went straight to About us page.• Did not find application

Task 12: Does the WVRHC offer tours?• Looked for tours under Visit.• Went to visit for tours, but no information on the tours page• Went under contact us and about us first• I would cross list tours under visit section• Gave up after visiting Visit page and not finding.• Used links at bottom of home page.• Broken link-- links at bottom of home page aren't linked with

new Services page• Instruction tours and research assistance link is broke• Didn’t see tours in left navigation under services - unpack• Toggling back and forth between about us and services.

Looking in Visit for tour info.• Tour information is not under visiting

STANLEY MARCUS

“There are only two things of importance. One is the customer, and

the other is the product. If you take care of the customer, they come

back. If you take care of product, it doesn’t come back.”