How Users Conduct Research Using Discovery Systems: Results of a Usability Study

How Users Conduct Research Using Discovery Systems: Results of a Usability StudyKelsey BrettDiscovery Systems LibrarianUniversity of Houston Libraries

1

Discovery System RedesignBegan in Fall of 2014

Discovery Redesign ActivitiesPrimo Sandbox at time of usability testingComparative analysis - Fall of 2014

Focus groups with service desk staff - Fall of 2014

Usability Testing - Spring of 2014

Usability Study: MethodologyReceived IRB approval prior to study Exempt category

Pop-Up Usability Study in the library atrium

Alternated between two version of the tests, each with two tasks - averaged test took 10 minutes

Caller, Facilitator, and Note taker Facilitator and Note taker alternated roles

Incentivized participation with library branded giveaways and snacks

Recorded the audio and screen of each session using Morae Usability Software

Usability Study: Test Version 1Task 1

You are trying to find an article that was cited in a paper you read recently. You have the following citation:

Clapp, E., & Edwards, L. (2013). Expanding our vision for the arts in education. Harvard Educational Review, 83(1), 5-14.

Please find this article using OneSearch

Task 2

You are doing a research project on the effects of video games on early childhood development. Find a peer-reviewed article on this topic, using OneSearch.

Usability Study: Test Version 2Task 1

Recently your friend recommended the book The Lighthouse by P.D. James. Use OneSearch to find out if you can check this book out from the library.

Task 2

You are writing a paper about the drug cartels influence on Mexicos relationship with the United States. Find a newspaper article on this topic, using OneSearch.

Usability Study: Participation 15 Participants15 undergraduate students4 Freshmen1 Sophomore7 Juniors2 Seniors1 graduate student1 post-baccalaureate student

Disciplines represented11 participants majoring in Sciences2 majoring in Business2 majoring in CommunicationsNo participation from Humanities students

This did was not a great representative sample, however the benefits of doing the study still out weigh the imperfections. 7

Results at a GlanceTest 1, Task 1 ParticipantTime on TaskTask Completion11m 54sY24m 13sY31m 26sY41m 17sY51m 26sY (required assistance)61m 43sY71m 27sY8 1m 5sY

Test 1, Task 2ParticipantTime on TaskTask Completion13m 44sY22m 21sY35m 23sY (required assistance)42m 5sY53m 32sY62m 45sY73m 8sY83m 1sY (required assistance)

Test 2, Task 1ParticipantTime on TaskTask Completion11m 7sY256sY3No recordingY42m 21sY51m 8sY62m 14sY71m 15sY

Test 2, Task 2ParticipantTime on TaskTask Completion14m 45sY (required assistance)259sY3No recordingN47m 47sY52m 52sY61m 33sY71m 30sY

8

Users do not understand library or academic jargon-Terms such as Peer-reviewed, Digital Library, and citation cause confusion

9

Pre-search filtering options-Users appreciate them, and except more of them

Users modify their search terms-If relevant results do not appear on the first page

Users do not easily distinguish between scholarly and non-scholarly resources-When asked to find a newspaper article, users did not distinguish them from other types of resources

Redesign DecisionsRemoved UH Digital Library from Drop-down menu and added Articles +

Changed Peer-reviewed journals facet to Peer-reviewed articles

Tabs were converted into buttons and renamed

Facets were re-ordered to promote most useful limiters

Configured the title link to go directly to full-text or catalog record

More white space incorporated into display

Future Possibilities Improve integration with stacks guide for location information

Remove Newspaper articles?

Incorporate instructional information to improve searching

Create search scopes for specific collections

Possibilities for improvements on vendor sideImprove relevancy ranking

Improve autofill and did you mean functionalities

Support natural language searching

Improve integration with competing ILS and ERM systems

Interface prior to RedesignMay 2014 May 2015

Redesigned InterfaceMay 2015 Present

THANK YOU!Kelsey Brettkrbrett@uh.edu