Usability TestingAllison BaileyLIS 4804

The purpose of this test is to evaluate the performance of the website with regard to usability, efficiency, aesthetics and functionality. Our goal is to evaluate the website by observing test subjects while they complete scenarios which will model the anticipated natural use of the collection.

MethodologyParticipants/user profile: Our users are those people active in the disability rights, woman’s rights and LGBT rights movements, as well as researchers, educators, scholars and students.Our test participants are fellow library and information science students who are also working on creating a digital collection.

The setting of the usability test: Classroom 305 in Ruffato Hall. Subjects will be provided access to a computer or they may use one of their own.

Testing procedures and instruments: The test will be administrated to five classmates, one at a time. Subjects will fill out a consent form and be asked complete the predetermined test scenarios one at a time, using think aloud protocol while being observed by all members of the Hershey Collection Team. After they have completed the scenarios, subjects will be asked to complete a post-test questionnaire asking for their subjective evaluation of the website based on specific criteria, as well as any of their own thoughts or suggestions they wish to add.

Roles of usability facilitator and observers: Prior to the start of the test, a facilitator will provide brief instructions to complete the test. The facilitator will provide the subject with a written copy of the scenarios one at a time and in the same order. The facilitator and observers will watch the subject complete the test scenario without providing any help in completing the objective. The observers will take notes regarding the test subject's use of the website as needed. The facilitator will monitor how long each task takes the user to complete. After all subjects have completed the test observers and moderators will meet to debrief and compare observations.

Tasks:Browse through the collection for an imageContact site administrators for more information.Search the site for specific informationAccess metadata informationBrowse through the collection for a text document

ScenariosScenario 1: Contact, picture browse

The user wishes to use an image from the Laura Hershey Collection in an article he/she is writing for publication. The user must find an appropriate image to accompany the article and needs to contact someone through the website to inquire about the rights and permissions for republication of the image. Estimated time: 4 minutes

Scenario 2: Text document, browse, social bookmarkingThe user wishes to find a poem by Laura Hershey and post it to their Facebook time line. The

user does not have a specific poem in mind, but wishes to find one that appeals to him/her.

Estimated time: 5 minutes

Scenario 3: Text document, search, metadataA student is researching the Disability Caucus and its role in the 1988 presidential elections and

wishes to find something written by Laura Hershey at the time. The student will need to find information about the original publication of the document in order to cite the source. Estimated time: 3 minutes.

Performance evaluation measures: Whether the user is able to complete the task without using site documentation or help from

members of the Laura Hershey Team. Can they complete it fully, partially or not at all? Whether the user is able to complete the tasks and scenarios in the amount of time estimated for

each. A timer will be used to measure completion time. Whether the user is able to complete the tasks with less than 5 mistakes.

Subjective evaluation measures will include a post-test questionnaire completed by the subject asking them to rate the website on a number of criteria.

Subjective evaluation measures will also include notes and thoughts of the observers and facilitators after watching subjects use the website.

Indicators of problem severity: A task takes over two minutes longer than the estimated time to complete. A user verbally expresses confusion, frustration or asks for help. A user makes over four mistakes while trying to complete the task/scenario.

Report contents and presentation:The report contents will include:

Whether any test subjects met the criteria for problem severity. Whether any test subjects completed the task or scenario in the estimated time and without

making mistakes Answers to the post-test questionnaire Observations made by members of the Laura Hershey group during the test Our recommendations for any changes to be made based on our findings.