Sample Usability Study

7/31/2019 Sample Usability Study

1/26

Usability StudyYouth: Ages 12 16

Conducted March 2011

Conducted by:Caitlin Anderson Karen King

[email protected] [email protected]

Lamont PearsonLeah Davies [email protected]@gmail.com

Brittany [email protected]

Joe Famighette

[email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


2/26

Table of Contents

I. Introduction ........................................................................................................3II. Methodology .....................................................................................................5III. Part 1. Demographics Results .........................................................................7

Figure 1. Respondent Demographics and Internet Use ....................................7Figure 2. Frequency of Internet Use..................................................................8Figure 3. Purpose of Internet Use .....................................................................8

IV. Part 2. Usability Assessment Results ..............................................................9Figure 4. Task One: Library Information (Contact) ............................................9Figure 5. Task Two: Library Services..............................................................10Figure 6. Task Three: Catalog and Book Bag .................................................11Figure 7. Task Four: Calendar.........................................................................11Figure 8. Task Five: Catalog and/or Databases ..............................................12Figure 9. Task Six: Student Support................................................................13Figure 10. Task Seven: Catalog and Multimedia.............................................13

V. Part 3. Exit Question Results..........................................................................14Figure 11. Ease of Tasks ................................................................................14Figure 12. Site Preferences.............................................................................14

VI. Analysis .........................................................................................................15VII. Recommendations........................................................................................16Appendix A. Consent and Waiver Form..............................................................19Appendix B. Survey Part 1..................................................................................20Appendix C. Survey Part 2..................................................................................21Appendix D. Survey Part 3..................................................................................24Appendix E. Initial Distribution Email ..................................................................25Appendix F. Reminder Email ..............................................................................26

2


3/26

I. Introduction

The Guntersville Public Library is located in Guntersville, a community of over7,000 people in Marshall County, Alabama.

As of the 2000 census, the community is comprised of 88.17% White, 8.53%Black, 2.87% Latino, and less than 1% Native America, Asian and PacificIslander. The community includes over 3,000 households and nearly 2,000families, with 25.7% households including at least one child under the age of 18.22.4% of the population is under the age of 18. The median household income is$29,882 and the median family income is $39,464, leaving 11.2% of families and14.2% of the population below the poverty line. The target audience of theGuntersville Public Library website is this community as a whole; this study,however, focuses on how the website serves the youth demographic, definedherein as those persons aged 12-16 years.

The Guntersville Public Library (and its website by extension) is a single centrallibrary within the Marshall County Library Cooperative system. It offers nine adultpublic internet terminals, three childrens internet terminals, one Macintoshterminal, and free WiFi service. 17, 622 users are registered to borrow librarymaterials and a library card is required for access to computers, databases, andthe catalog. The library serves between 1,200 and 1,800 patrons each month.

Of these users, approximately 5,000 are under the age of 18. Patrons that fallinto the youth demographic have access to a variety of library resources throughthe librarys website, including the Alabama Virtual Library and its databases,Accelerated Reader Programs, e-books, TumbleBook Library, Homework

Alabama, and Rosens Teen Health & Wellness website.

The purpose of the Guntersville Public Library website(http://www.guntersvillelibrary.org/Default.aspx) is to serve as an extension, or adigital branch, of the library by providing access to library resources, services,and general information. For example, users may consult the site for informationabout upcoming library events (such as yoga or story hours), to gain access toan online catalog of library materials, and to conveniently obtain a list ofresources and services available to patrons. According to Library Director BethWheeler Dean, the website will eventually offer increased functionality andinteractivity (book reviews, comments, blogs, online reference). It is the intent of

the Guntersville Public Library to increase services for the communitys youthfulpatrons in the future. At present, the Guntersville Public Library website seeks toaccomplish its goals by maintaining a site of the following format: a navigationmenu with several access points or tabs (tabs include Home, About, News,Services, Calendar, Contact, Catalog, and OverDrive). In addition to this menu,links to various resources (such as AVL) appear on the homepage.

3
http://www.guntersvillelibrary.org/Default.aspxhttp://www.guntersvillelibrary.org/Default.aspx


4/26

The goals of the usability study are to assess the ease with which youth canretrieve information from the Guntersville Public Library website, identify aspectsof the website that work well and those that are troublesome, and offerrecommendations for the existing site in the form of modifications or additions,such as individual age level pages with relevant content.

4


5/26

II. Methodology

The usability study was distributed to 24 people, aged 12 to 16. The decision tofocus on ages 12 through 16 was partly based on the expected use of thewebsite, as well as the problematic nature of the extremely varied skill sets and

intellectual capacities that occur across the differing stages of humandevelopment (e.g. a five-year-old does not have the same skill set or intellectualcapacity as a person in the 12-16 age range). Users were expected to have theskills necessary to conduct the survey and tasks independently without oversightof a parent, staff member, or the administering group. Members of the age groupselected possess these necessary skills. The language of the survey wasconstructed specifically with this audience in mind, endeavoring to convey acasual, informal tone and ensure that the vocabulary level was appropriate forthis age group.

The survey was originally intended to target 15 actualusers of the Guntersville

Public Library website. However, due to an extremely low return rate amongthese self-identified participants, the survey audience was expanded to includean additional 9 users, within the correct age demographic, living in areas outsideof Guntersville, Alabama. A sample authorization statement was drafted toinform users of the intent, risks and benefits of participating in the survey,although participants were not required to complete this document in conjunctionwith the study (Appendix A).

The survey was conducted online via e-mail and a GoogleDocs survey. In an e-mail users were given instructions and provided with a link to the 33-questionassessment survey. Users were instructed to open the Guntersville Public

Librarys website in a separate or adjacent window while taking the survey toallow for immediate recording of responses. The survey itself was comprised ofthree sections:

1. Part 1 generated demographic information, including name, age, sex,years using the internet, frequency of internet use, primary purpose ofinternet use, and past use of the Guntersville Public Library website(Appendix B).

2. Part 2 offered 17 quantitative and qualitative assessments associatedwith the execution of seven tasks using the library website. Participants

were asked to initiate each task at the homepage to ensureconsistency and better identify navigation challenges. Participantswere instructed to spend no more than two minutes on the execution ofeach task and reminded that the tasks are intended to assess thefunctionality of the website rather than the users expertise or ability(Appendix C). After each task was completed, space was provided inwhich participants were asked to describe steps taken to complete said

5


6/26

tasks, as well as to designate the ease or difficulty with which the taskwas accomplished.

3. Part 3 was comprised of seven exit questions intended to collectgeneral feedback about use of the website, including ease of use,

preferred sections, and navigation (Appendix D).

Initially, each member of the administering group sent the survey to twoparticipants via email outlining the expectations and week-long response period(Appendix E). A reminder email was distributed to the original 15-memberaudience by one group member three days before the survey deadline (AppendixF). Due to the low response rate, group members distributed the original emailand survey to the 9 additional participants after the audience was broadenedbeyond current Guntersville Public Library users.

6


7/26

III. Part 1. Demographics Results

Of the 24 surveys distributed, 10 were returned, which resulted in a return rate of42%. Six of the survey respondents were female and four were male. Theaverage age of respondents was 14 years old, with three aged 12 years, one

aged 13 years, one aged 14 years, two aged 15 years, and three aged 16 years.The average years of internet use were 4.8 years, with a majority of respondentsindicating 5 years of internet use (Figure 1).

Figure 1. Respondent Demographics and Internet Use

60% of the respondents use the internet daily. 20% use the internet on a weeklybasis, and 20% of respondents use the internet only occasionally (Figure 2).Approximately 65% of respondents use the internet for games and entertainmentas well as social networking. This was followed by homework usage at 28%, and

just over 6% of respondents use the internet for shopping (Figure 3). Due to thechange in the survey audience, only one participant had used the GuntersvillePublic Library website in the past for reading eBooks. All other respondents hadnever used the site previously, though many have had access to public library

websites in their respective areas.

7


8/26

Figure 2. Frequency of Internet Use

Figure 3. Purpose of Internet Use

8


9/26

IV. Part 2. Usability Assessment Results

The first task instructed respondents to determine the hours of operation for theGuntersville Public Library on Saturdays (Figure 4). 10% of respondents did notanswer, 10% of respondents were unable to find the correct information, and

80% of respondents were able to identify this information within the 2-minutewindow. 60% of respondents found the task very easy, 20% found the task easy,10% found the task neither easy nor difficult, and 10% found the task difficult.

Figure 4. Task One: Library Information (Contact)

The second task instructed respondents to identify and name a special serviceavailable to youth, described as kids and teens (Figure 5). 60% ofrespondents were able to identify a service available to youth, 20% ofrespondents were unsure if these services were available, 10% of respondentswere unable to identify a youth service, and 10% of respondents did not respondto this question. 40% of respondents identified one service available, 30%indicated that no services were available, 20% of respondents identified twoservices available, and 10% of respondents identified three youth servicescurrently available. Of the services identified, the most common service noted

was Homework Help, with 50% of respondents identifying this service. StoryHour and Learning CDs were each identified in 30% of responses, ChildrensComputers were identified in 20% of responses, and Teen Health wasidentified in 10% of the responses.

9


10/26

Figure 5. Task Two: Library Services

Tasks Three asked respondents to identify a book on killer whales. If they wereable to do this, they were then asked to place the book in their Book Bag

(Figure 6). 90% of respondents were able to complete these tasks. 50% ofrespondents indicated that Task Three was easy, 20% found the task neithereasy nor difficult, 20% found the task difficult or were unable to complete thetask, and 10% of the respondents found the task very easy. Using the BookBag was considered very easy by 60% of respondents, with 20% rating the taskas easy, and 10% of respondents being unable to complete the task.

10


11/26

Figure 6. Task Three: Catalog and Book Bag

Task Four instructed respondents to identify the date of the last pre-school storyhour in the month of March (Figure 7). 50% of respondents were able to correctlyidentify this information, 40% of respondents were unable to identify thisinformation, and 10% of respondents did not provide an answer. 60% of

respondents found the task very easy, 20% found the task easy, and 10% foundthe task neither easy nor difficult.

Figure 7. Task Four: Calendar

11


12/26

Task Five required participants to identify a print source and a non-print sourceon the Salem Witch Trials (Figure 8). 60% of respondents were unable to findeither resource, 30% of users were only able to identify a print source, and 10%of respondents were able to identify both a print and non-print source. 50% of

respondents indicated they were unable to complete this task, 30% found thetask easy, 10% found the task very easy, and 10% found the task neither easynor difficult.

Figure 8. Task Five: Catalog and/or Databases

Task Six asked respondents to identify what assistance is available to studentswriting a paper (Figure 9). 50% of respondents were unable to find anyassistance, 10% found contact information for staff, 10% found the AcceleratedReader and Reading Counts program, 10% found the homework page, and10%found the Learning Express link. 50% found the task difficult and were unable tocomplete it, 30% found the task very easy, and 20% found the task easy.

12


13/26

Figure 9. Task Six: Student Support

Task Seven instructed respondents to determine whether The Wizard of OzDVD is available through the library (Figure 10). 90% of respondents were ableto correctly determine whether the item was available. 60% of respondents foundthe task very easy, 30% of respondents found the task easy, and 10% ofrespondents were unable to complete the task.

Figure 10. Task Seven: Catalog and Multimedia

13


14/26

V. Part 3. Exit Question Results

User feedback indicated that Tasks Five and Six were the most difficult, whileTasks Three, Five and Seven were the easiest (Figure 11). The favorite featureof the website was the catalog and the least favorite was the services section;

however, the majority of respondents were unsure or did not indicate preferencesregarding the site (Figure 12).

Figure 11. Ease of Tasks

Figure 12. Site Preferences

14


15/26

VI. Analysis

The analysis has identified some usability challenges for users age 12 through16.

Respondents indicated that identifying services specifically geared towards youthwas challenging. This was demonstrated in Task Two and Task Six, whichinstructed users to identify youth services and highlight resources available toassist with paper writing. One female respondent, age 16, noted, The site coulduse a separate link for kids/teen programs rather than posting the links on thefirst page. She suggested that all services for youth be located on a distinctpage dedicated to this audience. A female respondent, also age 16, indicatedthat Services was the most dissatisfying section of the site(http://www.guntersvillelirbary.org/services/services.aspx). She stated, It doesntdescribe each of the services in enough detail and the times/calendars are notassociated with it.

The calendar application also proved difficult for users, as nearly half ofrespondents were unable to correctly identify the last pre-school story hour inMarch when asked to do so for Task Four.

Respondents returned mixed results with catalog and database searching. While50% of users indicated that catalog use was the easiest website feature to use,50% also found Tasks Five to be the most challenging, which endeavored toidentify catalog and database searchability. A large majority of respondents wereable to successfully complete Task Three and Seven, the other catalog searchtasks.

When discussing the easiest tasks to complete, a number of respondentsindicated that ease was determined by the task that required the fewest stepsand/or clicks. Tasks rated as easiest also appeared to correlate to the sectionsof the website that were preferred by respondents, as well as the tasks that werecompleted most successfully.

15
http://www.guntersvillelirbary.org/services/services.aspxhttp://www.guntersvillelirbary.org/services/services.aspx


16/26

VII. Recommendations

Based on the results of the usability study, as well as the observations of theadministering group, we propose the following recommendations as usabilityenhancements to the Guntersville Public Library website.

1. Navigation and Organizationa. Utilize drop-menu navigation. Doing so will increase the

likelihood that information will be located in the fewest number ofclicks. Take a look at Williamsburg Regional Library's drop-menunavigation: http://www.wrl.org/

b. Designate a navigation menu item for Youth Services, andconsider establishing sub-menus based on appropriate agegroups. These sections should highlight age-appropriate researchmaterials, services, and programs to better enable youth to obtain

the best information from the Guntersville Public Library website.Provide descriptions of the services, as well as any applicableschedules associated with service use.

c. Relocate Tumblebooks to the appropriate Youth Services pagewith a description of the service. Currently the Tumblebooksimage on the homepage looks like an advertisement, and itspurpose and usage instructions are unclear.

d. Move library hours of operation to the About Us or Homesection of the website. The current location under Contact Us is

counterintuitive and may make it difficult for users to identify thelibrary schedule.

e. Modify the calendar application by integrating it into the mainwebsite. Also consider providing search options, displaypreferences, and the website menu on any associated calendarpages.

f. Feature the catalog and databases prominently on the website.Consider developing a brief, clear set of instructions for use ofthese resources, as well as contact information for assistance.

g. Visit a variety of library websites for layout and navigationideas. Ann Arbor District Library's website won an ALA award:http://www.aadl.org/. For more information about why it waschosen, visit http://lorinet.info/rhodarian/?p=214. For ideas ondedicated youth pages, visit http://en.childrenslibrary.org/(children),http://www.cecil.ebranch.info/kids (children), andhttp://www.cecil.ebranch.info/teens/(teens).

16
http://www.wrl.org/http://www.wrl.org/http://www.aadl.org/http://lorinet.info/rhodarian/?p=214http://en.childrenslibrary.org/http://www.cecil.ebranch.info/kidshttp://www.cecil.ebranch.info/teens/http://www.cecil.ebranch.info/teens/http://www.cecil.ebranch.info/kidshttp://en.childrenslibrary.org/http://lorinet.info/rhodarian/?p=214http://www.aadl.org/http://www.wrl.org/


17/26

2. Visual Elements

a. Increase contrast within website visual elements, such as textboxes and navigation components. The current light blue boxesare difficult to see and fail to offset website sections.

b. Ensure consistency between pages, particularly with headingsand text. Specific fonts should be attributed to specific types of textand these fonts should be applied uniformly throughout the site.This assists users in visually orienting themselves to the content ona given page, and also enhances the readability of the page.

c. Identify relevant images and animation for the site whileavoiding irrelevant or distracting visual content. The animatedtext on the site banner is inconsistent and does not appear toprioritize relevant library services. Similarly, images areoccasionally irrelevant to the content they are associated with,

which may prove confusing for some users. Finally, imageplacement, especially in the case of certain linked images,occasionally appears similar to an advertisement rather than animportant website feature.

d. Aim for contrast, repetition, alignment, and proximity. For aclear and concise description of this concept, read this documentcreated by Pacific Lutheran University:http://www.plu.edu/~scotttj/designkiosk.pdf

3. Contenta. Add free interactive content to pages once Youth Services

page and sub-menus have been established. A number ofgames and educational recourses can be found at PBS Kids(www.pbskids.org), Storyline Online(http://www.storylineonline.net/), and Kids Yahoo(http://kids.yahoo.com/).

b. Consider integrating Web 2.0 technologies, such as Facebookfor Teen Services, blogs, and text or instant messagereference assistance. These resources will enhance the youthexperience and provide additional support for user needs. In thefuture if Guntersville Public Library opted to host teen Wiicompetitions, the library website would be used to announcetournaments and present press releases for the winners. (They lookgood on grandmothers refrigerator and help increase publicsupport.)

c. Create a Frequently Asked Questions page. Questions/answersshould include tips on searching the catalog, placing items on hold,

17
http://www.plu.edu/~scotttj/designkiosk.pdfhttp://www.plu.edu/~scotttj/designkiosk.pdfhttp://www.pbskids.org/http://www.storylineonline.net/http://kids.yahoo.com/http://kids.yahoo.com/http://www.storylineonline.net/http://www.pbskids.org/http://www.plu.edu/~scotttj/designkiosk.pdf


18/26

finding/using online resources, writing papers, and citing sources.For more ideas on what to include, poll library staff members.

d. Create an active Services page. Currently the information onthe Services page is static and not in a recognizable order. Group

like-items together under appropriate headings. Example: Create aheading for Genealogy, listing the appropriate services underneath(workstation, census records, etc.). Where possible, create links.For instance, link LearningExpress to its database.

18


19/26

Appendix A. Consent and Waiver Form

The University of AlabamaCONSENT AND WAIVER

TO WHOM IT MAY CONCERN:

I hereby grant full permission to the University of Alabama and the GuntersvillePublic Library to take notes of my comments during the usability test forGUNTERSVILLE PUBLIC LIBRARY Web Site.

I understand that other University of Alabama and Guntersville Public Libraryemployees involved with GUNTERSVILLE PUBLIC LIBRARY may review theseusability notes.

I understand that I may be quoted directly in internal University of Alabama and

Guntersville Public Library documents, with or without reference to my name.

I understand that usability notes, my name, address, phone number and e-mailaddress will not be shared with external third parties in compliance with theUniversity of Alabama and Guntersville Public Library Privacy Policies.

This consent and waiver will not be made the basis of a future claim of any kindagainst the Guntersville Public Library, the University of Alabama and any of itsagencies.

IN WITNESS WHEREOF I have hereunto set my hand and seal this

_________________ day of ____________________ A.D. 20___.

Name: _________________________________________________

Signature :______________________________________________

Address : ___________________________________________

___________________________________________

___________________________________________

Parent/Guardian Signature: _________________________________

Witness: ____________________________________

Date: _______________________________________

19


20/26

Appendix B. Survey Part 1Required fields are indicated by an asterisk (*).

1. Name:*

2. Sex:* Female Male

3. Age: * 12 13 14 15 16

4. Years Using the Internet (Approximate): *

5. How often do you use the Internet? * Daily Weekly Monthly Occasionally Never

6. What do you usually do on the Internet? * Examples: e-mail; homeworkresearch; play games; talk with friends; social networking; watch TV shows

7. Have you used the Guntersville Public Library website in the past? * Yes No Im not sure

8. If yes, for what purpose? Examples: looking up books, events, games, etc.

9. Can we contact you for more information in the future? If yes, please provideyour e-mail address. Your information will not be sold or given to outside entities;it is for internal use only.

20


21/26

Appendix C. Survey Part 2Required fields are indicated by an asterisk (*).

INSTRUCTIONS: Perform the following tasks to the best of your ability and thenanswer the questions. Only spend 2 minutes on each question; if you can't find

the answer within 2 minutes, explain any problems you had and move on to thenext question. These questions are testing the Guntersville Public Librarywebsite, not the person answering the questions.

1. What hours is the Guntersville Public Library open on Saturday? * Include thesteps it took you to find this answer.

2. Finding this answer was: * Very Easy Easy Neutral

Difficult Very Difficult I could not find the answer.

3. Return to the homepage. Does the library have any special services oractivities just for youth (kids & teens)? If so, name one. * Include the steps it tookyou to find this answer.

4. Finding this answer was: * Very Easy Easy

Neutral Difficult Very Difficult I could not find the answer.

5. Return to the homepage. Does the library have any books about killer whales?If so, name one. * Include the steps it took you to find this answer.

6. Finding this answer was: * Very Easy Easy

Neutral Difficult Very Difficult I could not find the answer.

7. Return to the homepage. If you were able to find a book on killer whales, put itin your "bookbag." * Include the steps it took you to find this answer.

21


22/26

8. Putting the book in your "bookbag" was: *

Very Easy Easy Neutral Difficult

Very Difficult I wasnt able to find a book on killer whales. I found a book on killer whales, but I couldnt put it in my bookbag.

9. Return to the homepage. What is the date of the last pre-school story hour inMarch? * Include the steps it took you to find this answer.

10. Finding this answer was: * Very Easy Easy Neutral

Difficult Very Difficult I could not find the answer.

11. Return to the homepage. You are doing a report on the Salem Witch Trialsfor English class. Look for ONE print source (like a book) and ONE non-printsource (like a movie). * Include the steps it took you to find this answer.

12. What were you able to find? * A print source A non-print source Both Neither Im not sure.

13. Finding this answer was: * Very Easy Easy Neutral Difficult Very Difficult I could not find the answer.

14. Return to the homepage. You have found some sources for your SalemWitch Trials report, but you are not sure the best way to start writing a paper. Isthere anything on the Guntersville Public Library website that can help you getstarted? * Include the steps it took you to find this answer.

15. Finding this answer was: * Very Easy

22


23/26

Easy Neutral Difficult Very Difficult I could not find the answer.

16. Return to the homepage. Now that your report is done, you want to relax andwatch a movie, but you've still got witches on the brain. Is a DVD of "The Wizardof Oz" available at the library? * Include the steps it took you to find this answer.

17. Finding this answer was: * Very Easy Easy Neutral Difficult Very Difficult

I could not find the answer.

23


24/26

Appendix D. Survey Part 3Required fields are indicated by an asterisk (*).

Please give us your general impressions of the website.

1. Which question would you describe as the most difficult? Why? *

2. Which question was the easiest? Why? *

3. What is your favorite part of the website? *

4. What about the website do you like the least? *

5. Was it difficult moving from one question to the next? *

6. Any other comments or suggestions about the website?

7. Do you think you'll use the library's website in the future? If yes, for whatpurpose?

24


25/26

Appendix E. Initial Distribution Email

Subject: Guntersville Public Library Website Survey

Hi (Participant Name),

I am a graduate student at the University of Alabama. Beth Wheeler gave meyour contact information, because you said you're willing to help with theGuntersville Public Library website. We have a short survey for you to takeonline. Please click on the link at the bottom of this e-mail to get started. Fromyour responses, we can determine which parts of the website are good andwhich ones still need work. Please complete the survey by Wednesday, March16, 2011.

We appreciate you helping Beth Wheeler and us make Guntersville PublicLibrary website even better!

Follow this link for the survey:https://spreadsheets.google.com/viewform?formkey=dDh3NFlINUlQcTJGb2Z0UldqWVdmbnc6MQ

Thanks,(Administering Group Member Name)(Administering Group Member Telephone Number)

25


26/26

Appendix F. Reminder Email

Subject: Reminder: Guntersville Public Library Website Survey

Good afternoon,

Last week you received a survey to help review the Guntersville Public Librarywebsite. If you have not yet completed your survey, please do so by thisWednesday, March 16. To access the survey, please click on the link at thebottom of this e-mail.

We appreciate you helping Beth Wheeler and us make Guntersville PublicLibrary website even better! If you have any questions, please let me know, oryou can also reply to the original email you received and a member of our groupwill help you.

Follow this link for the survey:https://spreadsheets.google.com/viewform?formkey=dDh3NFlINUlQcTJGb2Z0UldqWVdmbnc6MQ

Thanks!

Brittany Turner916-384-6735

Sample Usability Study

Documents

Transcript of Sample Usability Study