Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

15
The Unobtrusive “Usability Test”: Creating Measurable Goals to Evaluate a Website Tabatha Farney Web Services Librarian University of Colorado Colorado Springs Paper at: http :// www.ala.org/acrl/sites/ala.org.acrl/files/content/conf erences/confsandpreconfs/2013/papers/Farney_Unobtrusiv e.pdf Presented at the 2013 ACRL Conference.

description

Presented at the 2013 ACRL Conference. Full paper available at: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2013/papers/Farney_Unobtrusive.pdf

Transcript of Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Page 1: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

The Unobtrusive “Usability Test”: Creating Measurable Goals to Evaluate a Website

Tabatha FarneyWeb Services LibrarianUniversity of Colorado Colorado SpringsPaper at: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2013/papers/Farney_Unobtrusive.pdf Presented at the 2013 ACRL

Conference.

Page 2: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Usability is a constant process

How can we keep up with the pace of changing library users?

Page 3: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

What is Web Analytics?

The process of collecting, reporting, and analyzing website usage data to reveal how users are actually interacting with a website in order to make improvements.

Page 4: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Common Web Analytics Metrics

Basic Usage Engagement Goals

Visits Visitors Page Views

Duration of Visit Page Depth Bounce Rate

Conversions

…and the list goes on…

Page 5: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

All that data is gather seamlessly into a web analytics tool without

bothering the end user.

Page 6: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Usability Testing Web Analytics

“Validation” Testing

+

=

Page 7: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Validation testing takes place after the website is finished and evaluates that the site meets its desired goals based on set benchmarks.

Requires a large group of users and simple usability benchmarks.

Page 8: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Applying Web Analytics to Usability Testing

Usability Process*

Applying WA

1. Understand test’s purpose and goals Unchanged

2. Determine the research questions Unchanged

3. ID a target audience Website users (can further segment this group)

4. Select the tasksTasks must be measurable in website use data.

5. Define the usability metrics and reports

Select the web analytics metrics and reports.

6. Report the findings Unchanged* Modified from Rubin and Chisnell (2008).

Page 9: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Metric: Page Views

Report: Reverse Goal Path

Metric: Average Time on Site (average); Visit Duration (mean)

Report: Audience Overview; Audience Engagement

Metric:

Report: Goal w/ custom segment

Task 1: Find a research database within four web pages of the visit.

Identifying Usability Metrics

Metric: Conversion

Report: Goal w/ custom segment

Task Completion Success Rate

Successful ConversionsTotal Conversions

Common Pathways

Time to Complete Task

Page 10: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Example ReportSample report in Google

Analytics

Page 11: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

ReportingSeptember – November 2012

  Task 1Total Conversions 10154  Total Successes 6778  Total Failures 3376    Success Rate 66.84%  Average Time on Site 0:09:40Visit Duration   0-10 seconds 2187  11-30 seconds 2132  31-60 seconds 798  61-180 seconds 1153  181-600 seconds 1343  601-1800 seconds 1587  1801+ seconds 954

Any success under 65% triggers the error analysis process.

Page 12: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Drawbacks

Error analysis is a long process

Inability to track individual users

Only data from website users

Data focuses on actions, not motivations

Drawbacks & Benefits

Benefits

Easy to regularly assess website usage

Simple to implement Many WA tools have

automatic export

Can be combined with other usability efforts

Many libraries already use a WA tool

Page 13: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Keep test goals and

tasks manageable.

Invest in error analysis and

reporting steps.

Reuse previous

usability tests to guide goal/task creation.

Test the test.

Recommendations for You

Page 14: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Summary• Apply the structure of

usability testing to your website use data

• Make your tasks measurable using website use data

• Be prepared to report the data regularly

• Error analysis is an essential process

• Make changes based on the data and test again!

Page 15: Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website

Images Used

“Students Moving through HUB,” pennstatenews, Flickr, http://

www.flickr.com/photos/pennstatelive/4950729405 (slide 2).

“Web Analytics Framework,” Beantin webbkommunikation,

Flickr, http://www.flickr.com/photos/beantin/7649183772/

(slide 3).

“Peanut Butter,” sparktography, Flickr,

http://www.flickr.com/photos/sparktography/410446108/ (slide

6).

“Kshocolat Dark Chocolate Bar,” Boz Bros, Flickr,

http://www.flickr.com/photos/sparktography/410446108/ (slide

6).

“Giant Reese’s Peanut Butter Cup,” camknows, Flickr,

http://www.flickr.com/photos/camknows/8265569228/ (slide

6).

“Labs are Full!” clemsonunivlibrary, Flickr,

http://www.flickr.com/photos/clemsonunivlibrary/6096116865/

(slide 7).

“Thumbs Up,” P1r, Flickr,

http://www.flickr.com/photos/p1r/4730426448/ (slide 14).

ResourcesCitations for Slides

Barnum, Carol M. Usability Testing Essentials: Ready, Set—Test! Boston: Morgan Kaufmann Publishers, 2011.

Farney, Tabatha and Nina McHale. Web Analytics Strategies for Information Professionals: A LITA Guide. Chicago: ALA TechSource, 2013.

Nielsen, Jakob and Hoa Loranger. Prioritizing Web Usability. Berkeley: New Riders, 2006.

Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 2nd ed. Indianapolis: Wiley, 2008.

Tullis, Tom and Bill Albert. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Boston: Morgan Kaufmann Publishers, 2008.

Turner, Steven J. “Demonstrating Success: Web Analytics and Continuous Improvement.” Journal of Web Librarianship 6, no.1 (2012): 45-55.