What you can learn from usability testing
26
Need a little usability? What you can learn from usability testing Whitney Quesenbery Center for Civic Design
-
Upload
whitney-quesenbery -
Category
Design
-
view
115 -
download
5
description
Every study starts with a question. This session at CSUN 2014 started by examining the questions that usability testing can answer. Short case studies illustrate how the right technique will help us know not only what is happening but also why it’s happening. It's an overview of usability testing as a research method, and what you can (and can’t) learn from working with real people as they try to use a web site or other product.
Transcript of What you can learn from usability testing
Need a little usability?What you can learn from usability testing
Whitney QuesenberyCenter for Civic Design
2
What questions do you want to answer with a usability test?
3Usability testing answers questions, so we have to
understand the question we are asking.
If we want to know:
Which design helps users complete tasks more effectively?
We are asking a question that is
Comparative, quantitative (errors, time)
(and why they made those mistakes)
If we want to know:
Do voters understand how to mark their ballot?
We are asking a question that is abut
Mental models, variability, qualitative insights(and how prevalent each model is)
4
Helpful models for thinking about usability
A’s Ability, aptitude, attitude(what users bring to the interaction)
E’s Efficient, effective, engaging, error-tolerant, easy to learn(dimensions of usability)
Use Useful, usable (operable), desirable (context of use)
5
The As help you think about diverse participants
Attitude
motivation, emotion, risk tolerance, persistence, optimism, tolerance for frustration
Aptitude
current knowledge, ability to make inferences or innovate solutions, expertise, habits
Ability
physical and
cognitive attributes
Source: Dana Chisnell "Character Creator" http://www.slideshare.net/danachisnell/character-creator
6Different users – and different contexts of use – can
change user needs
7
5Es help you plan usability tests
Usability Goal Types of usability testing needed
Evaluate tasks for how accurately they were completed and how often they produce errors
Effective
Error-tolerant
Easy to learn Control how much instruction is given to test participants, or recruit participants with different levels of knowledge
Construct task scenarios to create situations with potential problems
Engaging User satisfaction surveys to gauge acceptancereview logs for ‘time on site’
Efficient Time tasks with realistic tasks and working versions of the software
8
The recipe for planning a usability test
Location and context: formal informal
Recruiting: defined opportunistic
Activities: instructed tasks free tasks
Questions: structured unstructured
Data collection: observation task/data only
Results quantitative qualitative
9
A few examples of questions and how we answered them
These examples all come from work on elections and civic design..but the principles apply to any project.
10Comparing two designs
Can a design improve user performance?
Plain vs. traditional language
About this project 45 participants 3 lab setting, 8 days 2 moderators Instructions for voting Careful observation Satisfaction survey
We learned about Voter accuracy Order effects Voter preference
Results Statistical analysis Participant preference
11Comparing two designs
Can a minor difference have a major impact?
Left aligned or right aligned names
About this project 100 participants 2 locations, 1 day 4 moderators + 6 others Instructions for voting Minimal observation
We learned about Voter preference Time to vote Number of errors
Results Statistical analysis of marked
ballots
12
Testing a new designWhat problems might voters have with a new ballot design?
About this project 1 of 5 similar tests 10 participants 1 locations, 1 day 2 moderators + 1 others Self-defined tasks Observation Post-task questionnaire
We learned about Navigation on the ballot Instructions
Results Found pattern of errors for one
interaction Time on task
13
Exploring new ideas
Can we extend a design to new audiences?
http://anywhereballot.orghttp://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/
About this project Research project Paper prototyping Low literacy
We learned about What they understsand Navigation Instructions Interactions
Results Simplified everything for a
clear, universal design
14
Answers quickly and inexpensively
Can we find participants "where they are?"
http://anywhereballot.orghttp://civicdesigning.org/featured-story/rapid-responsive-radical-the-anywhere-ballot-is-born/
About this project 13 participants Observed and interviewed as
they filled in a new voter registration form
We learned about How people filled in the form
Results Improved voter registration
forms
This project and poster were done by Jenny Greeve, the Design Fellow at the Washington Department of Elections
15
Learning about comprehension
Can voters explain and act on that understanding?
About this project 20 participants 4 locations, 1 day 8 moderators Self-defined tasks Minimal observation
We learned about Navigation on the ballot Instructions Mental models of counting Whether there’s a problem
Results Qualitative insights
16
Gauging opinionWhich accessible voting system do voters prefer?
About this project 496 surveys; multi-disabilit 11 systems Multiday “voting fair”
They learned about Voter assessment of usability Voter comments about
accessibility Differences by disability
Results Statistical analysis survey
results Qualitative analysis of
comments
This was a project of the Oregon Department of Elections
17
Planning a
usability program
18Look for a balance of methods by approach and
context of use
19
Early in Project
Many Users
Few Users
Comparative benchmarking
Summative testingCard sorting
Satisfaction surveysMarket research
Iterative evaluation of prototypes
Evaluation of Designs
Ethnography
Use the right tool at the right time
Stakeholder interviews
User observation
WalkthroughsExpert Reviews
A-B Testing
20Look for the 'cadence' of what kind of usability work
you do at different times
Seeing the Elephant: Defragmenting User Research by Lou Rosenfeldhttp://alistapart.com/article/seeing-the-elephant-defragmenting-user-research
21
Things to watch out for when planning a usability test
22
Recruiting for diversity
Gender
Race/ethnicity
Income/education
Disability
23
Treating participants with respect
Language and cultural issues
Managing context
Participant protections and informed consent
24
Avoiding bias
Leading questions
Closed vs. open questions
Sampling bias
Moderator influence
Observer (or moderator reactions)
26
Storytelling for User Experiencewith Kevin Brooks
Global UXwith Daniel Szuc
A Web for Everyonewith Sarah Horton
http://rosenfeldmedia.com/books/a-web-for-everyone/
