Crowdsourced Remote Unmoderated Usability Testing
19
Crowdsourced Remote Unmoderated Usability Testing Inge De Bleecker @ingedebleecker IASummit 2012 And how to make it work for you
-
Upload
inge-de-bleecker -
Category
Technology
-
view
2.660 -
download
0
description
Slides for my presentation at the IASummit 2012 on crowdsourced remote unmoderated usability testing.
Transcript of Crowdsourced Remote Unmoderated Usability Testing
Crowdsourced Remote Unmoderated Usability Testing
Inge De Bleecker @ingedebleecker IASummit 2012
And how to make it work for you
Overview
o What is it
o How to do it
o Get started today (or Monday)
What is it
Remote
Unmoderated
Crowdsourced
“The act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”
— JEFF HOWE
Crowdsourcing
“a model for enabling convenient, on-demand access to a (shared) pool of human workers with different skills that can be rapidly provisioned and released with minimal management effort”
— PANOS IPEIROTIS
Crowdsourcing
Different Flavors including
o Screenshot click test
o Screenshot timed test
o Task-based usability study with online survey
D I Y
Good for
o Use of personal devices
o In own environment
o Fast turnaround
o Cheap(er)
Not good for
o Can’t observe participants
o No additional questions
o Subject video recording
Process
Still the same process
1 Recruit participants
2 Write task plan
3 Run pilot
4 Run test
5 Analyze results
6 Write report
Need committed participants!
Not easy to write a good task plan
Committed participants
Commitment
Compensa/on Loyalty
Longer sessions Higher
quality
Higher comple/on
rate
Tips for writing a task plan
o Participants can’t get blocked o You only get one shot o Guide without influencing o Make all questions required o Encourage think aloud/write down o Task and question types depend on
tool
Tasks & Questions
o Screenshot click test (@usabilityhub)
Tasks & Questions
o Screenshot timed test (@usabilityhub)
Tasks & Questions
o Online survey testing (Loop11)
Tasks & Questions
o Online survey testing (DIY)
Tasks & Questions
o Online survey testing (DIY)
Results
Self-reported data
“I find grid views easier to scan because they show enough detail of several phones on one screen without forcing me to scroll” “I would select the "Compare this phone" option below the phones I wanted. But then I am at a loss as to what to hit next to actually carry out the comparison”
Getting started
Got money! o Engage crowdsourcing company o Use specialized tool
Broke? o Make your own crowd o Use generic survey tool (e.g. Surveymonkey,
Google docs)
Q & A
Thank you
