Remote Usability StudiesGet Great Results While Saving Time and Money

Jason Holmes

Aaron RosenbergAmerican Greetings Interactive

So…you need to do a user study…by tomorrow Your boss comes by and says, “We need to

test this on users. I need answers by the end of the week.”

Options Call a marketing/UX consulting firm Do it yourself at your desk

Do it yourself…remotely!

What is Remote Usability?

Research where the facilitator and participant are separated by time and/or space

Can involve unmoderated or moderated methods

Usually mediated by technology

Unmoderated tools

Optimal Workshop OptimalSort Treejack Chalkmark

Loop11.com Task-based testing

OptimalSort - Open

OptimalSort - Closed

Treejack – Optimal Workshop

Where would you find a baby book?

Where would you find a baby book?

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Occasions P r oducts Recipients

Visi ted fi r st

Visi ted dur ing

Treejack – Results

Chalkmark – Optimal Workshop

Loop11.com

What do you get with remote testing? More contextual, more ethnographic Stakeholder participation Remote logging Instant messaging between

Facilitator <> Observers Facilitator <> Participant

Import into Morae Compile in-person Morae studies with UserVue

studies

What do you give up with remote testing? Non-verbal cues

Facial expressions Head movement

Moments of confusion and hesitation are more difficult to detect

Less control More distractions

Are the results as good?

According to Bolt|Peters, yes!

In a comparative study between in-person and remote methods, they found remote was better Remote yielded more key findings Shorter sessions 1 day vs. 12 days $17,000 vs. $26,000

Getting Remote Participants

Tap existing customer base Limited to current customers…not prospective customers

Ethnio (Bolt|Peters) Quick screener DHTML popup using JavaScript Geographically neutral

By combining Ethnio and Uservue, you can live recruit and record what real users actually do!

UserVue

Setting up study Running study

Install plug-in Initiate call Initiate screen connection Logging

Analyzing results with Morae

Analyzing Results

Remote Moderating Tips

Explain that you can see everything on their screen “If you have anything of a personal or sensitive nature on

your screen, please close it now.” Use their name often Explain the technology

“There’s a slight delay between when you do something and when I see it, so I might ask you what you just did.”

Blame technology “You just cut out.”

Not everything is great

Participant commitment not as great Prepare for “no-shows”

Unforeseen logistical problems Give you their cell number but don’t want to use it

for an hour Computer is in basement, but their phone signal

won’t reach “Can you hold on a second…”