Remote usability testing and remote user research for usability

Remote User Research Methods, tools and practicalities Breakfast Briefing 1 October 2014

@UserVision

Remote User Research

What do we mean by remote user research?

Types of remote usability testing

Practicalities and tips

What to use when

Usability testing is the best way to get empirical evidence of the likely appeal or success of your product

What is remote usability testing?

• Participant far away – can’t interact in person

• It is NOT

…collecting analytics, Split testing or MVT VWO, Optimisely, Google Experiments

… where user is unaware they are being tested Background playback tools – Mouseflow, Sessioncam

Heatmaps of where user clicked – Clicktale, Crazy egg

… only a survey of opinion Eg. Surveymonkey, primarily based on opinion

… remote ethnography, diary study

Some things that we’re not talking about

• Various ways to record clicks, mouse movements, scrolling and secretly record users using your site.

• Crazy Egg

• Clicktale

• Inspectlet

• Sessioncam

• Mouseflow

• Anyone using these?

Remote research

• Advantages

Access users anywhere in the world

Remove travel costs

Faster turnaround – results within an hour

Natural environment –at home

Can ‘live recruit’ to capture ‘in the moment’ experience

More honest / representative?

• Disadvantages

No chance to talk with user (unless remote moderated)

Less rich results compared to standard F2F tests

Types of tasks may be constrained

Potential for technical glitches

Getting the right audience may be difficult

Other considerations

• Still need a good research plan

What do you want to find out?

Who participates? Are they familiar with your product?

Do you want qualitative or quant insights?

• Make it personal, friendly

• Be clear in task scenarios, and success states

• Consider the audience:

Will technology overly complicate the process?

How can we best replicate real-life circumstances?

Online research audience may be more web savvy or ‘professional testers’

Remote User Research

What do we mean by remote user research?

Types of remote usability testing

Practicalities and tips

What to use when

Types of remote research

1. Remote moderated – Like normal testing but aided by screenshare / audio share – live discussion

2. Remote unmoderated – User goes through tasks to collect behavioural & opinion data, no discussion

Remote Moderated Testing

Remote moderated testing

• Like a face to face test – over distance

• See their screen and hear the audio

• Captures experience from where they normally use the web

• Can allow 3rd party observation

• Technical issues may occur

Remote Moderated Testing - example

Edinburgh,

London

Mexico

Brazil

Nigeria

Egypt

Pakistan Bangladesh

Indonesia

Remote observation – with translation Tests conducted in

China – in Chinese Simultaneous

translation to English

Observ ed by UX

consultant in the UK, listening to the English

translation

Observ ed by the

client in Dubai Observ ed by the

agency in NY

Practicalities and tips for remote moderated tests

• Five common practical questions:

What can you test?

Who should you test with?

Where can participants be located?

How should you conduct the testing?

When will the sessions happen?

• Already covered why…

What can you test?

What can you test?

• Website/web services

Most common

‘Best fit’ for this form of testing

What can you test?


• Corporate IT systems/Intranets

Can present some security/firewall issues

Some participants can feel constrained when taking part from their employer’s environment

What can you test?



• Highly graphical/animated interfaces e.g. Flash

Can suffer in transmission due to frame-rate/image quality limitations

What can you test?



• Highly graphical/animated interfaces e.g. Flash

• Mobile

Much bigger challenge

No single, cross-platform solution

Many security-related restrictions (and risks) at OS level regarding screen-sharing

What can you test?

• Mobile: ‘Laptop hugging’ technique

Who should you test

with?


• Anyone…

…with some caveats


IT Confidence

Context

High

Low

Unrepresentative Representative

Where can

participants be located?


• Anywhere

…with some caveats:

Language requirements


• Anywhere

…with some caveats:

Language requirements

Geographical regions with ‘endemic infrastructure limitations’

How should you

conduct the testing?


• Many screen sharing apps available

Join.me

Webex

GoToMeeting

Adobe Connect

Skype


• Skype:

Good audio quality


• Skype:

Good audio quality

Relatively low bandwidth requirements


• Skype:

Good audio quality


‘Set up’ already done by the participant


• Skype:

Good audio quality



Implies some level of IT comfort/capability


• Skype:

Good audio quality




Gets the communications technology ‘out of the way’


• Skype:

Good audio quality





Risk that it restricts pool of potential participants


• Skype:

Good audio quality






Shifting sands of features/account types


• Skype:

Good audio quality


‘ ‘Set up’ already done by the participant




Shifting sands of features/account types

And…

ALWAYS have a back-up plan!

When will the

sessions happen?


• Time.is:


• Time.is

• Calendly:


• Time.is

• Calendly

• Google docs:

Calendar

Spreadsheet

Remote Un-moderated Testing

Self-moderated (video) testing

• Participants video themselves doing task & talking

• Can’t discuss in real time with the participant

• Recruited from provider’s panel, crowdsourcing / social media or your own list

• Participants set up own web cam

• Still need time to review the videos, analyse , recommend solution

• Better for general audiences rather than specialist

Self-moderated (video) testing

• Usertesting.com

Panel focused on US, Canada, UK

Can annotate, edit video clips

Peek as a free trial

• WhatUsersDo

Panel covering 26 countries, can be filtered

• Both can cover mobile platforms

Example - mobile - http://youtu.be/zb0HigU_rys

http://youtu.be/zb0HigU_rys

http://youtu.be/zb0HigU_rys

Remote unmoderated testing

• Various tools, most capturing quantitative & qual

• Track success by specifying success pages

• May be able to test competitor sites

• Both can test on mobile

Other Mobile variations

• Forward facing camera captures user’s face

• Apsee

a mobile iOS app analytics platform

Requires line of code inserted to the App

Captures user recordings, creates touch heatmaps

• Lookback

• UX Recorder

Mobile website (not Apps) testing

Quick Exposure tests

• Five Second Tests

• UI Tests – 10 people for 10 Seconds for $9

www.uitests.com/t/aXmgpXE

• Usability Tools

Click Testing

Web Testing

Card Sorting

http://www.uitests.com/t/aXmgpXE

Optimal workshop

Verify

• Test screenshots / designs to gain insights on expectations and reactions

1. Preference test – which of 2 options prefer and why

2. Yes / no test –Mark areas, see if click right place

3. Click test – See where people click based on instructions

4. Multiclick test – As above but can compare different sites

5. Memory test – like the 5 seconds test – what you remember

6. Annotate test – allow free form notes to be marked on page

7. Mood test –show screenshot, select from emotion options

8. Label test – Check users understanding of what link will do

Example output - Verify

Usabilla

• Usabilla Visual Survey

Collect visual feedback on wireframes, mockups or any other visual

Feedback shows where people clicked in a heatmap

Choosing remote methods

Qualitative

Concrete

Quantitative

Conceptual

Video Self-moderated

Remote Moderated

IA, Short tests, Verify

Remote Unmoderated

Resources

• http://remoteresear.ch/

• Books

Beyond the Usability Lab – Bill Albert

Remote research – Nate Bolt

http://remoteresear.ch/

http://remoteresear.ch/

55 North Castle Street Edinburgh EH2 3QA Tel: 0131 225 0853

www.uservision.co.uk

[email protected]

@UserVision

Contact us

Remote usability testing and remote user research for usability

Internet

Transcript of Remote usability testing and remote user research for usability