Guerrilla usability testing - the survival guide

@thedxw www.dxw.com @lily_dart @rosebotanic@thedxw www.dxw.com @lily_dart @rosebotanic

Guerrilla usability survival guide

A quick-start guide to designing and preparing guerrilla usability testing sessions

@thedxw www.dxw.com @lily_dart @rosebotanic

Guerrilla usability testing is a low cost method of usability

testing. The term ‘guerrilla’ refers to its ‘out in the wild’

style, in the fact that it can be conducted [...] anywhere

there is significant footfall.

Guerrilla testing works well to quickly validate how

effective a design is on its intended audience, whether

certain functionality works in the way it is supposed to, or

even establishing whether a brand or proposition is clear.”

“

GOV.UK Service Design Manualhttp://bit.ly/govuk-guerrilla


Contents

￫ The basics

￫ Environment

￫ The script

￫ Incentives

￫ Interviewing techniques


The basics


Research question

Before you start any kind of research, it’s useful to have one or two research questions or hypotheses to focus your study and your findings.

This can be as simple as:

￫ Can users complete this task easily?

￫ Can users find key pieces of content?

￫ Do users understand what the organisation does?


Structure of sessions

A guerrilla usability study usually comprises of 6-8 sessions, each is usually no more than 15 minutes long.

Each session includes:

￫ Your introduction and explanation

￫ Gaining permission and written consent

￫ Questions about your participant

￫ 2 or 3 scenario-based tasks to test the website or app


What you’ll need

Requirements for guerrilla testing can vary depending on what you’re researching, but generally you will require:

￫ A laptop or mobile device to test on

￫ Software on the device to record the session

￫ Wifi, or a mobile internet connection

￫ A script of what you’re going to cover


Recording

It isn’t strictly necessary to record your guerrilla sessions, but being able to refer to them later will increase the potential insights you might gain from them. There are apps for most devices:

If you are recording, you must get written consent.

￫ Silverback for Mac OSX

￫ Rec for Android

￫ UX Recorder for iPhone and iPad


Environment


Location

Running your guerrilla testing sessions in multiple locations is recommended, to get a broader variety of participants.However, it’s important to pick appropriate venues. You should consider:

￫ Whether your target audience are likely to be there

￫ Whether you can get permission to approach people

￫ Whether you can stop there for 10-15 minutes

￫ Whether there is access to the internet, or 3G/4G signal


Time and activities

The activities that occur in the environment you’re testing in are also important to consider, as it may affect whether or not people feel able to give you their time. You should consider:

￫ Whether participants will have time to stop

￫ Whether participants will be busy, or under pressure

￫ Whether participants might be with friends or family

￫ What time of day is most appropriate


Example

Let’s say you have an app that helps people to find local bars that serve their favourite drinks. Your target audience is people between 20-34, earning between 35-65k, who regularly visit city centres. You might choose to:

￫ Run your sessions at two mid-range bars in a city

￫ Approach people between 5-6pm, because:

￫ They’re likely to be alone, waiting for friends

￫ The bar will be quieter


The script


Introduction

Because you are approaching strangers, your introduction should be well practiced and confident. It should include:

￫ Who you are

￫ What you’re asking them to do

￫ Whether the session is being recorded

￫ The length of the session

￫ Details of any incentive they may receive


Example

Hi, do you have a few minutes?

My name is Rose and I’m from a digital agency called dxw.

I’m here asking people to take a look at a website and to let me know what they think, and I’ll be recording the answers.

It won’t take any longer than 10 minutes and we’ll buy you a coffee or a slice of cake to say thank you for your time.

Does that sound like something you’d be interested in?”

“

Be respectful, and walk away if they aren’t interested.


Screening questions

You can’t pre-select participants for guerrilla usability testing, so it’s important to find some information out about your participants on the day.

Consider what attributes your target audience have. What is the minimum amount of information that you’ll need to understand if your participant is part of your target audience?


Examples

“Where do you live?”

“What is your residential status?”

“Do you live with anyone?”

“What is your marital status?”

“Are you employed?”

Don’t collect information you don’t need, and check that the participant is comfortable discussing these topics.


Scenarios

Scenarios give participants real-world context as to why they might complete the task that you’re asking them to test.

They allow for more realistic responses from participants, as they more closely mirror how they might interact with a website in their own time.


Example

You have just been offered a full-time job and are about to sign the employment contract.

The contract says that you only get 5 days of paid holiday per year. You aren’t sure if, legally, that’s enough.

Where do you go to check?”

“


Written consent

If you are recording participants then you must ask them to sign a consent form. Your consent form should include:

￫ How you are storing the data

￫ What you will use the data for

￫ How long you will keep the data


Incentives


The affect of incentivesIt is often necessary to incentivise participants to motivate them to get involved, and to thank them for their time.

However, it is important to note that incentives can sometimes change the nature of participant responses. Because you are giving the participant something, they may tend towards being more positive.

Picking an incentive appropriate to your audience can help mitigate this. Incentives should seems proportional to the participants efforts.


Appropriate incentives

Design your incentives inline with what you know about your target audience, and proportional to the effort they have invested.

Consider:

￫ £5 will have a different value to someone earning 16k versus someone earning 60k

￫ If your audience has a personal investment in your product or website, they may not require a monetary incentive


Interviewing technique


Making participants comfortable

Participants will often be uncomfortable during usability testing sessions. They don’t know you, and may feel nervous if they struggle to complete tasks.

￫ Make sure they know you’re testing the site, not them

￫ Set clear expectations for what the session will contain

￫ Ask permission before moving onto new tasks

￫ Let them know they can stop at any time


Think aloud

During usability testing sessions it’s useful to ask the participant to speak their thoughts aloud.

This helps you to understand not only their actions, but their rationale for why they chose that approach.

Sometimes participants will forget to speak aloud. Some silence is encouraged, to allow them space to think, but sometimes you’ll need to remind them to speak their thoughts.


Example

When prompting participants it’s important not to influence their responses.

Use open, non-leading questions:

“What are you thinking at the moment?”

“Is this what you expected to see?”

“What would you do next?”


Being impartial

Try to keep your personal influence on participants responses minimal. It’s important to make participants feel comfortable, but being too friendly might make them less likely to express their honest opinions.

￫ Make impartial small talk, not personal comments

￫ Don’t express your own opinions about the site

￫ Be professional, but not distant


Checklist


☑ Decide on a research question

☑ Select an environment where:

a. You are likely to find your target audience

b. There is space to stop

c. People won’t be too busy or distracted

☑ Select an appropriate incentive remembering that incentives can affect responses. Your incentive should consider:

a. Your target audiences interests

b. Your target audiences financial means


☑ Produce a script including:

a. Your introduction

b. Participant screening questions

c. 2 or 3 scenario based tasks

☑ Get permission to talk to your participants and get written consent if you are filming them.

☑ Use good interviewing technique to:

a. Minimise influence on participants

b. Make participants feel comfortable

c. Get more insight out of your sessions


Further reading

￫ The art of guerrilla usability testinghttp://bit.ly/uxbooth-guerrilla

￫ GOV.UK service design manualhttp://bit.ly/govuk-guerrilla

￫ The importance of planning in guerrilla testinghttp://bit.ly/loop-guerrilla

￫ Why usability testing doesn’t have to be expensivehttp://bit.ly/uxmyths-guerrilla

http://bit.ly/uxbooth-guerrilla

http://bit.ly/uxbooth-guerrilla

http://bit.ly/govuk-guerrilla

http://bit.ly/govuk-guerrilla

http://bit.ly/loop-guerrilla

http://bit.ly/loop-guerrilla

http://bit.ly/uxmyths-guerrilla

http://bit.ly/uxmyths-guerrilla

Guerrilla usability testing - the survival guide

Data & Analytics

Transcript of Guerrilla usability testing - the survival guide