#users1st

Get Going with Users

30 APR & 01 MAY 2015

London Edition Hotel

#users1st

Objectives

By the end of this workshop you will be able to…

get buy-in for remote testing

design your own tests

launch them

analyse them

and present results internally.

#users1st

Agenda

What is remote UX Testing? [10 m]

Test Design and Launch [40 m]

Getting buy-in [20 m]

Live analysis [20 m]

Presenting results [20 m]

Q&A [10 m]

#users1st

The company

Started 6 years ago

N. London office

20 Full-time staff

30+ remote analysts

10,000+ panelists

1,500 + clients

250,000+ user videos

#users1st

Meet…Barend Faber

Onboarding Manager

Laura Bustaffa

User Researcher

Pete Medcalf

Chief Revenue Office

Georgia Rakusen

Senior Account Manager

Oliver Newton

Sales Manager

Nathan Bethell

Head of Marketing

#users1st

1. What is Remote UX Testing?

#users1st

#users1st

You

#users1st

#users1st

Simply…

Capturing in the moment experiences

With spoken thoughts

From profiled panel members

#users1st

2. Let’s design and launch a live test

What’s involved in designing a test?

5 x task writing tips

2 example tasks

Each table writes tasks for one of your sites

And launches it

#users1st

What’s involved in designing a test?

Setting objectives

Deciding on what type of test to run

Participants: Profile, Volume and Devices

Tasks for users to follow

Exit Question

1st

#users1st

5 x Task writing tips1. Number each task

2. Set a scenario

3. Focus on tasks NOT opinions

4. You need not start at your Home Page

5. Aim for 20 minutes(ish)

6. Use fake personal data, stop at the point of payment

7. Run a pilot to check your tasks

#users1st

Start the screen recorder and click on the displayed link. (Please make sure you follow every instruction and talk through your experience. Remember, we are testing the site, not you.)

Thank you – you can stop the screen recorder.

At the beginning of each task list:

At the end of each task list:

#users1st1. Pretend you’re looking to re-mortgage

2. Without thinking too much, say what three words come to mind about [the brand]?

3. Would you give your personal details if requested by this site? (You will not have to during this test, but we are interested if you would do so in real life.)

4. Imagine that you wanted to apply for a mortgage to buy a new home. Where would you click first on this page?

5. Explore the mortgage offers, as if you would be genuinely interested in applying for a mortgage with us. (Spend no more then 5 minutes on this task.)

6. Apply for a mortgage, and proceed as far as you can. (You can use fake data, if you are not comfortable with providing your personal details.)

7. If you had a magic wand how would you improve our website?

#users1st

1. Pretend you are moving to Reading and for your new home will need internet, TV and phone service.

2. Using any and as many websites as you like find a plan (or plans) that genuinely suit your needs.

3. At the end, please say which plan you would choose and why.

#users1st

Your mission [20 m]1. Decide on a website to test (general consumer is best)

2. What do you want to find out?

3. Write tasks for users (desktop only) + one person types them

4. We’ll review a couple (you read them out)

5. Launch them on the platform!

u: workshop1@whatusersdo.com

p: dream

5 x UK users (keep the demographics broad)

#users1st

Top 9 Test Types

1. Live site key journey to identify points of friction

2. Pre-live design assets e.g. Axure prototype to de-risk change

3. Cross device RWD

4. How Users Search starting at Google

5. Specific pages/journeys to inform AB Testing

6. Email campaign > online journey testing

7. Competitor benchmarking

8. Scaling Lab insights

9. International local language

We’ll send you 10 Test types with Tasks

#users1st

How many participants?

Usually 10 per segment for key journeys

5 if it’s very specific e.g. a specific area of a prototype

Usually multiplied by device types

20+ for Card Sorting/Tree Testing

50+ for user research or marketing type tests

100+ full scale Abandonment Audit, Digital Index

#users1st

Choosing participants

Online panel: Demographics + screener

or/and your customers: Private Panel

or/and Special Recruit: anywhere

We’ll send you the Panel Book

#users1st

Activity (5 minutes)

Share your burning UX issues or projects

How we would design a test

#users1st

4. Getting buy-in

Benefits and Objections

3 x Case Studies you can use

Role play: sell it to me!

#users1st

5 x Benefits discussed in the session

1. Users are in their natural browsing environment

2. Results are fast (days not weeks)

3. Videos are compelling when shared internally

4. Tests can be run internationally

5. Less logistical setup than in-person methods

#users1st5 x Objections discussed in the session1. Testers take many tests and are “semi-professional”

• Counter with: testers are quarantined so they don’t take too many

2. You can’t ask probing questions mid-test

• True, but the higher participant volume mitigates this

3. Unlike eye-tracking we can’t see where users are looking

• You don’t need to & Eye-tracking has a very poor (if any) ROI for UX: http://www.slideshare.net/harrybr/what-you-need-to-know-about-eye-tracking-new-uxlx-version

4. It can take a long time to analyse lots of video

• Allow 1 hour per video or ask WhatUsersDo to analyse for you

5. You can’t get the right types of users

• We can - through special recruits or a panel of your customers

#users1st

Case Study

3 x Case Studies:

Fashion retailer = to show how UX testing makes sense of analytics

AO.com = shows how UX Testing improves conversions (read online)

Shearings = shows how UX Testing improves marketing (read online)

#users1st

Fashion retailer Case Study

#users1st

AO.com Case Study Like you they had a development list/product roadmap

Based upon hunches and opinion

Prioritised by HiPPOs

#users1st

User eXposure Matthew Lawson insisted everyone watch users buy white goods

On their site, on competitors’ sites, starting at Google Listed the top issues and re-wrote the development list

In a few months made changes that:

Increased online sales by 9.5%

Reduced contact by 33%

Increased engagement (product reviews) 12%

Read the full Case Study

#users1st

Manufacturer video

ads on the old site

#users1st

User-centered videos on the new site

#users1st

Pizza + User Videos = Buy-in

#users1st

Shearings Case Study 2012 TV advert very successful

New 2013 TV advert performed poorly

#users1st

Shearings Case Study 25 users watching both versions

Images changed too rapidly

"Using WUD to A/B test our TV campaigns has been an extremely quick an efficient way to radically enhance the performance our campaign.”

Chris Barker, Head of Marketing, Shearings

#users1st

Getting buy-in

Rehearse the benefits and objections

Use 3 x Case Studies: Analytics, Conversions & Marketing

Use your free WhatUsersDo video

Bonus: call it Digital Customer Experience Testing

#users1st

5. Live Analysis

Tag types

Live Tagging

3 x Tagging Tips

#users1st

Tag Types: issues

High: conversion killers, that block users or can prevent people from

converting in real life.

Medium: may become conversion killers when they occur more than once.

Low: small niggles that slow down users and decrease trust in the site.

#users1st

Tag Types

Positive: highlights in the experience, moments where users go “wow”.

We don’t tag when the interface is just doing what it’s supposed to do.

Suggestions: when users suggest an improvement.

Other: everything else worth noting.

#users1st

3 x Tagging Tips

1. Structure Tag names as: Area + Issue e.g.

• Basket – page speed

• Homepage - Images not relevant

• Delivery - Shipping price not clear

• PDP - Detailed information needed

• Compare page - benefit not clear

• Calculator - Difficult interaction

#users1st

3 x Tagging Tips

2. Give context to the event described in the tag: for example if the user

struggles with navigation include in the tag the moment when they say

what they are looking for. The video will be easier to share.

3. Allow about 1 hour for each 20 minute video

#users1st

6. Compelling Presentations

6 Steps:

Extracting insight from the platform

Creating a presentation

#users1st

6 Steps to Build Presentations1. Read the WUD platform synopsis [MS clients]

2. Watch playlist + star clips

3. PPT: Replay the test objectives, tasks & participants

4. PPT: Count of issues by severity and type

5. PPT: Exec Summary of 5 main findings

6. PPT: Each issue: problem & recommendationWith a screen shot

With a video clip (from those you starred)

“User quotes”

Written verbosely so non-attendees can understand it

#users1st

1. Read the WUD platform synopsis

(Managed Service clients only)

• The written synopsis includes:

• Overall user experience, including trends

• Top 5 positives

• Top 5 issues

• Any user suggestions

#users1st

Use the synopsis as a springboard to explore the videos in the WhatUsersDo platform.

#users1st

2. Watch playlist + star clips

Watch a playlist of all the video tags from your project, prioritised by high, medium and low severity issues, positives and user suggestions.

Favourite the “killer clips” which support the main issues you need to address.

Download your favourites in a CSV file, or download individual clips, to embed them into your presentations.

#users1st

3. PPT: Replay the test objectives, tasks & participants

Replaying the test objectives and scope

of test is great for reminding those on

the project, as well as informing

stakeholders who might review many

different supplier reports.

It helps “set the scene”.

#users1st

4. PPT: Count of issues by severity and type

Screengrab the coloured bars in the

WUD platform, and embed into your

presentation, to illustrate number of

issues and issue weighting (high,

medium and low severity issues, and

positives).

Use icons to visually illustrate what type

of issues caused the most problems for

users.

#users1st

5. PPT: Exec Summary of 5 main findings

Briefly summarise the top 4 or 5 issues

from the project which users

experienced, and visually illustrate them

with icons.

#users1st

6. PPT: Each issue: problem & recommendation

Expand upon each issue in detail.

Illustrate the issue with:

- Screengrabs

- User quotes

- “Killer” clips

Follow up with recommended Action to

take, and what the expected outcome

would be if that action was taken.

This will give you a really compelling

deck of slides, to overcome stakeholder

objections, and achieve buy-in for

necessary change or further testing.

#users1st

#users1st

#users1st

We’ll send you the template

#users1st

Sum-up

We’ve now covered…

getting buy-in for remote testing

know what you can test

design your own tests

launch them

analyse them

and present results internally.

#users1st

You can do this…

Self Serve

barend@whatusersdo.com

= DIY, occasional.

Managed Service

nathan@whatusersdo.com

= Expert Team, BAU.

or a blend.

#users1st

Q&A

#users1st

“Read the f***!!!@@@ blog.”

-- M. Tucker