Jonathan Rubin - The benefits of do it yourself usability testing

The benefits of “Do It

Yourself” Usability Testing

By Jonathan Rubin

First Fridays Program Manager

General Services Administration

2

Let me tell you a story

3

Hanging chads

4

inspections

5

butteflu

They didn’t listen to customer’s

wants, needs and expectations.

TAKEWAY MESSAGE: Testing

with customers gets you

valuable feedback (duh)!

Why did problems occur?

7

In this hour…

1. What is usability

2. Why you need usability

3. Idea: Do your own testing!

1. What is usability?

Usability =

How people interact with a

product, system or service,

and how they expect it to

work.

Usability ≠ Quality Control

10

Good products are usable

11

12

What is it? How do you use it?

13

What is it? How do you use it?

14

Don’t feel stupid!

1. Observe users (customers) using your

product

2. (Re)design with users in mind

3. Get a usable product

4. Repeat

4 Steps to Working with Users

• Education • Demonstration Tests • Awareness

We’ve helped these folks

• DOT

• Interior

• IRS

• U.S. Army

• NASA

• Labor

• OMB

• GSA

17

• State

• FCC

• USAJobs

• Census

• NSF

• NIH

• And 35

others

Free @ Howto.gov/firstfridays

• Usability Testing Scripts • Email Templates • Best Practices • Final Report Examples • Project Management • Before and After Screenshots • Videos • Etc.

2. Why you need usability

Top 5 Government Usability

Problems

• Too many words


Problems

• Too many words

• Gov speak / jargon


Problems

• Too many words


• Top tasks hard to find


Problems

• Too many words



• Ineffective navigation


Problems

• Too many words



• Ineffective navigation

• Ineffective search results

Example 2: GSA Intranet Example 2: GSA Intranet

9/11/2012 27

Before After

• Reduced developer time

Usability’s Return on Investment (ROI)


• Reduced training time




• Reduced help desk calls / emails





• Higher task completion rate





• Higher task completion rate

• PR boost


Know your users, for you are not them

- Usability Yoda

3. Idea: Do your own testing!

38

Must read: ‘Rocket Surgery Made Easy’

• Commit to monthly

product testing

• Same time each

month

• More observers +

more testing = more

buy-in

Needed for “discount” usability testing

• Executive champion!

Needed for UT


• One morning a month

Needed for UT




• 1-2 rooms




• 1-2 rooms

• Laptops




• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)




• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers




• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers

• Food ($10/person)




• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers


• Incentives ($0- $80)




• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers


• Incentives ($0- $80)

• Training (FF)


48

How many people do I need?

Spending on usability

49

When usability is present, 10% of

development budget spent on

usability

- Usability Return on Investment (2008, Nielsen

Norman Group Report)

Staffing options (all are good)

1. Usability Contractor ($17,000 per test)

2. Dedicated Usability Staff ($60,000+)

3. Get training and do it yourself (Free)

50

Matrixed Approach to Staffing

51

• Part-time volunteers! • Borrow from other departments • You are “providing skill building opportunities” • Flexible needs: 1 -20 hours a month

Types of First Fridays Tests

• 1. Formal, talk-aloud tests

Stakeholder

Meeting

Develop

Scenarios

Recruit

Participants

+ Observers

Conduct

Test

Write

Report

Follow-up

Meeting

Formal Testing Process

Testing Room Observation Room

It takes two (rooms) to test

55

Formal Testing Schedule

• 8:20 – 8:30 Welcome

• 8:30 - 9:30 Tester #1

• 9:30 – 10:30 Tester #2

• 10:30 – 11:30 Tester #3

• 11:30 – Noon Lunch

• Noon – 1:30 Debrief / Critique

Absolute Minimum Usability Staffing

Testing Room

T

F

T = Tester,

F = Facilitator

Minimum First Fridays Staffing

Observation Room Testing Room

T

O

F M

O O

O O

O = Observer

F = Facilitator

M = Room Manager

Better Staffing

Observation Room Testing Room

T

F M

O

O

G = Greeter

DG = Data Gatherer

BU = Back up

G

F O O O O O

DG

BU

BU

BU

O = Observer

F = Facilitator

M = Room Manager


1. Formal, talk-aloud tests

2. Hallway Tests

2. Hallway Test

http://www.youtube.com/watch?v=pC0ODg2MpuE



2. Hallway Tests

3. Navigation Improvement / Card sort

3. Navigation Improvement



2. Hallway Tests

3. Navigation Improvement / Card sort

4. Quick Win Evaluation (Heuristics Analysis)

64

4. Quick Win Evaluation (Heuristics)

• Not for newbies

• Group or single person feedback

– Checklist

– Task run-through

• PRO: Fast! <1 week

• CON: No users

Teach em to fish Teach ‘em to fish



• 1-2 rooms

• Laptops

• Screen sharing

software

• Volunteers

• Food

• Incentives for testers

• Training (First Fridays!)

Review: What you need to fish

Howto.gov/firstfridays

Free usability resources!

• Observe a test

More info on First Fridays

• Observe a test

• Recommend a website


• Observe a test


• Volunteer as a tester

(remote or in person)


• Observe a test


• Volunteer as a tester

(remote or in person)

• Learn to fish (Facilitator

in Training program)


Once a month

Thanks!

[email protected]

[email protected]

Howto.gov/firstfridays

Jonathan Rubin - The benefits of do it yourself usability testing

Technology

Transcript of Jonathan Rubin - The benefits of do it yourself usability testing