Software Design Class, Session 10- User Testing.

8/9/2019 Software Design Class, Session 10- User Testing.

1/28

2015 COWAN+

SOFTWARE DESIGN: SESSION 10ARE WE THERE YET?USER TESTING

Alex Cowan


2/28

2015 COWAN+

WHERE YOU SHOULD BE NOW

S C A L

E ?

P I V O T ?

PRODUCT &PROMOTION

USER STORIES& PROTOTYPES

CUSTOMERDISCOVERY &EXPERIMENTS

VALUEPROPOSITIONS &

ASSUMPTIONS

S H

O W

M E

?

W H A T

I F ?

WHO?PERSONAS

W H A T ?

PROBLEMSCENARIOS &

ALTERNATIVES

Personas,including Think-See-Feel-Do(Converged)

Problem Scenarios Alternatives Value Propositions Before & After

Storyboards

Interview Guide 2 (or more) Interviews

Summary ValueHypothesis

MVP Ideas Assumptions for Value

Hypothesis AIDAOR Storyboard

Experiment(s) forhypothesis testing

Usability test suite

Epic Stories Child Stories Supporting Storyboards Parallel Prototypes Interactive Prototype


3/28

2015 COWAN+

WHAT WELL FOCUS ON TODAY

USER STORIES& PROTOTYPES

CUSTOMERDISCOVERY &EXPERIMENTS

S H O W M

E

?


4/28

2015 COWAN+

YOUR USABILITY HYPOTHESIS- TESTING ITUSABILITY HYPOTHESIS

QUESTIONSIs a strong narrative (user story) guidingyour work?Do the UIs affordance deliver against thosenarratives?

TOOLSUser StoriesComparablesPrototypes/WireframesTest Plans


5/28

2015 COWAN+

WHAT WELL WORK ON TODAYFinding the Right

PROBLEMFinding the Right

SOLUTION

time

a l t

e r n a

t i v e s

divergence convergence divergence convergence

source: adapted from The Design of Everyday Things

PersonasProblems

Alternatives

FieldDiscovery

Value Hypothesis &Assumptions

ProductHypothesis

Experiments on Motivation

User Stories

ComparablesPrototypes

Usability Tests

Software

(input to)


6/28

2015 COWAN+

WHAT ARE WE TESTING?

source: adapted from BJ Foggs Behavioral Model

Ability (Usability)

Motivation

Action Line

Inaction : (

Action : )


7/28

2015 COWAN+

WHAT ARE WE TESTING?

source: adapted from BJ Foggs Behavioral Model

Usability Testing

Proposition Testing(ala Lean Startup,MVPs)

DONT RUIN YOURRESULTS BY TRYING

TO TEST BOTH AT THESAME TIME

Ability (Usability)

Motivation

Action Line

Inaction : (

Action : )


8/28

2015 COWAN+

Your software paintsvirtual walls. Tell the

subject youd like to see the wall yellow and

observe what they do.(If you want to test motivation,

start with interviews, thenMVPs, experiments.)

IF USABILITY TESTING, MOTIVATION SUPPLIED


9/28

2015 COWAN+

Exploratory Assessment Validation

It looks like thisapproach willfundamentally work.

The implementationis sound and readyfor tuning.

Ready forprime time.

USABILITY TEST SUITES: A PROGRESSION


10/28

2015 COWAN+

TEST-DRIVEN PRODUCT DESIGN

70% of users didntget the drag and dropin this version

This change inthe annotation was enough sothey got it


11/28

2015 COWAN+

Objectives & MethodsProduct Version

SubjectsResearch CompositionPre-Session Checklist

Session Design

bit.ly/cdhandbook bit.ly/vdt-usability

A USABILITY TEST DESIGN FOR ANYONE


12/28

2015 COWAN+

STEP 2: CREATE SUBJECT SCREENERPersonaSam the SmallBusiness Owner

ScreenerHow many times lastmonth did you post to

social media for yourbusiness?


13/28

2015 COWAN+

STEP 3: DEFINE FUNCTIONAL SCOPE (USING STORIES)

Made up epics for small business social media automation:

As a small business owner, I want to sign up for the service, so I can

give it a try.As a small business owner, I want to connect my social mediaaccounts, so I can create automated posts to them.As a small business owner, I want to try out a post so I can see how

the service works.Define the stories (epicsand/or children) that

youd like to test (5 min)


14/28

2015 COWAN+

STEP 4: DESIGN RESEARCH ITEMS

Take at least one storyand design a researchitem (5 min)


15/28

2015 COWAN+

STEP 5: BUTTON UP THE REST OF THE DETAILSObjectives & Methods

Product VersionSubjects

Research CompositionPre-Session Checklist

Session Design

bit.ly/cdhandbook bit.ly/vdt-usability


16/28

2015 COWAN+

STEP 6: UPDATE, CREATE YOUR PROTOTYPE (OR SW)Your prototype shouldsupport your researchdesign, not the other way

around.


17/28

2015 COWAN+

STEP 7: PRE-TESTFor most projects, subjects are hard to get.

Try to nd some looser approximations and test the research design-

it will usually have at least a few kinks that would waste a subjectyou worked hard to arrange.


18/28

2015 COWAN+

KEY INTER-RELATIONSHIPS

The user stories shouldbe the nexus of yourideation, the otheritems supporting them.

USER STORIES

USABILITYTESTING

VALUEPROPOSITIONS &

ASSUMPTIONSPROTOTYPES


19/28

2015 COWAN+

A) I know the part and I want toorder it so I can figure out nextsteps on the repair.

B) I dont know the part and I wantto try to identify it online so I can

move the job forward.

C) I dont know the part and I cantdetermine it and I want help so I canmove the job forward.


20/28

2015 COWAN+

# RESEARCH OBJECTIVE NOTES 3 How are we doing on this story:

I dont know the part number and Iwant to try to identify it online so Ican move the job forward.

MODERATOR GUIDELets say you were looking at this unit and this was the part youneeded to replace (offer photos). You dont have the part number.How would you identify it?

OUTPUTResult on whether the subject understands and can complete theapplicable search process.

EXAMPLE TEST ITEM- EXPLORATION + STORY B


21/28

2015 COWAN+

CHILD STORIESA) I know the part number and I want to nd it on the system so I can gure out next steps on the repair.

B) I dont know the part number and I want to try to identify it online so I can move the job forward.C) I dont know the part number and I cant determine it and I want help so I can move the job forward.D) I want to see the cost of the part and time to receive it so I decide on next steps and get agreement from the customer.

EXAMPLE: AGILE USER STORIESEPIC STORYAs Ted the HVAC technician, I want to identify a part that needs replacing so I can decide my next steps.


22/28

2015 COWAN+

# RESEARCH OBJECTIVE NOTES

4 How are we doing on this story:I dont know the part number and Icant determine it and I want help soI can move the job forward.

MODERATOR GUIDELets say you were looking at this unit and (whether or not youactually know what it is) youre just stuck identifying it and youwant help from dispatch. Can you show me what you would do?


EXAMPLE TEST ITEM- EXPLORATION + STORY C


23/28

2015 COWAN+

CHILD STORIESA) I know the part number and I want to nd it on the system so I can gure out next steps on the repair.

B) I dont know the part number and I want to try to identify it online so I can move the job forward.C) I dont know the part number and I cant determine it and I want help so I can move the job forward.D) I want to see the cost of the part and time to receive it so I decide on next steps and get agreement from the customer.

EXAMPLE: AGILE USER STORIESEPIC STORYAs Ted the HVAC technician, I want to identify a part that needs replacing so I can decide my next steps.


24/28

2015 COWAN+

D) I want to see thecost of the part and

time to receive it so Idecide on next stepsand get agreementfrom the customer.

EXAMPLE: AGILE USER STORIES


25/28

2015 COWAN+

# RESEARCH OBJECTIVE NOTES 4 How are we doing on this story:

I want to see the cost of the partand time to receive it so I decide onnext steps and get agreement from the customer.

MODERATOR GUIDETell me what youre seeing here? What the price for this part? Howsoon can you get it? Given that, what would you tell the customer?


EXAMPLE TEST ITEM- EXPLORATION + STORY B


26/28

2015 COWAN+

TOP 3 REASONS WHY TEAMS DONT TEST MORE

1

2

3

Its painful to see how wrong you were about the user.Even though youll be much better off if you try something new soon.

Members are have been turned off by phase-inappropriate testing(usually too detailed, too soon).Even though testing early and often avoids waste, pointlessdiscussion.

Getting in front of real users is kind of a pain.Even though the customer is why were here.


27/28

2015 COWAN+

TESTING IS REWARDING

Everyone has an opinion aboutusability- until it comes time to test.

Testing improves product and avoidslots of waste.


28/28

2015 COWAN+

ASSIGNMENTS1. Do/Group: Draft usability Test Plan (30-40 minutes)2. Do/Group: Update Interactive Prototype (40-60 minutes)3. Do/Group: Dry Run the Test Plan + Prototype (~30 minutes)

v10

Software Design Class, Session 10- User Testing.

Documents

Transcript of Software Design Class, Session 10- User Testing.