Just Right Usability Testing Workshop by Jon Deragon - i ...€¦ · USABILITY TESTING Presented by...

JUST RIGHT USABILITY TESTING

Presented by Jon Deragon at August 25 2016

WORKSHOP

i-USEr Conference 2016 on User Science and Engineering Faculty of Information and Communication Technology

Universiti Teknikal Malaysia Melaka (UTeM), Melaka, Malaysia


Background

WORKSHOP


WHAT PEOPLE THINK

WireframesPost-It Notes

Flow Diagrams

WORKSHOP

UX =

UX: THE REAL BEAUTY OF UX

Artefacts

ValidationResearch


WORKSHOP

WireframesPost-It Notes

Flow Diagrams

Less Trial and ErrorLess Revisions

More Profitability

ProveResearch and Assumptions

Are Correct

VALIDATING THROUGH TESTING

=


WORKSHOP

REALITY STRIKES

Often under funded

Project plan blow out sacrifice

Insufficiently resourced

Seen as obstacle to launch

Many times misunderstood

Lack of clarity on outcome and benefit


WORKSHOP

IT’S MORE EXPENSIVE NOT TO


WORKSHOP

Redesign costs Redevelopment costs QA testing Promote to production server Lost potential customers and sales Negative impact on customer perception

Lab Testing

Sophisticated facilities,specialised equipment,

recruitment, scripts, preparation time, compensation for

participants.Guerrilla Testing

Sketches on paper and grabbing people at the

local cafe

SCALING TESTING

Cost Effort

Complexity Time

Resources Skill


WORKSHOP

Lab Testing

Guerrilla Testing

AGREEABLE SOLUTION


WORKSHOP

Just RightAdapt best of both worlds. Agility of gorilla, depth of

depth of lab testing inefficient middle ground.

EFFICIENCY


WORKSHOP

Costs Participant compensation (if necessary)Software or service

People Testing facilitatorsParticipants sourcing and scheduling

Equipment Computer or mobileWebcams and rigs

Time Objectives, scriptsRecruitment & SchedulingConductingAnalysis & Trending

PARTICIPANT NUMBERS


WORKSHOP

Optimal Range and Diminishing Returns Effect

1 2 3 4 5 6 7 8 9 10

WHEN TO TEST


WORKSHOP

Low Fidelity Partial Minimal wireframe fidelityUser paths with minor variance covered

High Fidelity Complete Wireframes are highly interactiveAll or majority of user paths are covered

Actual Product Looks and feels like the real dealLive or fully functional mock data sets

Less rework

More rework

(the earlier the better)

TYPES OF TESTING


WORKSHOP

Unmoderated Testing conducted in isolation by participant,often remotely, and test recordings areevaluated afterwards by facilitator

Moderated Facilitator participates and interactsdirectly with participant throughout theusability testing process

A B


WORKSHOP

The Process

THE PROCESS


WORKSHOP

2 Participants • Define participant criteria• Recruit from public• Schedule test sessions

1Define• Determine objectives• Write scripts for scenarios

3 Prototype • Ready the prototype• Setup environment• Prepare equipment

5 Analyse • Review findings (notes,

recordings, feedback)• Trend to actionable items

4 Conduct • Perform usability testing

with participants

6 Implement • Incorporate into product• Retest to validate

THE PROCESS


WORKSHOP

2Participants• Define participant criteria• Recruit from public• Schedule test sessions

1 Define • Determine objectives• Write scripts for scenarios





with participants


THE PROCESS


WORKSHOP



3Prototype• Ready the prototype• Setup environment• Prepare equipment




with participants


THE PROCESS


WORKSHOP






4Conduct• Perform usability testing

with participants


THE PROCESS


WORKSHOP




5Analyse• Review findings (notes,



with participants


THE PROCESS


WORKSHOP







with participants

6Implement• Incorporate into product• Retest to validate

DEFINE


WORKSHOP

What are your objectives and outcomes? Testing new designs or existing designs?

Write scenarios for critical or concern tasks: • 3-5 scenarios maximum • Ensure natural story-like flow • Limit to 45 minutes (within attention span) • Don’t make scenarios or instructions overly complex

SCENARIOS


WORKSHOP

Example of a mobile phone store site script:

“You’d recently dropped your mobile phone, and unfortunately it died. You needed to get a replacement phone and decided to order online.”

1. Find Samsung Galaxy 7 phone with 128GB memory

2. Add item to the cart and purchase phone

3. Change shipping address to new location

PARTICIPANTS


WORKSHOP

What is your target audience? Use personas Best represent scope of your audience Consider familiar and not familiar with buying online Diversity: young, old, female, male, nationality, etc.

Prevent bias: • Cannot be within company, friends, partners, etc. • Avoid existing loyal customers

COMPENSATION


WORKSHOP

Popular types of compensation: • Gift vouchers • Cash • Free product

Value depends on many variables: • Length, complexity of testing • Distance to commute to venue • Income bracket of participants

SCHEDULING


WORKSHOP

Consider weekdays or weekends? During or after work hours?

45 minute tests need 1 hour buffers between them: • Accomodate late arrivals • Overrun on testing • Notes from test then prep for next test

Assume 1-2 participants are either no-shows or late

VENUE


WORKSHOP

Venue easily accessible Consider public transit convenience Include map and directions On-site signage should be clear and official Greet at door, don’t have them wandering

TESTING AREA


WORKSHOP

Low distractions Quiet and neutral Sectioned off or fully isolated Sufficient work space Ergonomic desktop and mobile arrangements Appropriate lighting and temperature Seating arrangement beside, not behind or in front of participant!

PROTOTYPE


WORKSHOP

Mobile Testing LookBack, Webcam and Mount Capture: screen video, touch input, front camera, audio Your Notes (Evernote, OneNote)

Desktop Testing SilverBack, In-built or Tripod Webcam Capture: screen video, mouse, keyboard, touch (if supported), webcam Your Notes (Evernote, OneNote)

PROTOTYPE


WORKSHOP

Desktop Prep • Turn off all messaging apps (Skype, Slack, etc.) embarrassing • Silence speakers and system alerts (if applicable) • Disable screen savers, adjust screen brightness • Notebooks: Disable power saver and sleep mode, plug into mains • Minimise desktop clutter, funky wallpapers • Turn off programs that are bandwidth hogs (Dropbox, etc) • Consider local version as redundancy or in low bandwidth environments • Physically clean the screen and peripherals being used • Clear browser history and caches (if applicable) • Have prototype minimised, recording software loaded and ready to go!

PROTOTYPE


WORKSHOP

Mobile Prep • Turn off notifications and system messages (visual & audible) • Shut off messengers (WeChat, WhatsApp, Skype, etc.) • Disable automatic screen dimming, locking and similar distractions • Adjust sound volume to the needs of the test • Close all unnecessary background apps • Wallpaper and lock screens should be generic • Set screen brightness to suit general audience and context • Physically clean device screen, buttons and case • Remove unnecessary flip cases, Hello Kitty fashion accessories, stock device • Have prototype and recording software loaded and ready to go!

WELCOME


WORKSHOP

When participant arrives: • Open arms, warm and welcoming • Be fun, lighten the mood • People divulge more when comfortable • “Testing” interpreted as THEM being tested • You want to disarm them

PAPERWORK


WORKSHOP

Capture participant details: • Name, age, gender, location, demographic data,

market questions, competitors, etc. • Potential customer is in front of you, take advantage!

Legal agreement signature: • Liability, confidentiality, anti-compete, compensation,

consent, general legal terms, use of recordings, etc.

Start Recording

EXPLAIN


WORKSHOP

When ready to test: • What are we doing, their involving, how it helps • About product they’re testing • Not offending anyone to speak your mind • Method (think aloud, not in head)

About prototype: • Limitations • Abnormal aspects or behaviours

CONDUCT


WORKSHOP

Open prototype: • If wireframes: explain and allow to acclimatise • Ask initial impressions • Get their general observations

Scenarios Loop • Carefully explain test scenario, ask to perform • Repeat for each scenario • Observe fatigue symptoms, encourage (shorten if needed)

CONDUCT


WORKSHOP

When user is conducting their scenarios: • Don’t put pressure on them • Don’t assist, let them work out the issues • Don’t ask “lead-in” questions • Look as much at what they’re doing as what

they’re saying (it’s only part of the story) • Body language • Take notes to complement the recordings

WRAP UP


WORKSHOP

When scenarios are finished: • Ask open-ended questions about experience • Ask for general feedback

Wrap up with great appreciation: • Talk about how it will help • Notify when launched (they have sense of co-creation) • Compensation in nice envelope • Escort them out, don’t just point the way

COMPILE


WORKSHOP

Gather recordings and notes: • Re-watch recordings (if needed), elaborate notes • Build spreadsheet • Break into scenarios • Matrix of issues, participants • Problems, dislikes and likes • Combine similar issues • Enhance with time to complete tasks, failed completes, etc.

ANALYSE


WORKSHOP

Review matrix: • Rank and sort by severity and frequency • Identify trends, regular occurrence of issues • Examine general feedback

IMPLEMENT


WORKSHOP

Take sorted list of findings: • Determine solutions to problems • Design impacts • Engineering to determine development effort • Business and process implications

Shortlist actionable findings for implementation Drop into agile sprints or project plan for actioning


WORKSHOP

Workshop

WORKSHOP


WORKSHOP

1. Form Pairs - Split into teams of 2 people, choose test subject 2. Scenarios - Write 4 scenarios for your test 3. Conduct Testing - Test your partner, observe and record; swap places 4. Analyse - Mark down your top 5 findings, severities, fixes 5. Presentations - Present your findings

Total Time 1 Hour, 20 Minutes

WORKSHOP


WORKSHOP

Time 5 Minutes

Form Pairs

Pair up with someone else to do testing with Choose a mobile website or app on your phone to test

WORKSHOP


WORKSHOP

Time 10 Minutes

Scenarios

Go through website, determine up to 4 scenarios to test • i.e. browsing products, search flights, sign up for lead capture Write down scenarios

WORKSHOP


WORKSHOP

Time 40 Minutes (20 minutes each test)

Conduct Testing

Meet and greet your participant Run “LookBack” in your mobile phone, load website or app Start “LookBack” recording Commence performing testing with your partner Record findings and severity as you go

WORKSHOP


WORKSHOP

Time 10 Minutes

Analyse

Organise your observations into groupings Prioritise in order based on severity Determine what you could change to correct problems

WORKSHOP


WORKSHOP

Time 10 Minutes

Presentations

Let’s choose a few brave people to “speed round” present their findings to everyone!


WORKSHOP

WebsiteLinkedInEmail

www.jonderagon.com (workshop materials) www.linkedin.com/in/jonderagon [email protected]

Jon Deragon, Senior UX Consultant

THANK YOU EVERYBODY

Just Right Usability Testing Workshop by Jon Deragon - i ...€¦ · USABILITY TESTING Presented by...

Documents

Transcript of Just Right Usability Testing Workshop by Jon Deragon - i ...€¦ · USABILITY TESTING Presented by...