IWMW 2006: User Testing on a Shoestring Budget (1)

A centre of expertise in digital information managementwww.ukoln.ac.uk

UKOLN is supported by:

Usability on a Shoestring Budget (1)

Emma Tonkin & Greg TourteSoftware & Systems, UKOLNHCI Department, University of Bath

www.bath.ac.uk


Introduction• UKOLN, the University of Bath• HCI Group• Why this session?


Why do projects fail?Project Impaired Factors % of the Responses1. Incomplete Requirements 13.1%2. Lack of User Involvement 12.4%3. Lack of Resources 10.6%4. Unrealistic Expectations 9.9%5. Lack of Executive Support 9.3%6. Changing Requirements & Specifications 8.7%7. Lack of Planning 8.1%8. Didn't Need It Any Longer 7.5%9. Lack of IT Management 6.2%10. Technology Illiteracy 4.3%11. Other 9.9%


Introducing usability• Definition: the measure of a product’s

potential to accomplish the goals of a user• How easy a user interface is to understand

and use• Ability of a system to be used [easily?

Efficiently? Quickly?]• The people who use the project can

accomplish their tasks quickly and easily


Assumptions• There are several dimensions to

usability– Focus on users– ‘People use products to be productive’– Users are busy people trying to

accomplish tasks quickly– Users decide when a product is easy to

use• (Adapted from Redish & Dumas, A Practical Guide to User Testing)


However…• Are users always busy? Does this imply that

usability is only present in the workplace?!• Effectiveness vs. efficiency vs. satisfaction• Do users know when a product is ready?• Do all users agree about usability?• Is usability actually measurable?• Is there one statistic that == ‘% usability’?


Elements of usability

• Nielsen refers to five elements or components of usability:– Learnability– Efficiency– Memorability– Errors– Satisfaction – Usability Engineering, 1993, p.26

• These may not be of equal importance in all cases.


In other words…• Usability depends on context

– What does the user want to do?– Who is the user?– What’s the user’s perspective on life?

• Related to:– Internationalisation; cultural, social– Task analysis; working out what the user

wants to do (what the goal is) and how he/she would expect to accomplish it


Science vs craft• Formal approaches:

– Research-driven– ‘hard science’ – Laboratory-based

• Informal approaches:– Naturalistic, qualitative observations– Informal setting


A note about automated testing/validation

• ‘Should be’ vs ‘is’ – model vs reality

• Great handwriting does not guarantee a compellingly readable result

• Temptation to test the (computationally) obvious


Scenario-based user testing • Based around tasks• Simple scenarios (‘hypothetical

stories’/’abstract-level test cases’):– For a company web page, locating and

using contact details– Registration and login to a wiki

• Process: provide a task and ask the user to complete it– It is important to test the right tasks!


The test process• A facilitator with detailed knowledge about the site/software

is chosen to oversee the test– They must take care not to influence the user’s behaviour!

• The tester (user) is briefed about the site/software• They then go through each scenario

– ‘Think-aloud method’ – describing and explaining actions– ‘Talk-aloud method’ – describing without explanation

(considered more accurate)

• The facilitator keeps notes and prompts the user where necessary

• Alternatively/additionally, the session can be videoed


Creating scenarios• Must be:

– Motivating– Credible– Complex– Provide easy-to-evaluate results

• An Introduction to Scenario Testing, Cem Kaner, Florida Tech, June 2003

– Can be gleaned from documented requirements?

IWMW 2006: User Testing on a Shoestring Budget (1)

Education

Transcript of IWMW 2006: User Testing on a Shoestring Budget (1)