Lecture10 - Around the Beta Release - What Good Enough Means; Usability Evaluations

26 May 2015 CSS 360, Spring 2015, Lecture 10

Outline Good quality vs. “good enough” quality

Usability evaluations - how to…


References “Good Enough Quality: Beyond the Buzzword”, by

James Bach http://www.satisfice.com/articles.shtml

Usability lectures and courses

… by Paul ElRif (Microsoft), James Landay (UW Seattle), and others


Back to Basics:The Goal of Building Software To deliver a product that satisfies the customer(s)

on time on budget with good quality

But wait…


The Big Quality Question


How do we ensure good quality for software?

Improving the Quality of Software

Q: When does software become perfect?


Improving the Quality of Software

Q: When does software become perfect?

Q: How do we make software be perfect?


Improving the Quality of Software:“Good Enough” Quality Q: When does software become perfect? Q: How do we make software be perfect? Q: Is perfection even the goal?

Conclusion: At some point, one needs to stop and decide if the software is “good enough”.

Q: Isn’t “good enough” just a disguised attempt to hide poor quality and/or project team laziness?

Q: Under what conditions do you make this “good enough” decision?


“Good Enough” Quality(James Bach, http://www.satisfice.com/articles.shtml)

At some point, one needs to stop and decide that the software is “good enough”. Under what conditions?

Criteria for “good enough” quality:1. There are clear benefits (of the software).2. There are no critical problems.3. Overall, the benefits outweigh the problems.4. In the present situation and all things considered,

further work would be more harmful than helpful.

Question: Is your product good enough now ?


“Good Enough” Quality (cont.)Important questions to consider:

Good enough for whom? You? Your team? The customer?

Good enough for what? A demo? A beta release? Selling it? Capturing market

share?

Have you agreed on …: team standards for acceptable quality (exit criteria)? what would constitute success for your team in the end?

These are some of the team conversations we did earlier.


Taking a Step Back:What is Quality?

Quality is in the eyes of the beholder (customer). First, define what quality means (for the customer!). You and the customer must agree on the expected level of quality.

What constitutes good quality in one situation may not be considered good quality in another. E.g.: in a toy project vs. in a safety-critical system

A contract must have at least the following components: Who promises to do What for Whom by When with what Quality Criteria/Standards, and with what Notification Mechanism upon completion


Goal–Question–Metric (GQM) Paradigm For each goal …

… What questions will reveal if you’ve met that goal?

For each such question … … What measurements (metrics) can help to answer that

question?

Activity: Assume that goal = “high quality of the shipped software”.

Q: What are some appropriate questions and metrics?


The Beta Release needs to…

… be of sufficiently good quality to enable users of this early prototype to give you meaningful feedback E.g.: no “show-stopping” bugs in the main scenarios that

users will be testing / playing with.

… be given to a selected (limited) group of Beta testers, from the target audience, who can quickly offer feedback on usability-related aspects of your software


Usability throughout a Product Lifecycle


Field studies/surveys

Paper prototyping

Cognitive walkthroughs

Rapid iterative testing

Longitudinal field studies

Instrumented version (throughout cycle)

Usability studies

Baseline/comparative studies

Tim

e

Concept

Specification

Coding

Testing (QA)

Ship

Obtaining User Feedback –Usability Evaluation Process Observe actual users in action

Take notes and/or videotape user interactions

Emphasize that the product is being tested, not the users

Users perform controlled tasks with your WIP product Do not tell them how to accomplish something they need!

Do answer their (clarifying) questions, though.

Users are encouraged to think aloud Ask them to clarify what they mean, if needed


How Many Users to Recruit for a Usability Evaluation?

problems found benefits / costs

Caveat: The graphs are for a specific example.


Common Usability Questions to Consider Before/During Evaluations What user needs can the product satisfy?

What user needs does the current product miss?

What design(s) will solve those problems?

What can users currently do with the product?

How can the user succeed at task X?

Is the product enjoyable / compelling to use?

What can other similar products do for users?


Usability Evaluation:Pragmatics It saves a lot of time and money by catching

problems early enough Less wasted effort on unneeded features

Decreases the need for large and expensive changes to the implementation

Usability Evaluation takes time, so plan for it! Don’t allow obvious usability bugs to interfere with

receiving useful feedback.


Dispelling Some Myths or…How to Do Usability Badly? “Users should read the documentation first”

“Users will figure things out” Modal interfaces (if Y do X, else do Z) can be confusing

“Users want more features” The paradox of choice: feature-richness vs. ease of use

“Users are our testers. If they complain, we’ll fix.” Legacy UI is tremendously hard to change once inertia has

picked up (e.g., QWERTY keyboards)

Remember: Good UX is expensive; bad UX is more expensive!


How to Do Usability Badly?


Lecture10 - Around the Beta Release - What Good Enough Means; Usability Evaluations

Documents

Transcript of Lecture10 - Around the Beta Release - What Good Enough Means; Usability Evaluations