CMC/CC A

Usability Evaluation

Master IK, CIW, MMI

L.M. Bosveld-de Smet

Course 4; mon. 02/10/06; 16.00-18.00

Usability Evaluation

Design of interactionDix et al. (2004)

What is wanted

Analysis

Design

Implementand deploy

Prototype

Usability

Evaluation

Design process of usable interactive systems

Interaction design process Complex Iterative Never complete

User-centered design Within software engineering framework

Design rules Implementation support Evaluation techniques Universally accessible designs Provision of user support

3 related concepts

Usability: measure of success of a product User-Centered Design: design involving user

participation Usability Engineering:

Whole process ensuring usable interactive systems Process aiming at systems fit for the user, the task, the

environment Process implying user participation during software development Process committing itself to design-evaluate-redesign

development cycle In search for objective measures for user interface

Usability: different views

Same intuitions vs. Different approaches Different views on:

Definition of usability Usability attributes

Importance of user-centered design When and how to deal with user participation How to build usable systems How to measure usability

Usability Attributes: Shackel’s view “A usable product is one that users find

satisfactory for the tasks for which it was designed.”

“Good design for usability depends upon achieving successful harmony in the dynamic interplay between user, task, system, and environment.”

Usability: 4 usable criteria Learnability Effectiveness Attitude Flexibility

Usability Attributes: ISO definition

“Usability is the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments.” Effectiveness: “accuracy and completeness with which

users achieve specific goals” Efficiency: “accuracy and completeness of goals in

relation to resources expended” Satisfaction: “comfort and acceptability of the system”

An aside: User Interface Standards

Disadvantages: Constrain design Stagnate innovation Describe principles, do not provide solutions Become quickly obsolete

Advantages: Define ‘good practice’ Affect attitudes w.r.t. ‘software development’

User-Centered Design: Karat’s view

Karat (1996) “UCD: Quality or Quackery?”

“UCD is an iterative process whose goal is the development of usable systems, achieved through involvement of potential users of a system in system design”

“I suggest we consider UCD a nice fluffy little catch phrase. It captures a commitment that the usability community supports – that you must involve users in system design – while leaving fairly open how this is accomplished”

UCD: a more objective view

UCD can be accomplished through the application of Usability Engineering

Design should centre on users End-user should be consulted Needs of end-users should be considered

HCI in software process

Software engineering Usability engineering Iterative design practices Design rationale

Waterfall model

Requirements Specification

Architectural design

DetailedDesign

Implementationand Unit testing

Integration andTesting

Operation andMaintenance

Usability Engineering

Usability specification as part of requirements specification

Iteration and testing List of usability measurement requirements

(Whiteside, Bennett and Holtzblatt, 1988) Time to complete a task Ratio of successes to failures Time spent in errors Number of commands used Frequency of help and documentation use …

Iteration in waterfall model

Requirements Specification

Architectural design

DetailedDesign

Implementationand Unit testing

Integration andTesting

Operation andMaintenance

Usability Engineering LifecycleFaulkner (2000)

Know the user Know the task User requirements capture Setting usability goals Design process Apply guidelines, heuristics Prototyping Evaluation with users Redesign and evaluation with users Evaluation with users and report

Role of prototyping

Design

Redesign

Prototype Evaluate Done!OK?

Not OK?

Iterative design and prototyping

Three main approaches to prototyping Throw-away prototyping within requirements

specification Incremental prototyping within the life cycle Evolutionary prototyping throughout the life cycle

Throw-away prototyping

PreliminaryRequirements

BuildPrototype

EvaluatePrototype

FinalRequirements

Adequate?yes

no

Evolutionary prototyping

Build Prototype

Evaluate Prototype

Operation andMaintenance

Req

Arch

Det

Impl

Int

Usability Evaluation (UE)

Methodologies for measuring usability aspects of system’s user interface and identifying specific problems (Dix et al. 1998; Nielsen 1993)

Should occur throughout design life cycle Common activities:

Capture Analysis Critic

There is a wide range of UE techniques Each technique has its own requirements Different techniques uncover different usability problems

Taxonomies of UE (1)

Formative vs. Summative Analytical vs. Empirical Expert analysis vs. User participation

Analytic methods Review methods Model-based methodsvs. Experimental methods Observational methods Query methods

Taxonomies of UE (2)

Automated vs. Non-automated Ivory and Hearst (2001):

Testing Inspection Inquiry Analytical modeling Simulation

Expert Analysis

Controlled Experiment

User participation

Important factors choice UE

Stage in cycle at which UE is carried out Style of UE Level of subjectivity or objectivity of UE

technique Type of measures provided Information provided Immediacy of response Level of interference implied Resources required

Analytic UE techniques

Cognitive Walkthrough

Heuristic Evaluation

Review based Model based

Stage throughout throughout design design

Style laboratory laboratory laboratory laboratory

Objective? no no as source as source

Measure qualitative qualitative as source qualitative

Information low level high level as source low level

Immediacy N/A N/A as source N/A

Intrusive? no no no no

Time medium low low-medium medium

Equipment low low low low

Expertise high medium low high

Experimental and query UE techniques

Experiment Interviews Questionnaire

Stage throughout throughout throughout

Style laboratory lab / field lab / field

Objective? yes no no

Measure quantitative qualitative / quantitative

qualitative / quantitative

Information low/high level high level high level

Immediacy yes no no

Intrusive? yes no no

Time high low low

Equipment medium low low

Expertise medium low low

Observational UE techniques

Think aloud Protocol analysis

Post-task walkthrough

Stage implementation implementation implementation

Style lab / field lab / field lab / field

Objective? no no no

Measure qualitative qualitative qualitative

Information high/low level high/low level high/low level

Immediacy yes yes no

Intrusive? yes yes no

Time high high medium

Equipment low high low

Expertise medium high medium

Monitoring UE techniquesEye tracking Physiological

measurement

Stage implementation implementation

Style laboratory laboratory

Objective? yes yes

Measure quantitative quantitative

Information low level low level

Immediacy yes yes

Intrusive? no yes

Time medium/high medium/high

Equipment high high

Expertise high high

Vocabulary application

Task analysis Feasability study Design representation

Feasibility study

Why is system needed? How will system help to improve user task

performance? Are there critical processes that need to be

supported by system? What are the technical implications? Can system be produced within given budget? Is there a timescale for the development of the

system?

Strategies for representing design

Storyboards State transition diagrams Simulations Scenarios Rapid prototyping Wizard of Oz …

Storyboard

State Transition Diagram