CS 5150 1

CS 5150 Software Engineering

Lecture 12

Usability 2

CS 5150 2

Course Administration

Test 2

Answer books can be collected from the reception at 301 College Avenue

Mid-semester break

Next lecture is Wednesday, October 14

CS 5150 3

Usability: The Analyze/Design/Evaluate Loop

Evaluate

?Design

Build

Analyze requirements

User testing

CS 5150 4

Evaluation

• Iterative improvements during development.

• Making sure that a system is usable before launching it.

• Iterative improvements after launch.

• Categories of evaluation methods:

Analytical evaluation: without users (not in CS 5150)

Measurements on operational systems

Empirical evaluation: with users

CS 5150 5

Evaluation

How do you measure usability?

Usability comprises the following aspects:

• Effectiveness – the accuracy and completeness with which users achieve certain goals Measures: quality of solution, error rates

• Efficiency – the relation between the effectiveness and the resources expended in achieving themMeasures: task completion time, learning time, clicks number

• Satisfaction – the users' comfort with and positive attitudes towards the use of the systemMeasures: attitude rating scales

From ISO 9241-11

CS 5150 6

Evaluation based on Measurement

Basic concept: log events in the users' interactions with a system

Examples from a Web system

• Clicks (when, where on screen, etc.)

• Navigation (from page to page)

• Keystrokes (e.g., input typed on keyboard)

• Use of help system

• Errors

May be used for statistical analysis or for detailed tracking of individual user.

CS 5150 7

Evaluation based on Measurements

Analysis of system logs

• Which user interface options were used?

• When was was the help system used?

• What errors occurred and how often?

• Which hyperlinks were followed (click through data)?

Human feedback

• Complaints and praise

• Bug reports

• Requests made to customer service

CS 5150 8

The Search Explorer Application: Reconstruct a User Sessions

CS 5150 9

Evaluation with Users

Testing the system, not the users!

Stages of evaluation with users:

Prepare

Conduct sessions

Analyze results

User testing is time-consuming, expensive, and essential.

CS 5150 10

Evaluation with Users: Preparation

• Determine goals of the usability testing

“Can a user find the required information in no more than 2 minutes?”

• Write the user tasks

“Answer the question: how hot is the sun?”

• Recruit participants

Use the descriptions of users from the requirements phase to determine categories of potential users

CS 5150 11

Usability Laboratory

Concept: monitor users while they use system

Evaluators User

one-way mirror

CS 5150 12

Evaluation with Users: Sessions

• Conduct the session– Usability Lab– Simulated working

environment

• Observe the user– Human observer(s)– Video camera– Audio recording

• Inquire satisfaction data

CS 5150 13

Evaluation with users:Results analysis

• If possible, use statistical summaries.

• Pay close attention to areas where users

– were frustrated

– took a long time

– could not complete tasks

• Respect the data and users' responses. Do not make excuses for designs that failed.

• Note designs that worked and make sure they are incorporated in the final product.

CS 5150 14

Evaluation Example: Eye Tracking

CS 5150 15

Evaluation Example: Eye Tracking

CS 5150 16

Refining the Design based on Evaluation

Do not allow evaluators to become designers

Designers are poor evaluators of their own work, but know the requirements, constraints, and context of the design:

• Some user problems can be addressed with small changes

• Some user problems require major changes

• Some user requests (e.g., lots of options) are incompatible with other requests (e.g., simplicity)

Designers and evaluators need to work as a team

CS 5150 17

Usability: Design Tensions in Networked Systems

Designers wish to control what the user sees, but users wish to configure their own environments.

• Accessibility requires that designers do not control parameters such as font size.

• Client computers and network connections vary greatly in capacity

• Client software may run on various operating systems, which may not be the current version.

Be explicit about the assumptions you make about the user's computer, Web browser, etc.

CS 5150 18

Usability and Cost

• Good usability may be expensive in hardware or special software development

• User interface development may be a major part of a software development project

Programming environments provide powerful user interface toolkits

• Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems

Web browsers provide a general purpose user interface where

others maintain the user interface software

CS 5150 19

Changes in user interface design

Examples of change: 1995 to 2009

CS 5150 20

1990

SEARCH I NSPEC Dat abase- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Type keywor ds and pr ess RETURN - - orent er a command

Def aul t i s ADJ : aci d f r ee- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Set #3: aci d adj f r e 0 r ecor ds I NSPEC Dat abase

Set #4: aci d adj f r ee 5 r ecor ds I NSPEC Dat abase

Set #5: aci d and paper 448 r ecor ds I NSPEC Dat abase

Set #6: deaci di f i cat i on 4 r ecor ds I NSPEC Dat abase- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

CS 5150 21

1995

CS 5150 22

2003

CS 5150 23

2009

CS 5150 24

2001

CS 5150 25

2003

CS 5150 26

2009

CS 5150 27

1995

CS 5150 28

2006

CS 5150 29

1995

CS 5150 30

2003

CS 5150 31

2009

CS 5150 32

1995

CS 5150 33

2006