CS 5150 1
CS 5150 Software Engineering
Lecture 12
Usability 2
CS 5150 2
Course Administration
Test 2
Answer books can be collected from the reception at 301 College Avenue
Mid-semester break
Next lecture is Wednesday, October 14
CS 5150 3
Usability: The Analyze/Design/Evaluate Loop
Evaluate
?Design
Build
Analyze requirements
User testing
CS 5150 4
Evaluation
• Iterative improvements during development.
• Making sure that a system is usable before launching it.
• Iterative improvements after launch.
• Categories of evaluation methods:
Analytical evaluation: without users (not in CS 5150)
Measurements on operational systems
Empirical evaluation: with users
CS 5150 5
Evaluation
How do you measure usability?
Usability comprises the following aspects:
• Effectiveness – the accuracy and completeness with which users achieve certain goals Measures: quality of solution, error rates
• Efficiency – the relation between the effectiveness and the resources expended in achieving themMeasures: task completion time, learning time, clicks number
• Satisfaction – the users' comfort with and positive attitudes towards the use of the systemMeasures: attitude rating scales
From ISO 9241-11
CS 5150 6
Evaluation based on Measurement
Basic concept: log events in the users' interactions with a system
Examples from a Web system
• Clicks (when, where on screen, etc.)
• Navigation (from page to page)
• Keystrokes (e.g., input typed on keyboard)
• Use of help system
• Errors
May be used for statistical analysis or for detailed tracking of individual user.
CS 5150 7
Evaluation based on Measurements
Analysis of system logs
• Which user interface options were used?
• When was was the help system used?
• What errors occurred and how often?
• Which hyperlinks were followed (click through data)?
Human feedback
• Complaints and praise
• Bug reports
• Requests made to customer service
CS 5150 8
The Search Explorer Application: Reconstruct a User Sessions
CS 5150 9
Evaluation with Users
Testing the system, not the users!
Stages of evaluation with users:
Prepare
Conduct sessions
Analyze results
User testing is time-consuming, expensive, and essential.
CS 5150 10
Evaluation with Users: Preparation
• Determine goals of the usability testing
“Can a user find the required information in no more than 2 minutes?”
• Write the user tasks
“Answer the question: how hot is the sun?”
• Recruit participants
Use the descriptions of users from the requirements phase to determine categories of potential users
CS 5150 11
Usability Laboratory
Concept: monitor users while they use system
Evaluators User
one-way mirror
CS 5150 12
Evaluation with Users: Sessions
• Conduct the session– Usability Lab– Simulated working
environment
• Observe the user– Human observer(s)– Video camera– Audio recording
• Inquire satisfaction data
CS 5150 13
Evaluation with users:Results analysis
• If possible, use statistical summaries.
• Pay close attention to areas where users
– were frustrated
– took a long time
– could not complete tasks
• Respect the data and users' responses. Do not make excuses for designs that failed.
• Note designs that worked and make sure they are incorporated in the final product.
CS 5150 14
Evaluation Example: Eye Tracking
CS 5150 15
Evaluation Example: Eye Tracking
CS 5150 16
Refining the Design based on Evaluation
Do not allow evaluators to become designers
Designers are poor evaluators of their own work, but know the requirements, constraints, and context of the design:
• Some user problems can be addressed with small changes
• Some user problems require major changes
• Some user requests (e.g., lots of options) are incompatible with other requests (e.g., simplicity)
Designers and evaluators need to work as a team
CS 5150 17
Usability: Design Tensions in Networked Systems
Designers wish to control what the user sees, but users wish to configure their own environments.
• Accessibility requires that designers do not control parameters such as font size.
• Client computers and network connections vary greatly in capacity
• Client software may run on various operating systems, which may not be the current version.
Be explicit about the assumptions you make about the user's computer, Web browser, etc.
CS 5150 18
Usability and Cost
• Good usability may be expensive in hardware or special software development
• User interface development may be a major part of a software development project
Programming environments provide powerful user interface toolkits
• Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems
Web browsers provide a general purpose user interface where
others maintain the user interface software
CS 5150 19
Changes in user interface design
Examples of change: 1995 to 2009
CS 5150 20
1990
SEARCH I NSPEC Dat abase- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Type keywor ds and pr ess RETURN - - orent er a command
Def aul t i s ADJ : aci d f r ee- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Set #3: aci d adj f r e 0 r ecor ds I NSPEC Dat abase
Set #4: aci d adj f r ee 5 r ecor ds I NSPEC Dat abase
Set #5: aci d and paper 448 r ecor ds I NSPEC Dat abase
Set #6: deaci di f i cat i on 4 r ecor ds I NSPEC Dat abase- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CS 5150 21
1995
CS 5150 22
2003
CS 5150 23
2009
CS 5150 24
2001
CS 5150 25
2003
CS 5150 26
2009
CS 5150 27
1995
CS 5150 28
2006
CS 5150 29
1995
CS 5150 30
2003
CS 5150 31
2009
CS 5150 32
1995
CS 5150 33
2006
Top Related