Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software...
Transcript of Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software...
![Page 1: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/1.jpg)
Evaluating and
Improving
Software Usability 902 : Thursday, 9:30am - 10:45am
Philip Lew
www.xbosoft.com
![Page 2: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/2.jpg)
Understand, Evaluate and
Improve
2
![Page 3: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/3.jpg)
Agenda
• Introduc7on
– Importance of usability
– What is usability?
– What is UX?
• Usability Modeling and Measurements
• Case studies on measuring and improving
• Summary
![Page 4: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/4.jpg)
What is Quality?
Pertaining to soEware J
4
Requirements…
![Page 5: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/5.jpg)
Importance of Usability
![Page 6: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/6.jpg)
• Usability is important especially for web applica7ons
– And now especially mobile applica7ons
• Saas, mobile are in alignment with and soon becoming equivalent to cloud • Most prevalent implementa7on method in the next 5 years
• Without good usability:
– Users will leave the applica7ons
– For mobile, if they can’t learn in 30 seconds, they won’t come back
Why is Usability Important?
![Page 7: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/7.jpg)
Web and Mobile have Changed the
Terrain • Business models
have changed
– Instead of paying
upfront and
‘owning’ the
soEware
– Pay as you go, pay
by subscrip7on
• Behavior and
expecta7ons have
changed
![Page 8: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/8.jpg)
What is Usability
In rela7on to quality
In rela7on to user experience
![Page 9: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/9.jpg)
When you think Usability…
• Naviga7on
• Efficiency
• Responsiveness-‐performance
• Learnability
• AZrac7veness
• Understandability
Anything
else come to
mind?
![Page 10: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/10.jpg)
Current Research – Usability
Quality
Usability Satisfaction
ISO 9126-1
ISO 9241-11
Usability Design
Experts ISO 25010
User
Experience
Product Owners
![Page 11: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/11.jpg)
Usability -‐ Key Characteris7c of
Product Quality
Source: ISO 25010
![Page 12: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/12.jpg)
Quality in Use ISO 25010
![Page 13: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/13.jpg)
Notes on Sa7sfac7on and Usability
• Sa7sfac7on is a subjec7ve feeling dependent on many things other than usability:
– A user can be highly sa7sfied but the applica7on with low usability.
– An applica7on can be highly usable (high usability) but the user is not sa7sfied!
Highly usable software
My password doesn’t work
don’t have what I want I’m unsatisfied
I’m satisfied!
! low usability
software
Finished my work today
Nice weather today
![Page 14: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/14.jpg)
Usability-‐Major Component of Quality Quality and Usability in the so1ware development lifecycle
• Can be measured from the design point of
view or of the product
• Can be measured ‘in-‐use’ with real users
Usability
Usability
![Page 15: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/15.jpg)
Evolu7on of SoEware quality
Software processes
Code
Process quality
Software Quality
(internal)
Software Quality
(external)
Software Quality
(in use)
Product ?
CMMI assessment
model
white box testing
black box testing ?
CMMI Assessment
Company
Programmer Tester End User
Type of quality
What is measured
How measured?
Who measures?
ISO 9000 ISO 9241 ISO 9126 ISO 25010
![Page 16: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/16.jpg)
Source: ISO 25010
What is Usability-‐Effect of the
SoEware Product
Degree to which specified
users can achieve specified
goals with effectiveness,
efficiency and satisfaction in a
specified context of use.
![Page 17: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/17.jpg)
Usability in Actual Usage
• User role
• Objec7ve
• Task
• Environment
• Domain
• …
specified users
specified goals
specified context of use
What else can you think of?
![Page 18: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/18.jpg)
Usability
UX, Usability and Quality in Use • Difficult to understand their relationships
• Lack of consensus in meaning in literature
• Not totally clear how related to quality
• Standard definition for user experience is still not available
and User Experience
![Page 19: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/19.jpg)
Some UX Defini7ons
UX DESCRIPTION
D1 a continuous process of user engagement with the product [10] D2 entire set of affects that results in user-product interaction [11] D3 the evolution of usability [12] D4 elaboration of the satisfaction component of usability [13] D5 a categorization of “do-goals” (pragmatics) and “be-goals”(hedonics) [3], [14]
D6 infinite small experiences relating to people, products and contexts [6] D7 consequence of user’s and product’s characteristics when interacted in a
specific environment [1] D8 degree to which specified users can achieve actual usability, safety, and
satisfaction in use in a specified context of use [15] D9 A person's perceptions and responses that result from the use or anticipated use
of a product, system or service [16]
D10 the degree of positive or negative emotions that can be experienced by a
specific user in a specific context during and after product use and that
motivates for further usage [17]
![Page 20: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/20.jpg)
User Experience (UX)
• Pragmatic (do-goals) : refers to the product's
perceived ability to support the task achievement
and focuses on the product’s utility and usability in
completing tasks, the ‘do-goals’ of the user.
• Hedonic (be-goals) : product's ability to support
the user’s achievement of ‘be-goals’, such as
being satisfied, pleasure.
• Real goal of the end user is fulfilling be-goals such
as being autonomous, competent, related to
others, stimulated, etc.”
• Be-goals are the driver of UX
Hassenzahl
![Page 21: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/21.jpg)
21
Rela7ng Usability and User Experience
Quality
characteris7c
508
compliancy Func7onality
User Interface
Usability Learnability
Product
aZributes
Text/Sound
aZributes
Appropriate
Func7ons Easy to use Easy to learn
UX Pragma7c
Do Goals Effec7ve and Efficient
UX Hedonic
Be Goals Sa7sfac7on (pleasure, sense of community…)
Usability
Measurement Effec7veness in use Efficiency in use Learnability in use
Sa7sfac7on
Measurement Pleasure Comfort
Sense of
Community Trust
Context
USER EXPERIENCE
![Page 22: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/22.jpg)
Now that we understand it ALL…
We Need to Define Usability For Our
Own Purposes
22
![Page 23: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/23.jpg)
Let’s Define Usability/UX
From the Product Viewpoint
Usability
Save Loca7on
Characteris7c 1 Characteris7c 2 Characteris7c n
Subcharacteris7c 1 Subcharacteris7c 2 Subcharacteris7c n
AZribute 1 AZribute 2 AZribute n
Understandable Naviga7on
Control Stability Previous-‐Next
Home Loca7on Ease of finding
![Page 24: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/24.jpg)
Defining Usability from an
Effect-‐Real usage Point of View
Quality
Errors
Effec7veness Efficiency Characteris7c n
Accuracy Completeness Subcharacteris7c n
AZribute 2 AZribute 3 AZribute 1
Usability
Sa7sfac7on
![Page 25: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/25.jpg)
Some Poten7al AZributes
Measurements for ‘in use’ Usability • Effec9veness
– Comple7on rates
– Error rate
– Help usage
• Efficiency
– Task 7me
– Backtracking
• Learnability
– Learning rate
– Task 7me devia7on
Can you think
of any others,
par9cular to
your
applica9on?
![Page 26: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/26.jpg)
26
Adding in UX
Sa9sfac9on in use
• Degree to which users are sa7sfied in a specified context of use. Sa7sfac7on is the user’s response to interac7on with the product, including amtudes towards use of the product. Subdivided into sub-‐characteris7cs:
– Likability (cogni7ve sa7sfac7on)
– Pleasure (emo7onal sa7sfac7on)
– Comfort (physical sa7sfac7on)
– Trust
UX Component
![Page 27: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/27.jpg)
External Quality Requirements (for
Shopping Cart Entity)
1 Usability
1.1 Understandability
1.1.1 Icon/label ease to be recognized
1.1.2 Information grouping cohesiveness
1.2 Learnability
1.2.1 ………………………………………………………..
1.3 Ease of Use
1.3.1 Control permanence
1.3.2 Helpfulness
Example of Product Quality Model
with AZributes (Operability in ISO 25010)
![Page 28: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/28.jpg)
28
Learnability
• Degree to which the soEware product enables users to learn its applica7on -‐ 7 principles (ISO 9241-‐10)
– Suitability for the task -‐ should be suitable for the user’s task and skill level
– Self-‐descrip7veness-‐ should be clear what the user should do next
– Controllability -‐ the user should be able to control the pace and sequence of the interac7on
– Conformity with user expecta7ons -‐ should be consistent
– Error tolerant -‐ forgiving
– Suitability for individualiza7on -‐ should be able to be customized to suit the user
– Suitability for learning -‐ should support learning
![Page 29: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/29.jpg)
29
Ease of use
• Degree to which the soEware product makes it easy for users to operate and control it.
– controllability
– error tolerance (by operator)
– conformity with user expecta7ons
![Page 30: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/30.jpg)
30
Ease of Use -‐ Helpfulness
• Degree to which the soEware product
provides help when users need
assistance including help that is:
– easy to find
– comprehensive
– effec7ve
![Page 31: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/31.jpg)
Example: Usability Measurement
31
ACribute
Scale How Calcula9on Goal Current
Help
completeness Percent of
Menu items
with
help
Compare
menus and
help items
% 90% 40%
Ease of
access Keystrokes
to
find/use a
feature/
func7on/
informa7on
Sample 50
items average 3.5 12
Consistency Number
loca7ons for
same buZon
Examine
menus and
doc.
integer 1 5
What types of usability measures are these?
![Page 32: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/32.jpg)
32
Using a measurement model
Measurements
Metrics
Evalua7on
Measurement
Func7on
Indicator
Evalua7on Evalua7on
Sub-‐characteris7cs
Evalua7on Evalua7on Characteris7c
Usability
composed of
composed of
![Page 33: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/33.jpg)
Usability-‐Review
• Can be measured from the design point of
view or of the product
• Can be measured ‘in-‐use’ with real users
usability usability in use
UX
![Page 34: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/34.jpg)
34
Measuring Usability
Methods and models
Now that we know what it is, and
what to measure, then HOW?
![Page 35: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/35.jpg)
Measurable AZributes
• Usability characteris7c
• Descrip7on and purpose
• How to measure
• What is measured
• Measurement/Calcula7on
• Range (min, max)
• Metric
• Objec7ve (goal) versus Current (indicator)
Once you have a
model (what you
are going to
measure), then
you start doing IT!
![Page 36: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/36.jpg)
UX/Usability
Measurement Methods
Let’s get started
Logging
Heuris9c Evalua9on
Observa9on
Labs
Focus groups
Ques9onnaires
![Page 37: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/37.jpg)
37
Measurement methods
Expert Evalua9on (Heuris9c evalua9on)
• Define a set of rules or criteria with measurements and evaluate against them
Web-‐based logs
• Collect user ac:vity data
– Mistakes and errors
– How long it takes to complete tasks
– Comple:on rates
Ques9onnaires
• Quan7ta7ve subjec7ve measurement of UX characteris7cs
![Page 38: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/38.jpg)
Heuris7c Evalua7on
Using a checklist
But a liZle more complicated than
yes/no
38
![Page 39: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/39.jpg)
39
Heuristic Evaluation of a Pharmacy
application Each usability attribute has a quantifying metric.
• For example, for Predic9ve textual aCribute, users should be able to understand a buCon’s results prior to pressing it.
• Direct metrics need to be designed, i.e.:
– (0) No support at all
– (1) Par9al
– (2) Complete
• Need a mapping from 0, 1, 2 to something more understandable, i.e. 2 = 100, and 1 = 60
• Need an indicator to interpret the level of sa9sfac9on met with decision criteria with acceptability ranges in a percentage scale:
– 0-‐40 (unsa9sfactory –red) means changes must take place with high priority;
– 40-‐70 (marginal –yellow) indicates a need for improvement ac9ons;
– 70-‐100 indicates a sa9sfactory level –green-‐ for the analyzed aCribute.
![Page 40: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/40.jpg)
40
2.2.2 Error Recovery Support
" In the current state, users filling a new prescription are supported well in error recovery (2.2.2) in automatic cursor positioning
![Page 41: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/41.jpg)
41
Heuristic Evaluation – Pharmacy Software The purpose of the evaluation • Understand the external quality level of the learnability sub-characteristic for
filling a new prescription
Table 2 - excerpt of the whole current evaluation
![Page 42: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/42.jpg)
External Quality Requirements Measure EI value P/GI value
Global Quality Indicator 61.97%
1 Usability 60.88%
1.1 Understandability 83%
1.1.1 Icon/label ease to be recognized 100%
1.1.2 Information grouping cohesiveness 66%
1.2 Learnability 51.97%
1.2.1 ……………………………………………… …
1.3 Operability 49.50%
1.3.1 Control permanence 100%
1.3.2 Expected behaviour 50%
2 Content Quality 63.05%
2.1 Content Suitability 63.05%
2.1.1 Basic Information Coverage 50%
2.1.1.1 Line item information completeness 2 50%
2.1.1.2 Product description appropriateness 50%
2.1.2 Coverage of other Contextual Information 76.89%
2.1.2.1 ……………………………………………….. …
2.1.2.2 Return policy information completeness 33%
Heuris7c Evalua7on of Usability-‐Shopping Cart
![Page 43: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/43.jpg)
JIRA Heuristic Usability Evaluation
![Page 44: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/44.jpg)
Usability Logging
Collec7ng user ac7vity and behavior
Tradi7onally used for other things
like…
44
![Page 45: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/45.jpg)
Usability Logging
• Iden7fy users by using session ID to iden7fy a unique user.
• Itera7vely insert code into the applica7on
• Collect data
• Analyze the data for each aZribute in different dimensions and aggrega7ons
• Determine the need for further calcula7ons and what aZributes to measure further
• Revise the data we are collec7ng, adding or decreasing granularity
![Page 46: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/46.jpg)
Some easy and some hard
46
![Page 47: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/47.jpg)
47
Jira Evaluation Summary
![Page 48: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/48.jpg)
Using Both
Product Perspec7ve and the User Perspec7ve
• If we find some aZribute with low
performance from the user perspec7ve
• Then we can go back to the product
perspec7ve, figure out why, and improve
48
![Page 49: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/49.jpg)
49
JIRA Evaluation by Attribute
Excerpt
![Page 50: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/50.jpg)
For example:Login BuZon Visibility
(product usability aZribute)
50
![Page 51: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/51.jpg)
Using Ques7onnaires
The process:
1. Use your model of what you want to measure and improve
2. Design the ques7onnaire according to the model
3. Execute the ques7onnaire
4. Analysis
![Page 52: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/52.jpg)
One UX Model
![Page 53: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/53.jpg)
Ques7onnaires Used
SUMI (So1ware Usability Measurement Inventory )
QUIS (Ques:onnaire for User Interface Sa:sfac:on )
PSSUQ (Post-‐Study System Usability Ques:onnaire )
ASQ (A1er-‐Scenario Ques:onnaire )
PUTQ (Purdue Usability Tes:ng Ques:onnaire )
PHUE (Prac:cal Heuris:cs for Usability Evalua:on )
SUS (System Usability Scale )
IUI (Isometrics Usability Inventory)
![Page 54: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/54.jpg)
54
![Page 55: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/55.jpg)
55
![Page 56: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/56.jpg)
What do you do with all this?
56
![Page 57: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/57.jpg)
Itera7ve Improvement
• Measure external quality – usability with
heuris7cs
• Measure in-‐use usability with logging or
ques7onnaires
• Go back to external and improve
![Page 58: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/58.jpg)
Goal is to Understand, Evaluate and Improve
58
![Page 59: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/59.jpg)
Next Steps
• Produce an ac7on plan
– What usability aZributes are important to your
organiza7on?
• Develop a model
– What data can you collect/Which technique can
you use
• Maybe some elements of the model drop out-‐can’t be
measured that easily
• Start collec7ng and developing benchmarks
![Page 60: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/60.jpg)
Conclusion
• Usability and UX are both abstract concepts
– Cri7cal component of quality
• Defining is different for each organiza7on
• Need a model for your organiza7on
• The model is the founda7on of what to
measure
• Once you can measure, then you can evaluate
and improve
![Page 61: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/61.jpg)
Resources
61
These are all journal publications that can be accessed via these links…
![Page 62: Evaluating and Improving Software Usability · PDF fileEvaluating and Improving Software Usability 902: Thursday, ... white box testing black box testing ? ... – Backtracking](https://reader034.fdocuments.us/reader034/viewer/2022051405/5a9cc4e47f8b9a01398b8750/html5/thumbnails/62.jpg)
Thanks
Ques7ons and Answers
Please fill out an evalua:on form and drop it in the
collec:on basket located at the back of the room.
Philip Lew
@xbosoE
phone: 408-‐350-‐0508
www.xbosoE.com