Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and...
-
date post
18-Dec-2015 -
Category
Documents
-
view
219 -
download
2
Transcript of Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and...
![Page 1: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/1.jpg)
2FJK 2005-2011
User-Centered Design and Development
Instructor: Franz J. Kurfess
Computer Science Dept.
Cal Poly San Luis Obispo
![Page 2: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/2.jpg)
5FJK 2005-2011
484-W10 Quarter
• The set of slides I use in class is close to the one in the PowerPoint Custom Show “484-W09”. Since I’m using mostly Keynote now, I use the “Skip” feature to achieve a similar result.
![Page 3: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/3.jpg)
6FJK 2005-2011
Chapter Overview
• user testing• DECIDE framework for user testing• experiments
– variables and conditions– data collection and analysis
• predictive models– GOMs– keystroke level model
![Page 4: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/4.jpg)
7FJK 2005-2011
Motivation
• in addition to asking users about their experiences or tasks, it may be necessary to conduct tests where they perform tasks
• ideally, these tests should be done with the final product– in practice, this is often not feasible– user testing can also be done on prototypes to
find out how well the general approach works
• user modeling tries to predict user performance for tasks performed on a system
![Page 5: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/5.jpg)
8FJK 2005-2011
Objectives
• become familiar with the main principles and constraints of user testing
be aware of the differences between user testing, usability testing and research experiments
be able to design simple experiments become familiar with the GOMS user model, the
keystroke level model, and Fitts’ law determine when these techniques can be applied know how to do a keystroke level analysis
![Page 6: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/6.jpg)
10
Usability testing• Involves recording performance of typical
users doing typical tasks.• Controlled environmental settings. • Users are observed and timed.• Data is recorded on video & key presses are
logged. • The data is used to calculate performance
times, and to identify & explain errors. • User satisfaction is evaluated using
questionnaires & interviews. • Field observations may be used to provide
contextual understanding.
![Page 7: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/7.jpg)
11
Experiments & usability testing
• Experiments test hypotheses to discover new knowledge by investigating the relationship between two or more things – i.e., variables.
• Usability testing is applied experimentation. • Developers check that the system is usable
by the intended user population for their tasks.
• Experiments may also be done in usability testing.
![Page 8: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/8.jpg)
12
Usability testing & researchUsability testing
• Improve products• Few participants• Results inform design• Usually not
completely replicable• Conditions controlled
as much as possible• Procedure planned• Results reported to
developers
Experiments for research
• Discover knowledge• Many participants• Results validated
statistically • Must be replicable• Strongly controlled
conditions• Experimental design• Scientific reported to
scientific community
![Page 9: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/9.jpg)
13
Usability testing• Goals & questions focus on how well
users perform tasks with the product.• Comparison of products or prototypes
common.• Focus is on time to complete task &
number & type of errors.• Data collected by video & interaction
logging.• Testing is central.• User satisfaction questionnaires &
interviews provide data about users’ opinions.
![Page 10: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/10.jpg)
14
Usability lab with observers watching a user & assistant
![Page 11: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/11.jpg)
15
Portable equipment for use in the field
![Page 12: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/12.jpg)
16
![Page 13: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/13.jpg)
17
Testing conditions• Usability lab or other controlled space.• Emphasis on:
– selecting representative users;– developing representative tasks.
• 5-10 users typically selected.• Tasks usually last no more than 30
minutes.• The test conditions should be the same for
every participant.• Informed consent form explains
procedures and deals with ethical issues.
![Page 14: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/14.jpg)
18
Some type of data Time to complete a task. Time to complete a task after a specified.
time away from the product. Number and type of errors per task. Number of errors per unit of time. Number of navigations to online help or
manuals. Number of users making a particular
error. Number of users completing task
successfully.
![Page 15: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/15.jpg)
19
Usability engineering orientation
Aim is improvement with each version.
Current level of performance. Minimum acceptable level of
performance.Target level of performance.
![Page 16: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/16.jpg)
20
How many participants is enough for user testing?
• The number is a practical issue.• Depends on:
– schedule for testing;– availability of participants;– cost of running tests.
• Typically 5-10 participants. • Some experts argue that testing
should continue until no new insights are gained.
![Page 17: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/17.jpg)
22
FJK 2005-2011
User Testing with DECIDE
• Determine the goals and Explore the questions
• Choose the paradigm and techniques
• Identify practical issues• Deal with ethical issues• Evaluate, analyze, and present
data
![Page 18: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/18.jpg)
23
Experiments
• Predict the relationship between two or more variables.
• Independent variable is manipulated by the researcher.
• Dependent variable depends on the independent variable.
• Typical experimental designs have one or two independent variable.
![Page 19: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/19.jpg)
24
Experimental designs
• Different participants - single group of participants is allocated randomly to the experimental conditions.
• Same participants - all participants appear in both conditions.
• Matched participants - participants are matched in pairs, e.g., based on expertise, gender, etc.
![Page 20: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/20.jpg)
25
Different, same, matched participant design
![Page 21: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/21.jpg)
26
Field studies• Field studies are done in natural settings.• The aim is to understand what users do
naturally and how technology impacts them.
• Field studies can be used in product design to:- identify opportunities for new technology;- determine design requirements; - decide how best to introduce new technology;- evaluate technology in use.
![Page 22: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/22.jpg)
27
Data collection & analysis
• Observation & interviews– Notes, pictures, recordings– Video– Logging
• Analyzes– Categorized– Categories can be provided by theory
• Grounded theory• Activity theory
![Page 23: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/23.jpg)
28
![Page 24: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/24.jpg)
29
Example of categories from Activity Theory
where-to artefacts The cinema experience as it is formed in the shop, in sales brochures, and at Sarah’s sister’s house. why artefacts Press number button for number of loudspeakers to be connected. how artefacts Press LIFT [LIST] button a few times. what artefacts The electronic curtains rising and falling.
![Page 25: Chapter 14: Usability testing and field studies. 2 FJK 2005-2011 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept.](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d255503460f949fbbbe/html5/thumbnails/25.jpg)
30
Key points Testing is a central part of usability testing. Usability testing is done in controlled conditions. Usability testing is an adapted form of experimentation. Experiments aim to test hypotheses by manipulating certain
variables while keeping others constant. The experimenter controls the independent variable(s) but
not the dependent variable(s). There are three types of experimental design: different-
participants, same- participants, & matched participants. Field studies are done in natural environments. Typically observation and interviews are used to collect field
studies data. Categorization and theory-based techniques are used to
analyze the data.