Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and...
-
Upload
danielle-mccall -
Category
Documents
-
view
214 -
download
0
Transcript of Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and...
![Page 1: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/1.jpg)
Methodology and ExplanationXX50125
Lecture 2Part 1: Usability testing
Part 2: Interviews and questionnaires
Dr. Danaë Stanton Fraser
![Page 2: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/2.jpg)
Methodology and Explanation 2007
Lectures
1. Introduction to Methods
2. Experiments
3. Usability testing
4. Interviews/questionnaires
5. Ethnography
![Page 3: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/3.jpg)
Methodology and Explanation 2007
“Know thy user” Hansen (1971).
![Page 4: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/4.jpg)
Methodology and Explanation 2007
What is Usability Testing?
Usability testing is a systematic approach to evaluating user performance in order to inform design.
Designs should be: • Based on observation of users• Refined by thoughtful analysis of the frequency and
sequences in which tasks are carried out• Validated through early prototype, usability tests• Direct interaction with users during the design phase
and throughout the system lifecycle
![Page 5: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/5.jpg)
Methodology and Explanation 2007
The Move towards Usability Testing
Traditionally, computer programmers designed for computer programmers
Designs should be: • Based on observation of users• Refined by thoughtful analysis of the frequency and
sequences in which tasks are carried out• Validated through early prototype, usability tests• Direct interaction with users during the design phase
and throughout the system lifecycle
![Page 6: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/6.jpg)
Methodology and Explanation 2007
• Iterative design allowing for early testing of prototypes, revisions based on feedback and then incremental refinements
• When competitive products provide similar functionality usability testing important for product acceptance
• Karat (1994) at IBM reported up to $100 payoffs for each dollar spent on usability, with benefits in reduced program-development costs, reduced programme maintenance costs, increased revenue due to higher customer satisfaction, and improved user efficiency and productivity.
![Page 7: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/7.jpg)
Methodology and Explanation 2007
Design is a process..
![Page 8: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/8.jpg)
Example :Designing Games
• Interdisciplinary team – audio and visual design, programming, project management etc…
• Iterative design process
• cost benefits
• does design process prevent creative coding?
• Design process should be a shared vision that is built upon
![Page 9: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/9.jpg)
Example:Designing Games
• A videogame takes approx. 30 to 36 months from inception to market. A Pre-production phase (prototyping) will generally take 12 months plus, and if it meets approval, will get funded for full development
• A PS2 videogame in the US will typically have a team of 50 plus people working on it, and will cost $5m plus (Jeffery, 2003, Lucas Arts)
![Page 10: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/10.jpg)
Methodology and Explanation 2007
Usability Testing and Laboratories
• Emerged since the early 1980’s • To provide expert reviews • To conduct usability tests of products • Usability testing often found to increase speed of
projects and produce cost savings• Usability tests are designed to find flaws in user
interfaces• Includes set of tasks, fewer participants than
experimental study and a report of recommended changes.
![Page 11: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/11.jpg)
Methodology and Explanation 2007
Typical procedure
• Usability staff:– Participate in early task analysis/design review– Provide information on software tools, literature– Develop set of tasks for testing (numbers, types of
participants)– Pilot test– Voluntary participation with informed consent– Think aloud technique– Video recording
![Page 12: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/12.jpg)
Methodology and Explanation 2007
Many types of Usability Testing
• Discount usability testing• Field tests• Paper prototypes• Can-you-break-this• Competitive usability testing
• Two main limitations:– Emphasises first time use– Limited coverage of features
• Therefore often supplemented with expert review
![Page 13: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/13.jpg)
Methodology and Explanation 2007
Expert Reviews
• Heuristic evaluation• Guidelines review• Consistency inspection• Cognitive walkthrough• Formal usability inspection
![Page 14: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/14.jpg)
Methodology and Explanation 2007
Part 2: Interviews and questionnaires
![Page 15: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/15.jpg)
Methodology and Explanation 2007
Why Interview?
• Good for obtaining information about– Opinions– Feelings– Goals (organisational, personal and system)– Things you didn’t know you needed to know
• Bad for gathering (large volumes of) data
![Page 16: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/16.jpg)
Methodology and Explanation 2007
Interview Preparation
• Decide objectives• Read background material• Decide whom to interview• Set up the interview• Decide interview structure• Write down questions you will ask and pointers for
questions you might ask
![Page 17: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/17.jpg)
Methodology and Explanation 2007
Open Questions
E.g. “How do you find using this technology in your lectures?”
• Little control• Hard to analyse • Hard to keep focussed on interview objectives• May unsettle interviewee• Time consuming and can be difficult to analyse
However:• May gain more depth, rich data• May tell you things you had not considered• May give you ideas for future projects
![Page 18: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/18.jpg)
Methodology and Explanation 2007
Closed Questions
E.g. “How many courses do you teach?” “How many times have you used this technology?”• May settle interviewee• Easy to analyse• Easy to maintain focus• Observation and use of data may be more efficient
However:• Data may not be as rich
![Page 19: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/19.jpg)
Methodology and Explanation 2007
Question Pitfalls
• Leading questions– “How bad is the current system?”
• Double questions/poor structure– “How often does the system fail, and how do you
fix it when it does?”
• Stay focussed. If you are unlikely to be able to make use of the answer don’t ask the question!
![Page 20: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/20.jpg)
Methodology and Explanation 2007
Question Structure
• Pyramid
– Start with closed questions, finish with open
– Good for settling nervous interviewees as answers are ‘easy’ early on
• Funnel
– Start with open questions, finish with closed
– Good for aggressive interviewees as opinions are asked for early on• Diamond
– Closed -> Open -> Closed, Good all-round technique• Structured/unstructured
– All interviews may vary depending on how focused you intend to be on the prepared structure
![Page 21: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/21.jpg)
Methodology and Explanation 2007
Starting the Interview
• Dress• Greeting – first time
– Who you are – Why you want to interview them– What objectives you want to achieve from the interview– That the information they give is confidential and how it
might be used• Subsequent greetings should discuss an issue from or
summarise the previous interview• Don’t just “be yourself”
![Page 22: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/22.jpg)
Methodology and Explanation 2007
During the Interview
• Introduce each line of questioning
• Probing
– E.g. “Why?”
– Shows that you are listening
– Good for keeping interview focus
– Double-check information and notes• Taking notes
– Not too many
– Maintain eye contact
– Go back over important points• Use of sound recorders
![Page 23: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/23.jpg)
Methodology and Explanation 2007
Focus Groups
• Normally 3-10 people
• A representative sample of typical users
• Enables people to put forward ideas in a supportive environment
• Facilitator guides and prompts
• Discussion recorded
• Low-cost but need good facilitator
![Page 24: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/24.jpg)
Methodology and Explanation 2007
Questionnaires
• For wider coverage
• Need to know the right questions (through interviews and background)
• Focus on objectives
• Don’t ask unnecessary questions
• Keep it short and facilitate response
• Open and Closed questions
• Scaling
![Page 25: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/25.jpg)
Methodology and Explanation 2007
Likert Scale
• Short statements about the features to be evaluated
• Divide the items into groups with same number of positive and negative statements in each group
• Decide on scale – even or odd, number of points
strongly agree
agree neutral disagree strongly disagree
![Page 26: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/26.jpg)
Methodology and Explanation 2007
Questionnaire difficulties
• The ‘halo effect’– Respondent answers everything the same by not
differentiating between questions– Mix questions up on different subjects
• Leniency– Some respondents rate everything too well– Move the ‘average’ category left (or right) of centre and
make scale descriptors stronger• Central tendency
– Respondents rate everything as average– Increase the number of points on the scale and adjust the
scale descriptors to cover a smaller range of opinions
![Page 27: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/27.jpg)
Methodology and Explanation 2007
Also…evaluation during active use
• System refinement as a result of experience or in response to changes in users
– interviews and focus group discussions
– continuous user-performance data logging
• look at both frequent and infrequent error messages
• analyse sequences of actions to suggest improvements or new actions
• BUT respect people’s rights and consult them first!
– User feedback mechanisms
• on-line forms, email and bulletin boards
• workshops and conferences
![Page 28: Methodology and Explanation XX50125 Lecture 2 Part 1: Usability testing Part 2: Interviews and questionnaires Dr. Danaë Stanton Fraser.](https://reader033.fdocuments.us/reader033/viewer/2022051515/5515e0a2550346dd6f8b4ce9/html5/thumbnails/28.jpg)
Methodology and Explanation 2007
Preparation for Practical