Usability Testing Script

Client: Project:Date:

Usability Testing Script, Method, Template

What is Usability Testing?

Usability testing is a method where representative users try typical tasks on a product such as a website or prototype. Generally, a moderator will be sitting with the user. A second person may be observing the session and taking notes.

The purpose of testing is to identify any usability issues that occur, collect qualitative data on the users’ experience on the website and find out how satisfied users are with the product.

A usability testing script is used to help:

Remind the moderator which topic or elements need to be covered,

Ensure that specific topics/tasks are covered during the session within the timings of the interview

Facilitate the discussion with the respondent

Who Runs Usability Testing?

The project manager should be aware of the different factors on the checklist at the start from the project and confirm and enter the details as early as possible.

Ideally, an experienced moderator will run the session with a second person observing and taking notes.

Prior to testing, it is useful to run through the script to see if the timings fit and the script flows well. It also allows you to see if the tasks work and the website/equipment is behaving as it should. You can ask a colleague to play the role of the participant while you moderate.

Again, the key internal stakeholders in the project should review the usability testing script to make sure all the key elements are covered.

Why do Usability Testing?

Usability testing gives you a chance to see real users use your product/site in a way that simulates the real user experience. Things will happen during the usability testing that you can never predict. No matter how well designed a product/website is, it will invariably confuse users.

Usability testing clearly indicates which areas work well and what improvements are necessary to improve the user experience. Usability is measured in terms of efficiency, effectiveness and satisfaction.

How?

Test with a reasonable number of participants—at least five and no more than 24 dependent on the key user group(s) you are testing with.

Things to remember when moderating a test:

- Don’t explain the interface the participant is working with to avoid getting a biased feedback- Keep the participant focused on the task at hand- Leave the participant to make mistakes without helping them out but investigate any

issues/mistakes later on- Answer questions with questions to get more feedback- Probe expectations and use the question ‘why?’ to probe further- Take notes either on paper or electronically dependent on if an observer is available to do this or

not

The usability testing checklist and script template can be found in this document. Note: It is not a questionnaire or a fully comprehensive list of questions. Questions should not to be read out as they is

Objective Digital Pty Ltd 3/50 York St Sydney NSW 2000 T 1300 85 80 15 ABN 98 121 747 188www.objectivedigital.com

1

http://www.objectivedigital.com/


written; the moderator should use the prompts to form their own questions. The moderator should probe and follow-up respondents behaviour, comments and language.

There are two methods used in the script: RTA and Think Aloud. Each technique has its own advantages.

What is RTA ?

RTA stands for Retrospective Think Aloud protocol. With eye tracking hardware and software, users are asked to complete a task silently and on their own. Once the task is complete, the moderator plays back the dynamic eye tracking video while asking the user about their underlying motivations and experiences. This allows the participant to get on with their task so they are not distracted during the task. A user’s feedback is often more considered when replaying the eye tracking video.

What is Think Aloud?

Think aloud is widely used in usability research. During a task, the user is verbalising their actions and thoughts out loud whilst navigating the site. This technique gives immediate feedback during a task and means the participant is not relying on memory to provide feedback on things they experienced. However, the participant is under more cognitive load (talking to the facilitator) than they would be if they would be sitting at home and this may change their behaviour.

When?

Test early and test often.

The earlier you start with testing the easier and less expensive it is to change the system. The more you test, the more confident you can be that the end product will actually meet the users’ needs and the experience is optimised.

You can test paper prototypes or HTML prototypes of working websites. What matters is that you test throughout the UCD project to ensure a successful outcome.


2



Usability testing checklist

Research details Notes

Method 1:1 interview, RTA.

Location of research

Number of days of research in location and dates

Number of interviews per day

Respondent requirements

Dates and times of interviews

Drinks – what and time

Food – what and time

Incentive receipt/Consent forms

Confidentiality forms (if required)

Client requirements

Time of client arrival on day/s

Number of clients for food (breakfast, lunch, dinner) – type of food and time

Research room requirements

PC laptop with usability testing software

Eye tracker

Mouse

Keyboard

Microphone

Cables i.e. LAN, power etc.

Other


3



Testing script templateLocation: Time:

Participant: Date:

Facilitator:

Facilitator script:

Welcome

Thank you for coming. Today we are testing [X]. Your feedback will help us make sure the system works well. We are going to get you to [explain main scenario/task], and observe you as you go.

With your permission, we're going to record the computer screen. During the session we will also track your eye movements with our eye tracking machine. The eye tracking will be used only to help us work out how to improve the site.

Once you’re finished you’ll be asked to complete a questionnaire and have a chance to ask questions or make comments about the experience.

Some Important Points

The test is intended to test the website, and not you.

First of all we are going to ask you to complete some tasks silently on the website, and then we will ask you about it afterwards.

Sometimes you will be asked for your name and address. At the end of the day, all data is deleted, so nothing will be kept.

If relevant: When you are asked to put in a credit card number, please use the number on this sheet.

Are you ok with being taped? Do you have any questions before we begin?


4



SCENARIOS

This first task you will complete silently (using RTA method).

Stages of RTA

1. User is calibrated for eye tracking.2. User is asked to complete a task on their own, without intervention and without being asked to voice their thoughts.3. Moderator/observer watches user eye traces in real time and notes any interesting gaze patterns.4. On completion of the task, the moderator asks the user to comment on the dynamic replay of their eye-movements. The user

is shown the video of their eye traces while the moderator can probe further on the underlying motivations of the user.

Task 1

Moderator to explain scenario 1 or give user sheet with description

[Explain scenario 1]

To fill in by moderator:

First click: _____________________________________

SUCCESS RATING SEVERITY RATING

1 - Easy (1st try) 1 - No problem

2 - Medium (2nd/3rd try) 2 - Minor usability problem: low priority fix

3 - Hard (4rd/5th try) 3 - Major usability problem: high priority fix

4 - Assist (Help from moderator) 4 - Showstopper: critical fix before release

5 - Fail (Failed or gave up)

Questions:

- How satisfied were you with your ability to achieve the task’s objective (circle one)

Strongly dissatisfied Dissatisfied Neutral/Don’t know Satisfied Extremely satisfied

1 2 3 4 5

- Take user through the screens visitedo Probe:

Clear? Likes/dislikes? Missing information? Improvements?


5



Task 2

Moderator to explain scenario 2 or get user to read off sheet

[Description Scenario 2]


First click: _____________________________________







Questions:



1 2 3 4 5




6



Task 3 Moderator to explain scenario 3 or get user to read off sheet

[Description scenario 3]


First click: _____________________________________







Questions:



1 2 3 4 5




7



Now, as you complete the remaining tasks, we would like you to think aloud as you go, so we can understand your thinking as you use the site. Please as you navigate let me know what you are thinking and why are doing things.

Task 4




First click: _____________________________________







Questions:



1 2 3 4 5




8



Task 5




First click: _____________________________________







Questions:



1 2 3 4 5




9



Task 6




First click: _____________________________________







Questions:



1 2 3 4 5




10



GENERAL QUESTIONS

Moderator to ask general questions summarising the usability testing session. Customise/add/remove questions where necessary.

What is your overall impression of the site/prototype you have seen today?

What was your favourite aspect of the website that you worked with today?

What was your least favourite aspect of the website that you worked with today?

Is there anything that you feel is missing on this site? (Probe: content or site features/functions)

Was [X] on this website fast enough for you?

Was [X] on the website easy enough for you?

Is there anything you would change on the website? What would be the first thing?

Do you have any other final comments or questions?


11



SATISFATION RATINGS

[Respondent to fill in the satisfaction rating questions.]

Please circle the number to rate the product you used today. Don’t think too long about each statement. If you don’t know how to respond, simply check box 3.

Strongly agree

2 3 4 Strongly disagree

1. I think that I would like to use this website frequently.

1 2 3 4 5

2. I found the website unnecessarily complex. 1 2 3 4 5

3. I thought the website was easy to use. 1 2 3 4 5

4. I think that I would need the support of a technical person to be able to use this website.

1 2 3 4 5

5. I found the various functions in the website were well integrated.

1 2 3 4 5

6. I thought there was too much inconsistency in this website.

1 2 3 4 5

7. I think that most people would learn to use this website very quickly.

1 2 3 4 5

8. I found the website very awkward to use. 1 2 3 4 5

9. I felt very confident using the website. 1 2 3 4 5

10. I needed to learn a lot of things before I could get going with this website.

1 2 3 4 5

Thank you very much for your participation. The website will be better thanks to your effort


12


Usability Testing Script

Documents

Transcript of Usability Testing Script