Usability Testing Moderator

DesignDojo

May 2016

http://www.youtube.com/watch?v=wVH--bLiVZ8

Founded in 2010 with offices across the globe

More than 1b words subtitled in 200+ languages

35m+ active users, 8m+ registered users

18,000+ hours of premium video

Quiz

Moderating in Usability Testing

What is Usability Testing?

Observation ParticipantThink Aloud

Set of tasks

Measuring:EfficiencyEffectivenessSatisfaction

ParticipantModerator

Roles of a moderator

● Make the participant feel comfortable.● Take the lead and set expectations.● Ask unbiased questions. ● Guide the data collection.

Small details matter

● Observe and read participant’s nonverbal behavior● Encourage participants to talk more by nodding with

tokens such as “uh huh”, “mm hm”.● Take notes but don’t write all the time● Let the participant speak.. Don’t rush or cut them off● Don’t give away information or provide answers

Small details matter

● Pace your talking speed to match participant’s.● It’s ok for awkward silence.● Pay attention to inconsistencies.● Reflect what you heard and check your

understanding by paraphrasing. ● Ask unbiased and open-ended questions.

Small details matter

● Be relaxed● Lean forward instead of backward● Sit at the same level as your participant, facing forward● Maintain eye contact

Types of activities1st impressions (What’s the first thought that comes to your mind? Can you tell me what does this page do for you?)

Exploratory (What would you expect to see if you click on this? What does this label mean? I noticed that you …., what’s on your mind right now?)

Directive tasks ( ....Find the working hours for the restaurant on a Sunday)

Types of activitiesWrapping up(On a scale of 1- 10, how would you rate the ease of use for this website? What should we do to make it 10 out of 10? Do you have any questions or feedback?)

Post-site questionnaire - System Usability Scale- Post-site word choice

Moderation Tips...“Can I sort the order of items here?” Set expectations

If you ask me questions, I may not answer you or I may be vague. If I’m doing that, I’m not unfriendly but just trying to stay neutral.

Turn a question aroundI’m not sure but tell me more about what you’re thinking here... What would you expect if... What do you think...

“.....” Prompt silence or unresponsive participantWhat are you trying to do right now? What is going through your mind right now?

Moderation Tips...“Oh no, I’m so sorry. I don’t think I’m good at this”

Provide reassurance There are no wrong answers here, you’re here to help us... This is just the kind of feedback we want to hear...

“.....hahaha. By the way, I also think that...” Redirect or cut feedback shortThis is all very helpful to see and hear. Just for the sake of time, I’m going to ask you to go to the next task/go back to...

End session earlyThat’s actually all I had for you today, so you’ll get some time back in your day. Thank you so much for your feedback!

Take on the role of a moderator20 mins per individual + 5 mins analysis

Checklist for pretest briefingBe a gracious host. Say “welcome” and introduce yourself.

Explain purpose and procedure of session (how long it will last, take breaks at any time)

Review informed consent

- Tell participant that the session will be recorded. It will only be used for internal analysis and will not be publicised.

- Participant’s name will not be mentioned with any data collected.- Participant can stop at any time.- Hand out consent form to participant

Explain your role (as an observer who will be taking notes and clarifies)

Explain participant’s role and provide assurance - Think aloud (give narration and describe steps)

Checklist for pretest briefingAsk participant if he/she has any questions before test commences

Ask background information of participant

The tasks

- Assure participant that it is the website you are testing and not him/her- Let participant know that you are not the designer - so it’s ok to be honest- Read each task aloud- Tell participant to inform you when he/she completes the task

Ask participant if he/she has any questions

Usability Tasks (Viki)

Background Questions (2 mins)- Occupation of participant- What does participant do in his/her free time? How often?- Ask question relating to the activity of product

Usability Tasks (Viki)

Tasks (13 mins)

1. You started watching the 1st episode of a new Korean drama called “Dramaworld” on TV last night and you actually like it. Find the 2nd episode of this drama called, “Dramaworld” on your Viki mobile app to continue watching it.

2. Your friend from Russia saw you watching it and would like to watch it too. However, she could only understand in Russian. Find out if the drama supports Russian language subtitle.

3. You enjoy watching the acting of one of the female casts named “Han Ji Min”. Start watching another drama that she acted in.

Usability Tasks (Zalora)

Background Questions (2 mins)- Occupation of participant- What does participant do in his/her free time? How often?- Ask question relating to the activity of product

Usability Tasks (Zalora)Tasks (13 mins)1. You are planning to purchase a Casio watch as a farewell gift for your female

colleague. She has a liking for golden watches. Without using Search feature, pick a Casio golden watch that is $100 and below on your Zalora mobile app.

2. You remember that Zalora provides a free delivery but only with a minimum amount of money spent on Zalora. Find out the minimum amount needed to be spent to obtain the free delivery.

3. You thought that this watch will be an ideal gift and you would like to ask your friends for their opinions tomorrow. Add this watch to your wishlist for now.

Reflection

Usability Testing books to pick up

Thank You!

@RVikiDesignon Twitter and Facebook

@borryshasian @choongchingteo @estherfanLL