10 cognitive biases to avoid in User Research (and how to ... · 4/22/2019 10 cognitive biases to...
Transcript of 10 cognitive biases to avoid in User Research (and how to ... · 4/22/2019 10 cognitive biases to...
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 1/13
10 cognitive biases to avoid in UserResearch (and how to avoid them)
Sundar Subramanian Follow
Jun 7, 2018 · 11 min read
Cognitive biases have become quite popular in mainstream culture in
the last decade, thanks to books like Thinking Fast and Slow and
Predictably Irrational. Along with human centered approaches, it has
also gained quite a lot of prominence in Experience/Business design.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 2/13
Since we have come to rely more and more on quantitative and
qualitative research to take informed product/business decisions, it’s
also important to ensure that the data and its method of collection is
not impaired by an ignorance of cognitive biases, so as to provide
meaningful value to end customers.
There are more than a 100 of them (188 according to this exhaustive
infographic), but for the context of User Research, I’m going to focus on
10 of them and give examples/anecdotes on why and how to avoid
them.
1. Framing effect
This is one that I have seen most repeatedly occurring in the context of
User Research and one which is also tricky to avoid if you don’t pay
close attention to your words and actions.
What is it?
Simply put, human beings don’t make choices in isolation. We arehighly dependent on the way it is presented to us. A simple example
would be this — A big meal on a small plate is more fulfilling than a
small meal on a big plate. The empathy map is designed to help
overcome this bias by including what we see and hear along with what
we do and say.
Example
When you are presenting prototypes or asking users about their
experiences while using a product, be careful about how you frame the
question.
A question such as “What did you like/dislike while using this product”
can cause the users to only focus on the positives/negatives of the
product (even for the rest of the interview duration) and it might lead
to false positive/negative insights.
A neutral, non-indicative way of questioning, such as “Can you describe
the last time you used it?” or “How do you feel when you use this
product?” can yield better-unbiased results.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 3/13
2. Confirmation Bias
This is a super-villain of biases. Extremely common and difficult to
rectify. But it is one of the biggest goof-ups that researchers can make,
and is often used by proponents of quantitative research to justify why
it is better to go with large data sets.
What is it?
We like data that confirms our existing hypothesis/beliefs anddiscard those which challenge them. This is an evolutionary safety
net that has been programmed into us, to protect our brains from the
threat of opposing information which challenges our identity (we
evolved as tribes with shared beliefs). That makes it very difficult to get
rid of it completely.
Example
A typical example would be a researcher/product manager asking the
participant — “Have you ever done/considered doing Action X through
App Y”, and the participant says, “Yes! All the time!”. Hooray! you have
hit a jackpot and your evidence is confirmed. Users love taking that
action through your app. But in reality that might be far away from the
truth.
When you get an affirmation/positive response for something,
check/recheck in several different ways. Why did the user take that
action? Was it because they did not have other options? Did they like
the process? How many times in the last week did they do it? Can they
show any evidence? Is there a possibility they might just be wanting to
please you?
The best way to avoid confirmation bias is to play devil’s advocate to
your thoughts and hypothesis consistently during the research process.
3. Hindsight Bias
Human beings are really bad at thinking in time. And hindsight bias
stands testimony to it.
What is it?
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 4/13
In order to create synchronicity and order to the ebb and flow of time,
we try and find reasons for the events that happened in our past,without having any factual evidence for it. It is often called the “I
knew it all along” phenomenon.
Example
When we conduct research, we often ask users to dig into their past to
find examples and anecdotal pieces of evidence. And often when we
dig deep into the why’s, we hear several reasons about how they faced
certain difficulties/took certain actions.
For e.g: A user was complaining about his business not running well
due to the oncoming of the internet and people buying things online
rather than coming to the stores. So when I posed the counter question
of why he was not getting into e-Commerce, I received a rather
surprising answer: “The websites are not taking good care of customers
and if there is any damage/problem with our product it reflects badly
on our name”. It was a clear indication that he was not really aware of
how e-Commerce works and the fact that customers had the option to
return damaged items and provide reviews to sellers.
Customers can never be blamed for making up such reasons, but it’s
really important for interviewers to be aware of them and constantly
double check the evidence to support their statements/anecdotes.
As a researcher, this also means that when we don’t have all the
answers to a certain aspect of what we were supposed to uncover, we
need to admit that instead of covering up with false reasons.
4. Social Desirability Bias
One word to encapsulate this is, ‘SELFIES’
What is it?
We are social animals, and this means that our actions and wordsare presented in a way that makes us look good amongst others,even though they might be inaccurate. This is so deep-rooted in our
behavior that we even disdainfully label those who don’t follow these
norms as anti-social.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 5/13
Example
This bias particularly comes up when you are researching on a topic
which has a social capital associated with (social media, online
platforms, matchmaking services etc.)
For e.g: A middle-aged social media user who wants to portray an
image of a leader among his/her family. He/She would never post or
say something that might hamper that image, and hence even if there is
a usability issue and the user is finding difficult to navigate through the
app, they might still not complain about it. A skilled researcher would
take the efforts to re frame the questions in a way as to boost their
social desirability (for e.g: If you could design this better for your
dad/mom, what would you do?). This could give clues as to issues they
really find annoying but never speak up about.
5. Sunk Cost Fallacy
Again one of the most common biases. Causes a lot of damage not only
in research but also in major life choices (marriage/debt etc.)
What is it?
Our decisions are tainted by the emotional investments weaccumulate, and the more we invest in something the harder itbecomes to abandon it. In other words, the deeper we get into the
maze the harder it becomes to come out (subtle Westworld reference).
The next time you end up drinking ‘just one drink more’ and get passed
out or keep calling with a bad hand in poker, you know which bias to
blame.
Example
As researchers, we invest a lot of time into conducting research and
collecting data. Over a period of time, this data can become a burden
rather than being helpful. Obsessing over the findings, we can easily
get lost and miss the bigger picture of what we really need to achieve
and deliver.
In order to avoid this bias, it is important to balance our efforts and
rewards. This means breaking down the research into smaller chunks
and having go/no-go decisions after each of those chunks. The Lean
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 6/13
Startup methodology is in a way aimed at reducing this bias, by forcing
entrepreneurs to run small experiments and objectively test the results,
rather than spending time and effort to eventually reach a futile
conclusion.
It needs a mindset to ultimately get acclimatized to and become at
peace with the fact that losses and failures are an inevitable part of life.
6. Serial-Position Effect
The U-shaped destiny of long lists.
What is it?
If your name starts with an M and your name is part of an
alphabetically ordered list, then sorry to say but your name likely won’t
be noticed.
This list illustrates the % of word recall compared to its position in a
sequence. Due to the way our memory gets constructed, we paymore attention to the earlier and later parts of long lists.
Example
In tasks such as card sorting, this bias can lead to users omitting or
ignoring the middle elements, which might hamper the effectiveness of
the activity/experiment.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 7/13
An interesting case of this bias affecting your studies is in continuous
field studies, where you go out on the field to interview more than 5–
10 users on a stretch before getting to synthesis. Where it gets tricky is
when you tend to overemphasize observations from the first and last
interviews more if you place them in the order (on sticky notes/word
docs).
A good way to get around is to break them down into smaller chunks or
randomize the arrangement a few times to nullify the bias.
This bias can also creep into things such as to-do-lists and feature lists,
especially for those who are in charge of getting many things done at
the same time. Categorization is a highly effective strategy to counter
this.
7. The illusion of transparency
A dangerous bias that leads to misinterpretation and
miscommunication
What is it?
If you have ever played charades, you already know this bias. We tendto overestimate the extent to which others know what we arethinking/trying to convey.
Most of our subjective experience is not observable, and we tend to
overestimate how much we telegraph our inner thoughts and emotions.
Example
In interviews, many participants try to convey their emotions through
body language, pauses, and other non-verbal cues. The illusion of
transparency makes it difficult for them to really know whether the
message is being conveyed rightly.
Which means that we need a different mechanism to figure out
whether we are missing out on some of these aspects which might not
be caught in the interview. This is why providing affirmative feedback
is important. It can be as simple as, “So from what you said I feel like
you are feeling this way about this feature, pardon me if I’m wrong.”
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 8/13
Often you would be surprised to know that whatever they said in the
first place was interpreted by you in a completely different manner.
8. Clustering Bias
Can lead to a lot of false positives and false negatives in cognition
What is it?
Clustering bias is what happens when we describe someone as having a
‘hot streak’, such as a night of poker or a football tournament where
everything seems to go right for the player. But in a short streak ofrandom events, a wide variety of probabilities are expected,including some streaks that seem highly improbable.
Example
As researchers, finding patterns in data is bread and butter for us. But a
drawback of qualitative analysis is that with such a short sample size, it
is often impossible to avoid seeing patterns that might be just smaller
sets of randomness that appear to have a commonality. An effective tool
to counter it is to triangulate patterns, and to match data-driven
insights based on large sample sizes with the deeper insights found in
qualitative research.
It can be also reduced by conducting research and prototyping with
completely different and diverse sets of users.
Another way of avoiding it is to have silent brainstorming before
discussing patterns and include a set of diverse stakeholders in the
analysis process so that the bias has a better chance of getting
canceled/evened out.
9. Implicit Bias
Again a really tricky and dangerous one. Often termed as stereotyping
in popular culture.
What is it?
These are our attitudes and stereotypes we associate to peoplewithout our conscious knowledge. Implicit Bias is really difficult to
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 9/13
root out since it has been embedded into our consciousness from a very
young age through media, people around us and popular culture.
Example
One of the most prominent examples of implicit bias is that of police
officers associating black people with crimes without realizing they are
doing it.
In the context of user research, it can happen when we talk to people
from certain demographic, racial or ethnic groups of whom we already
have preconceived notions and generalizations about. It can lead us to
behave in certain ways which might not be totally necessary (such as
being overly polite to disabled people when they would rather be
treated like a normal person). A good practice to avoid this would be to
write down all the preconceived notions about the person before going
into the interview, and knowing as little as possible about them before
speaking to them.
It’s always important to always remember that as a good researcher,
our duty is not to be socially appealing or to become friends with our
users but to really understand what is going on inside their mind and
how they think, even if it that means there needs to be certain
uncomfortable awkward silences or small disagreements.
10. Fundamental Attribution Error
A favorite among designers and usability enthusiasts, it is what
happens when people blame themselves for not being able to
understand technology
What is it?
It is the tendency of people to overemphasize personalcharacteristics and ignore situational factors in judging others’behavior. Because of the fundamental attribution error, we tend to
believe that others do bad things because they are bad people. We’re
inclined to ignore situational factors that might have played a role.
Example
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 10/13
When you conduct usability tests and you hear a user talking about
making a ‘mistake’ doing a certain task, pay attention! That might be
the biggest clue towards creating a better product/experience.
Really good products make the user think less and get more done.
Think of any app/website where you blame yourself for not being able
to do a certain thing right. There is an opportunity for improvement.
A good way to avoid this bias is to also to complement interviews with
observations/heat maps. They provide a better account of how
researchers use the product and the kind of errors they commit due to
poor design. It’s common among engineers to also push the blame on to
users and guide them on the correct usage. But as researchers, we need
to fight this notion. Products and their learning curves need to be
designed in a way that even a beginner can pick them up with a few
tries and not feel guilty about failing.
In summary, these are the 10 biases you can take away to improve your
research practice
Framing effect — Framing of inquiries can influence responses
Confirmation Bias —Humans tend to only look for evidence
confirming their hypothesis
Hindsight Bias —Humans always find reasons for their actions in
the past
Social Desirability Bias — Humans tend to speak in a way that
makes them look good
Sunk Cost Fallacy — Humans tend to stick on longer to their
losses than they should
Serial-Position Effect —Humans tend to value items at the
end/beginning of lists more
Illusion of transparency — Humans tend to overestimate the
extent to which others know what they are thinking
Clustering Bias — Humans tend to find patterns amidst
randomness, when there are really none
1.
2.
3.
4.
5.
6.
7.
8.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 11/13
Implicit Bias — Humans have implicit associations about certain
groups and their behavior
Fundamental Attribution Error —Humans tend to attribute
errors to internal characteristics even when it is situational/caused
by an external forces
9.
10.
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 12/13
4/22/2019 10 cognitive biases to avoid in User Research (and how to avoid them)
https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6 13/13