THE TEETHING TROUBLES OF SURVEY RESEARCH

In today’s business reality, decisions cannot be based on random,

uncontrollable factors such as luck. In this fast-moving environment the chance

to fail is greater than ever. Figures reported by the Doblin Group show that

96% of all new product introductions and innovations fail to return

their cost of capital (Marsh, 2012). The market space requires brands to

validate their communication and advertising efforts before an actual market

launch. Organisations are tracking more and more consumer perceptions on all

valuable touch-points. Businesses and marketers are striving for a more

data-driven decision-making process. We need those hard-core numbers

to help us select the ideas to take forward and the ones to leave behind. This

need for fact-based decision-making is the reason why survey research

remains a very powerful and commonly used research method.

The 2012 global research report by ESOMAR shows that no less than 76% of

all market research projects conducted worldwide are in the field of quantitative

research. The power of survey research lies in its validation strength,

surveys helps us provide those (no) go decisions on a brand, product

or strategy level. Yet, considering the stimulating innovations that have

moved the research industry in the past years, surveys research has been

lagging behind on several important aspects.

What to

expect?

Online surveys are part of an engaging

environment - everything online is fun, gamified

and interactive. Nonetheless, filling in an online

survey is not that much an engaging activity

and we need to realise that participants are

always just one click away from exiting

our surveys. Besides the high drop-out rates,

this lack of engagement has an impact on the

data quality - researchers are confronted more

and more with straight-lining behaviour and the

level of detail in participants’ answers is

dwindling. This low(er) engagement is at the

foundation of what we could call “the global

warming of panels”. It is becoming

increasingly harder to attract people to

participate in research. Especially

considering some important target groups such

as youngsters and Millennials who grew up in

this fast-pacing environment.

1 From boredom to engagement

These people are considered to be stimuli junkies

and will stop any activity that does not succeed in

crossing the minimum expected engagement

level.

Some research players have turned to

gamification as the Holy Grail to enhance

participant engagement (Puleston & Sleep,

2011), while others have redefined the research

landscape by introducing fresh survey formats

(e.g. Google consumer surveys) or by focusing

on behavioural data. Yet, is this enough and

what is the ultimate recipe for re-engaging

people to participate in survey research?

Survey research insufficiently copes with the complex reality of consumer behaviour. Decisions are

influenced by a number of dynamics and it is important that surveys mirror these different aspects

in order to provide valuable input for clients and their business needs.

2 From quarantine to consumer context

Consumers are social animals

Consumers are social animals and our decisions are coloured by group thinking or herd behaviour

(Earls, 2009). The majority of consumer decisions are taken in a social setting (both conscious and

unconsciously). We tend to copy the behaviour of people around us rather than to use our

own (maybe more rational) information to guide our decisions. Nevertheless, we do not

take into account this social dimension in survey research. We keep on conducting surveys in an

individualistic setting, where participants are asked to answer one question after another without

being able to connect and reflect with other participants.

Consumers are bad witnesses of their own behaviour

Survey research traditionally taps into the so-called

“system 2 thinking” of our brain. Nonetheless, the

whole thinking behind behavioural economics and

the work of Daniel Kahneman (2011) show that our

decisions are mainly taken quickly, automatically by

the so-called “system 1” side of the brain. We are

not rational thinkers and we use heuristics for

our decision-making. One of these heuristics is

“emotions”, our decisions are wired by

emotions. We thus need to tap more into implicit

attitudes and procedural knowledge in our surveys.

The majority of consumer decisions are taken in a certain context or occasion. It is important to

grasp the contextual background consumers are in when making certain decisions (e.g.

consuming or buying a product). An answer to a question might not be as simple as yes or no, but

might be explained by ‘yes when’ or ‘yes because’. Context is a better predictor of consumer

behaviour than individual characteristics (Spruit, 2012). We need to get a better

understanding of the variations in consumer behaviour depending on the consumer situation or

context.

The majority of consumer decisions are taken in a certain context or occasion

Are we losing game to other research methods?

Considering the importance of data for the decision-

making process, the need for accuracy and

projectiveness is present more than ever. Can survey

research reclaim its position to provide consistent and rich

data for decision-making by capturing the complex

consumer reality, while at the same time increasing the

engagement level?

A PARADIGM SHIFT

Taking into account the above mentioned truths, we have designed a new survey

approach. A two-paradigm shift rests at the base of this new survey thinking.

Why is it that we, researchers, fail to create an

engaging experience for participants? In

order to get a better understanding of

participant engagement, we need to

capture the concept of motivation. The

self-determination (SD) theory, initially

developed by Edward L. Deci and Richard M.

Ryan (2000), explains that motivation, or in this

case participant motivation, is a continuum and

not a fixed state. It relates to a task within a

context. This continuum (see Figure 1) has two

anchoring points, a-motivation on the one

hand and intrinsic motivation on the

other. In between lay a number of levels of

extrinsic motivation. Naturally, intrinsic

motivation is the ideal state.

1 The empowered consumer

This is where participants are mainly driven by

internal factors and are therefore more likely to

sustain the activity. Yet, the motivation level to

participate in surveys is generally located in

the middle of this continuum, where

participants are mainly driven by external

factor such as incentives. Furthermore, the self-

determination theory describes how to move

people’s motivation along the continuum. To do so,

you need to foster feelings of autonomy,

competence, relatedness and value.

AMOTIVATION

INTRINSIC

MOTIVATIONEXTRINSIC

MOTIVATION

Task is not done properly

Interest, enjoyment

highly competent

Figure 1 – Motivation continuum (SD Theory)

AUTONOMY

Autonomy refers to the fact that you allow

participants to choose to do something or

not. Traditionally, when filling in a survey, you

enter this tunnel experience - getting one question

after the other, screen after screen. In order to

avoid this, we can develop a modular survey

approach dividing the survey into different

modules or building blocks. These modules can

contain both question types and tools. In this

modular survey, participants can choose which

building block to start the survey with. This

modularity principle in its pure form may not be

realisable for all survey set-ups. Some research

projects may need direction and a strict flow in

questionnaire design; for these set-ups we can

use the idea of ‘perceived non-linearity’. When

participants complete a building block, by filling in

all the questions within that block, they will unlock

a new module.

Combining this gamified element with a nice

modular survey layout, where every building

block is a visual element, enables participants

to break out of the traditional survey tunnel.

Even in the best case, where we put our efforts in

creating an engaging experience or where we invest

in gamification techniques, participants are only

asked to answer our questions without being

given the option to share extra feedback. Yet

consumers are valuable brand consultants and we

should start leveraging this, even in survey

research. This can be done rather easily by giving

participants the possibility, after their initial survey

participation, to enter a second survey dimension

where they can truly collaborate with the brand and

researcher.

Next, in traditional survey research we do not

allow participants who are topic, brand- or

experience-engaged to collaborate with

brands or researchers beyond their survey

participation. Some participants would like to

enter an additional collaboration dimension

and help out brands, but we do not allow them

to do so. A study conducted by InSites

Consulting at the end of 2013 showed that

44% of consumers would like to

collaborate with brands. Yet, the traditional

survey format does not allow participants who

have something to say to share their advice

with brands or researchers. We basically let

participants go after their survey

participation, without truly leveraging

their full potential. Why is it that we do not

allow engaged participants to go further in

their collaboration with the brand?

COMPETENCE

The feeling of competence, which is about

showing participants that they are good at

something, can be reached by empowering

consumers to do more than simply

answering a researcher’s questions. We

should give participants a role beyond responding,

by involving them in tasks that are normally on a

researcher’s repertoire. An example here is the

use of crowd interpretation (Verhaeghe et al,

2011), where we show the live survey results to

participants, for example, and ask them to

interpret them by using their own background and

knowledge as a reflection point, in order to get a

greater understanding of the research results.

This is where participants engage in a co-

researcher role. Research conducted by Balcetis

and Dunning (2011) revealed that we, as

individuals, fail to take into account the

influence of the situation when predicting our

own behaviour. By contrast, when predicting the

behaviour of others, we correctly take into account

the influence of these circumstances. These

findings show that by involving consumers in

an interpretive role, we might gain greater

understanding.

RELATEDNESS

The third dimension stipulated by the self-

determination theory is relatedness, which is about

showing participants that other people just

like them are doing this. Filling in a survey is a

rather a-social and lonely activity; as a participant

you might even wonder whether you are alone in

this. Theories such as Herd by Mark Earls have

highlighted the importance of recognising a social

dimension in marketing and accordingly marketing

research. A first step would be to make

participants aware of the fact that they are

not the only ones filling in the survey. This

could be done by real-time visualisation of the

number of people participating in the survey. The

true feeling of relatedness is fostered when

allowing participants to connect and reflect

with other participants on the research topic.

Important here would be to allow participants to

introduce themselves to one another. The latter

might function as a conversation starter and will

show participants that people just like them are

contributing to the research project as well.

VALUE

The last feeling we should foster as researchers in

order to increase the engagement level is ‘value’.

Namely, showing participants that what they

do has a meaning. Traditionally in survey

research we do not share with participants who

the research is for, in order to avoid any potential

bias. If we want participants to become an

active empowered partner in research, we

can benefit from introducing the brand

behind the research project and by even

sharing the objectives of the research.

Researchers and brands have to acknowledge

that participants can provide valuable feedback to

help a brand, yet we often do not allow them to

share any with us. Being open and transparent

towards participants shows them that their

contribution matters. By uncovering the real

research objective participants get a clear idea of

the value of their contribution.

Today, consumers expect to go beyond simply

‘responding’. Yet the foundation of survey research,

like Pete Comley (2006) describes, is a parent-

child relationship between researcher and

participant. The sole role of participants is to

respond to a researcher’s questions, without

allowing them to ask questions in return.

Nonetheless, researchers can benefit greatly from

a partner relationship with participants. It is

therefore time that we allow participants to play a

more active role in research and become an

active empowered partner. This empowerment

starts with creating an engaging survey experience

for participants by fostering feelings of autonomy,

competence, relatedness and value.

2 Moving out of that box

Next to the importance of boosting

participant engagement, we are feeling

the need to capture a more contextual

understanding in survey research. We

researchers currently fail to mirror the

complex consumer reality in our surveys.

We need to redefine our current survey

thinking in order to capture the different

dynamics that influence consumer

behaviour. The second paradigm shift is a

result of inside- the-box thinking. The sole

role of participants in survey research is to

respond to a researcher’s questions. Traditional

survey research is thus mainly about asking

questions in an individual setting. Yet, in order

to grasp the consumer reality, we might need to

go beyond asking questions and to think out of

the box. In research we can identify three

supplementary collaboration modes

between researcher, brands and

consumers: listening, doing and co-

creating.

LISTENING

The researcher openly listens to and observes

the consumer. When looking into the research

landscape, we can detect plenty of approaches

tapping into observing spontaneous consumer

behaviour. This can involve the direct observation

of consumers in their natural setting through

(online) ethnography but also listening to

what consumers spontaneously share on social

media (e.g. social media netnography or

monitoring). We also observe an increasing

importance towards integrating behavioural data

(e.g. purchase data, online clicking behaviour etc.)

into research. Also within survey research,

examples have popped up where data was enriched

with the social graph of research participants

(Rodenburgh, 2012) or even physiological data

coming from for example eye tracking or ERP

research.

But listening in surveys has moved beyond

capturing behavioural data. It involves every

attempt where the research participant has the

chance to speak up, to share whatever is on his or

her mind on a certain topic.

It is about giving people the chance to give

answers without asking questions. It is about

taking a bottom-up approach where the

consumer determines the rule of

conducting research. This can be done, for

example, by integrating an exit forum at the

end of the survey where consumers can share

whatever is on their minds, but it could also

involve approaches where consumers have

the chance to formulate questions for other

research participants (Schillewaert, 2009).

The benefits of embracing this mode of ‘just

listening’ are twofold: on the one hand it leads

to more objective fact-based information

giving us insights into what consumers

DO and not only say.

On the other hand, it helps us uncover new insights

into consumer behaviour that we were not

aware of before, simply because we did not think of

asking any questions about it.

DOING

The researcher involves participants in

different task-based exercises, where they are

asked to undertake a certain activity. There are

generally two kinds of approach within this

collaboration mode. To start with, at times

consumers cannot express themselves because

they are simply not aware of their own behaviour.

We are bad witnesses of our own behaviour. This is

in line with the whole thinking around behavioural

economics, which expresses that the majority of

what we do is done implicitly. So why ask people

about it? We need to use new and creative ways

within survey research that allow us to capture

this ‘system 1’ thinking. One way to do so is by

using time-pressure exercises. As people do not get

the time to reflect on their answers, we get a glance

of automatic behaviour leading to new information

(Verhaeghe et al, 2012). A second reason why

asking consumers to do things can be more powerful

than asking them to explain things has to do with

how our brain stores information.

In cognitive psychology (Schneider and Schiffrin,

1977) we make a distinction between declarative

(knowledge about something) and procedural

knowledge (knowing how to do something).

Neuroscience has uncovered that both types of

knowledge are stored in different part of our brains

and hence require different techniques to trigger

recall. However in survey research we do not adapt

our approach based on what we need to identify.

We mainly tap into the declarative brain. Research

shows that procedural knowledge can be

explored by allowing people to do things. This

can be reached by embedding more scenario

thinking and storytelling in research but also by

allowing participants to explain something through

visual cues. There already exists a need in survey

research to move more towards task-based

research. Or better: it is not what you ask

people, it is what you do with them.

CO-CREATING

This is the ultimate collaboration dimension, where

participants take up a very active role in

helping a brand beyond just giving their

opinion. Here consumers have the opportunity to

think along with brands and be involved in tasks that

traditionally are considered as being beyond their

capability. Examples of co-creating tasks are

involving consumers to shape the solution

space by including them in idea brainstorms

and asking them for their advice based on

survey results. Qualitative research has already a

long tradition of co-creation but so far this mode has

not entered survey research yet. The benefit of

empowering consumers to explore the solution

space together with us is twofold: it is a known fact

that innovations or campaigns that are co-created

with consumers have a bigger impact (Schreier et al,

2012).

Moreover a great idea can sometimes come from

somebody who enters the discussion with a fresh

unbiased perspective. But there is more: by

looking at the actions consumers propose

based on the research results, we can learn a

great deal also about what they are looking

for.

These collaboration modes can be plotted against

a second dimension representing the inter-

consumer relations or interactions. Theories such

as Herd make us realise that we are more

socially determined than we think we are.

However, today survey participants function in

isolation, not being allowed to reflect and connect

with other participants. We need to move away

from solely looking at the individual

respondent and to recognise the value of

inter-consumer reactions. This is where our

second dimension comes in, a continuum going

from individual to connected in 3 phases: me,

crowd and group.

Me: no real inter-consumer interaction is present

and the focus here lies on the individual participant.

Crowd: there is an interaction between

participants, yet no real closed feeling of

belongingness. We have a group of individuals who

are not yet a team working together towards a

common goal.

Group: close interaction between participants

who share a feeling of belonging to the same

community.

By combining both dimensions we can identify

a framework with twelve quadrants (see

Figure 2). Traditional survey research

primarily focuses on one single cross-point in

this framework, namely individual and asking.

Yet, survey research can greatly benefit

from going beyond this single-box

thinking. This does not imply that we should

completely let go of asking questions to

participants, this will still remain the core of

quantitative survey research. However,

combining the different collaboration

modes will allow us to better uncover the

underlying dimensions of the research

topic. Some aspects require activities beyond

asking questions in order to expose them.

Moving beyond the border of asking should

thus allow us to better capture the complex

consumer reality.

Figure 2. Research collaboration framework

EBAY CASE STUDY

eBay UK wanted to get a better understanding of why consumers lapse (or decrease in buying

frequency) by exploring the perceived site experience and its drivers and barriers. Besides

optimising the current eBay proposition, eBay also wanted to explore a new solution space for online

shoppers. The research ran in the UK amon different types of lapsers within the fashion and consumer

electronic product categories. A total of 834 consumers took part in the research. The scope of this

research was threefold (see Figure 3) :

1 Project background

Describe the behaviour and attitudes related to lapsing and pinpoint

areas of improvement for eBay

Generate insights into key success factors for eBay to manage

lapsing

Explore the solution space for lapsing, with a focus on inspiration

and findability

1.

2.

3.

Figure 3. Research flow

The research method used explored the boundaries

of survey research. In order to assess the impact of

this new approach we split-ran the survey. Some

participants got a traditional survey whereas others

got the enriched version containing the modular

approach and the village dimension. The research

approach is based on our new survey thinking

where we go beyond asking questions and

apply the principles of the self-determination

theory to better engage participants. The main

survey consisted of a modular survey design, in

which the survey was composed of different building

blocks or modules. The self-determination theory

claims that autonomy is a key driver of engagement.

Therefore, we introduced the concept of perceived

non-linearity by combining the modular survey

approach with a gamified unlocking element.

2 Project methodology

After the main survey, participants were invited

to enter “The Village”, a second optional

survey dimension which allowed engaged

participants to go further in their

collaboration with the eBay brand. After

participants filled in the survey, they could thus opt

in for this optional part where they could connect

with other participants and reflect on the research

topic together. The Village is a platform

consisting of different buildings, each of which

contains a different task-based element.

This study thus went beyond the traditional

single-box thinking of individual and asking.

The different tools in and after (The Village)

the survey can be plotted on our framework

(see Figure 4). The survey still consisted of

various research questions, yet on the

individual dimension we also introduced

some task-based exercises. Next to that,

the introduction of The Village and the

contagious tool allowed us to involve

the crowd through the social dimension

embedded in these tools. More detailed

information on each of these tools is

available in the next section.Figure 4. eBay project framework

Next to the traditional questions, we

introduced some new tools in the survey:

3 Project approach

IMPLICIT MEASUREMENT TOOL

The eBay brand image was measured through implicit attitude testing,

allowing to recognise natural and potential brand associations.

Participants got to see each statement for three seconds and were asked

to press the spacebar if they felt the statement could be attributed to eBay.

The tool is located in the doing and individual cross-point of the

framework. This Implicit Measurement exercise allows plotting all

statements on two dimensions: (1) the percentage of participants

linking the statement to eBay and (2) the time before pressing the

spacebar, resulting in four quadrants (see Figure 5):

Figure 5. Implicit Measurement quadrant

Natural associations: These are

spontaneous associations - the majority of

participants link the item with the brand with an

above average reaction time.

Potential associations: These items are

highly associated with the brand, however they

require some reflection (response time is below

average). These items trigger some “ahaa”

feeling of recognition, however they could

become more natural when investing in additional

communication efforts.

Niche associations: These items are only linked with the brand by a few participants, yet the reaction

time is above average. These are mainly items that are recognised by a specific target group (e.g. heavy users).

Limits: Few participants link the item to the brand and the reaction time is below average. If an

important brand item is contained in this quadrant, the brand needs to invest the required resources to turn the item

in question into a potential or natural association.

MY SHOPPING TRIP

In order to get an understanding of eBay’s role (and

that of its competitors) in the purchase process, we

introduced a task-based question in which

participants could reconstruct their purchase

process visually. Participants got an empty timeline

representing their purchase process from start to

finish. They could fill the timeline by dragging icons of

sources (information channels or platforms) onto the

visual and by explaining for each source how they

used it. Furthermore, we asked them to rate each

source on the extent to which it facilitated them

and contributed to their final purchase

decision. The outcome of this task-based exercise

allowed us to grasp the perceived positioning of eBay

in the purchase process and the use in each stage.

This gave us the potential to identify and quantify

differences in the search process across

subgroups.

CONTAGIOUS TOOL

In order to understand the lapsing behaviour, an

important element was to explore the drivers and

barriers for buying on eBay. These were

measured by means of a new tool which allows

highlighting a so-called ‘contagious effect’. In a first

phase, participants were asked to write down

what they considered to be ‘reasons for buying

on eBay’. They could write down spontaneous

answers, using key words, in five open-answer boxes.

In a second phase we showed them a word cloud

including all answers from all respondents for

this question up until that point in the survey. The

word cloud combined their own answers with those

given by the other participants. In a second phase, the

participants could click on those words in the

word cloud that they felt were also ‘reasons for

buying on eBay’.

This exercise allows identifying both

spontaneous and prompted drivers for buying on

eBay. In addition, the word cloud allowed

participants to see that other participants

also contributed to the research; this semi-

social dimension thus taps into the relatedness

aspect as specified by the self-determination

theory.

THE VILLAGE

After participants had filled in the survey, they could opt in for The Village where they could

connect with other participants and further collaborate with the eBay brand. The eBay

Village consisted of five buildings: the Lounge, the Ideation, the Inspiration wall, the Newspaper

stand and the Gallery (see Figure 6).

Figure 6. The Village

We did not simply introduce eBay as the brand behind

the research project, we also openly shared the

objectives of the research. This is the only way to truly

encourage consumers in providing valuable feedback

to help a brand. Apart from these featured topics,

participants could create their own posts related

to the research topic, which allowed them to

discuss and interact with other participants. This

open social space helped to gain additional insights as

it provided us with answers to questions we did not

even ask.

The Lounge (1) is the central building of the village where participants can connect with one

another, start a discussion on topics created by the researcher and even post topics of their own. In other

words: participants can connect with one another, the researcher and the eBay brand. In the Lounge we

introduced three featured topics where participants could introduce themselves, give feedback on the survey

and share their advice with the eBay brand. The topic in which participants were asked to introduce

themselves functioned as a conversation starter and showed participants that people just like them were

also contributing to the research project. This introduction topic enhances the feeling of being visible

as a consumer and thus taps into the relatedness aspect as defined by the self-determination theory. In

another topic, participants were asked to share their advice with eBay.

Figure 7. The Lounge

In the “Inspiration wall” building (2)

participants were asked to share inspirational

examples they believed eBay could learn

from. They could do so by uploading images,

YouTube videos and links to interesting

webpages. Participants were motivated to look

for example in- and outside the online shopping

platform environment. A real task-based

element which taps into the co-creation

dimension of the collaboration framework,

where participants were involved in generating

inspirational output eBay could learn from. In

this exercise we emphasise the value

aspect, as explained by the self-determination

theory, by openly sharing the objective for eBay.

The result is their own inspirational board

with explanations on how eBay should

implement this in order to optimise its

current offering. In the Gallery (5)

building, participants could view the work of

others, Like it and comment on it.

Figure 8. Inspiration Wall

In the Ideation building (3) participants

were asked to brainstorm and share ideas on

three topics related to inspiration and

findability, two areas in which eBay wanted to

optimise their current offering. Besides posting

their own ideas, participants could see what other

people posted and Like it or comment on it. This

idea sharing allows involving participants in

discovering the solution space. This task-based

element taps into the co-creation

collaboration dimension, where participants

are asked to think along with brands and tap into

the solutions space. The output of this exercise is

the creation of ‘idea cards’, which combine a

consumer idea with an inspirational visual that

can be used in future workshops or strategic

meetings. In the Newspaper stand (4)

participants were asked to write a critical

review of their shopping experience. For this

challenge, we asked participants to think of a

fashion/ consumer electronics item that they

would like to buy in the very near future.

Positioning this as a real task-based challenge,

participants had to pretend they were going to buy

that item, so we invited them to visit eBay and other

platforms and to write a review about their experience.

The strength of this exercise is that we are not

focusing on recall, which is visible in the added value

from positioning this as an in-the-moment task-based

challenge. In the Gallery (5) participants could

view the reviews of others, Like them and

comment on them.

Figure 9. The Gallery

4 Research findings

IMPACT ON PARTICIPANT ENGAGEMENT

Introducing a modular survey approach showed to

have a positive impact on the participants’

satisfaction level. The use of a second optional

survey dimension, The Village, also contributed to

a positive increase in satisfaction level. Statistics

from SSI, our sampling partner for this project,

showed that the extent to which participants were

extremely satisfied increased with 30% in

comparison with the benchmark condition. In

addition, the interest to participate again in this

type of research projects doubled, compared with

the benchmark setting. Overall, participant

experience was very positive: “Definitely loved

the new take on surveys you guys used. And I

very much prefer this method.” Nevertheless, the

major positive impact of this new approach was

visible in the contribution level throughout the

different task-based elements in The Village.

The Village was optional and only

available after participants completed a

15-20-minute survey. We know from previous

research that the maximum attention span is

about 20 minutes, yet 10% of the participants

actively participated in these additional

contextual tasks. Taking into account that these

contextual tasks are used to get additional

sensing and understanding, such proportion is

well suited for this purpose. Additional

experiments showed that this proportion varies

depending on the research topic, the active

participation rate ranging from 10% to 25%.

Overall, all active participants are

characterised by a high topic and brand

identification level and they are mainly driven

by intrinsic motives to participate in research.

The output of the task-based elements showed that

participants went beyond the expected in their

involvement. In the Newspaper stand, for example,

we noticed that the level of detail in their stories

was significantly higher compared to what we

receive in traditional surveys. This can be explained

by the increased engagement level entailed by

the task-based character of the challenge and

the social visibility the platform creates.

Overall, participants in the new-style survey

reported a significant 15% increase in ‘feeling

understood’ in comparison with the benchmark

condition. Next, participants felt that eBay was

more open to their suggestions and feedback,

compared to the results of the traditional

survey approach. This shows that survey

research can have an impact on the consumer

brand perception. Considering that surveys are yet

another touch point between consumers and

brands makes this another interesting finding.

The enriched set-up used a modular design.

As the questionnaire design required a fixed order,

we used the concept of perceived non-linearity.

The order of the different questions was the same

for both the traditional and the enriched survey

design. The enriched survey only differed through

its design and gamified unlocking element. In a

first dimension we needed to understand

whether the latter would lead to different

results. Comparing the data from the traditional

survey approach with those of the modular design

showed that the new survey design did not lead to

significant deviations in the data. These findings

highlight that a modular approach, combined with

a nice design, could be a first step towards

increasing the participant motivation level as a

result of an increase in perceived autonomy. This

does not mean that from now on we can ask

hundreds of questions in a single survey as long

as we introduce the modularity concept.

IMPACT FOR THE RESEARCHER

Researchers and research users should still take into

account the natural limit of acceptable interview length

for participants. Next, the output of the contextual

tasks allowed us to form more tangible

recommendations. In a first analysis phase we

analysed all questions included in both the traditional

and enriched survey designs (excluding any new tools

and task-based challenges). This formed the basis of

our traditional research report and conclusions. The

second analysis phase included all enriched elements

and tools. Comparing both reports taught us that not

only we had more data, but more importantly that the

data was also richer. The contextual output from the

new tools and challenges, composed of consumer

visuals, stories and ideas, allowed us to bring more

sensing and understanding into the research

results. In addition, the involvement of consumers in

shaping the consumer space and the possibility to

share their advice and feedback, allowed us to shape

very tangible recommendations for improvement.

The impact for eBay was threefold: gaining

contextual understanding, the ability to uncover new

insights and achieving more actionable research

output.

IMPACT FOR EBAY

A first key benefit was the addition of contextual

understanding to the survey data. Survey research

helps to answer the predefined research questions and

objectives; however these answers often lead to

additional questions. There is a need to combine the

survey output with some additional sensing and

understanding. The task-based elements in The

Village allowed eBay to grasp the contextual space

consumers are in, leading to a better understanding of

the ‘why’ behind the survey data. The ‘my shopping trip’

exercise in the survey allowed to identify and quantify

the role of eBay and its competitors in the purchase

process across target groups. The output of this

exercise showed where eBay enters the flow, what

its role is and how it was evaluates in terms of

facilitating power for the final purchase decision.

1.

In The Village we challenged participants to

pretend entering a new purchase mission,

by visiting eBay and other e-commerce

platforms and to write a review about the

experience. These review stories allowed us to

get a detailed understanding of the perception

and performance of the different platforms. Yet

the real value came to life when combining the

consumer stories and visual output of the

review exercise with the quantification of the

‘my shopping trip’ exercise. Not only did we get

clear view of eBay’s role in the purchase

process for the different subgroups, we

also got a clear sensing and understanding of

how eBay could improve its role. Additionally,

the visual output from these contextual tasks

(user-generated pictures) proved to be stronger

at conveying a message.

The survey highlighted that eBay was lagging

behind on some important buyer

perception items in comparison with its

competitors. The output from the Inspiration

wall, where participants were asked to upload

inspirational pictures and videos that eBay could

learn from, visualised the same critical aspects

as mentioned in the survey. Yet, the visual

storyline from this exercise helped convey the

message in a much more impactful way. Images

are very powerful to convey a similar message

because of their emotional load.

Another key benefit resulting from this new

approach was the ability to discover new

things, applying new tools that went beyond

the single-box thinking, thus allowing to

uncover new insights. The Implicit

Measurement approach allowed for eBay to

distinguish which category items evoke

spontaneous and potential brand associations.

The tool allowed eBay to identify the items that

currently form a barrier and need closer

attention. Apart from measuring the associations

towards the eBay brand, we repeated the same

exercise for eBay’s main competitors. This

approach allowed us to define a new performance

indicator of brand strength: brands for which the

items were located more towards the right upper-

side of the framework have a clearer brand image

than those with items located more towards to

lower bottom-side. This allowed eBay to get an

understanding of its brand strength relative to its

competitors.

Next, the social dimension in our survey set-

up allowed for eBay to capture the

contagious effect arguments might trigger.

This is an important aspect to understand, as

this effect can have an influence on consumer

perception and reflects the actual consumer

reality. The word cloud exercise, in which

participants could indicate other words in a

prompted setting, allowed uncovering this effect.

2% spontaneously indicated PayPal as a driver

for buying on eBay, yet the contagious effect

showed that this increases to 52% in a prompted

setting. Using PayPal in a communication

campaign might thus trigger understanding and

recognition amongst people. This contagious

effect is important for eBay to recognise

and to take into account for future

communication efforts.

2.

A third and last interesting benefit is that the

co-creation tasks led to more actionable

research output. Some of the task-based

challenges allowed participants to help shape

recommendation for eBay. In the ideation tool,

participants were asked to brainstorm on

potential ideas related to inspiration and

findability. The outcome of this exercise was

50 idea discussions, which allowed us to

create idea cards for eBay. These cards,

formulating tangible instructions for

improvement, could then be used in future

workshops or strategic meetings. Next, the

output from the Lounge, where

participants could share their advice

with eBay, allowed to shape very tangible

recommendations based on consumer

feedback.

3.

Is it ‘game over’ for surveys? Reflecting on

the approach and learnings described

above, we can conclude that the answer is

definitely “no”. Yet, we need to realise that

it is high time to walk away from our

traditional survey thinking.

5 To conclude

We need to invest in a survey approach

which empowers consumers. Consumers

expect to go beyond responding, so why not allow

them to do so? We have given some examples of

how you can foster the feelings of autonomy,

competence, relatedness and value in order to

boost participant motivation. Yet, we, researchers

and brands, can do many other things. We need

to acknowledge that we have to step away from a

20-minute survey where the only thing

participants can do is answer the researcher’s

questions. An engaged consumer will

contribute better to research, will provide

better and richer answers and is more likely

to share his experience with others.

Considering the fact that surveys are yet another

touch-point between consumers and brands, this

will result in a win-win-win for all of us. In addition,

we need to go beyond that single-box

thinking in survey research. Each set-up

should ideally combine all four collaboration

modes and allow consumer to connect and reflect

with one another.

The activities and tools used in each of these

collaboration dimensions will depend on the research

objective, yet only combining the ‘asking’ dimension with

other modes will allow to uncover the real consumer

reality. Traditional survey research is still far too

focused on the individual consumer, yet this

illustrative case shows that we can benefit greatly from

introducing a social dimension in survey research. The

framework can be used beyond survey research. Online

research communities, for example, mainly touch upon

the group dimension (shared feeling of belongingness

amongst participants), with the different community

activities tapping into the different collaboration modes.

This framework thus allows us to think beyond the

boundaries of traditional research methods and

will extend the possibilities of true fusion research.

We should move beyond silo thinking and realise that, in

fact, it is not only about thinking out of our ‘individual

asking’ box, it is about realising that there really should

be no box at all in research. It is about grasping

consumer reality to the fullest and using creative

methods to reach that goal.

I would like to take this opportunity to thank some people who helped me

throughout all phases of this project.

Firstly, I would like to thank Annelies Verhaeghe for all her support, creative

thoughts and knowledge sharing. I also thank Ioana Joanta and Dieter

Verschueren for their contributions in developing the tools we used for this

project. A special thank you goes to Joost Van Eyck, Helmont Siau and

Ken Vanderbeken, all part of the development team at InSites Consulting,

for creating The Village platform. Also to Hannes Willaert, the creative brain

behind the look and feel of the modular survey design and The Village.

This project would not have been possible without the collaboration opportunity

from eBay and Barbara Langer in particular. Also great thanks to

Christophe Vergult and Annelies Verhaeghe for managing the content

side of this project.

Last but not least I would like to thank our sampling partner SSI, who helped

us with the recruitment for this study, and in particular Pete Cape, for sharing

his knowledge on participant motivations.

Barbara Langer

Head of Market Insights

eBay Europe

Katia Pallini

Senior Research Innovation

Consultant

InSites Consulting

Annelies Verhaeghe

Managing Partner

InSites Consulting

www.insites-consulting.com

Thank you!

@InSites

marketing@insites-consulting.com

www.facebook.com/insitesconsulting

www.slideshare.net/InSitesConsulting