The paper also shows how ‘Online Communities’ are becoming a true ‘Fusion Research’ tool more and more which allows for ‘triangulation’ on different levels (data sources, research methods, research environments, theories and investigators),

leading to more valid research results, fresh inspiration and a deeper understanding of the issue researched. Best practices concerning: moving an existing qualitative project online, creating internal buy-in for emerging research methods, engaging internal audiences with research findings, running communities in different cultures and reactivating an MROC over time, are shared as well.

What to expect?

For IKEA, the yearly Catalogue is the main communication channel with existing and potential customers globally. This case study show

how the 2013 edition of the Catalogue and possible covers for the

2014 edition were evaluated qualitatively around the world, through ‘Market Research Online Communities’ (or ‘Consumer Consulting Boards’) in five different countries.

The IKEA Catalogue

Evaluating an icon of a global brand

IKEA has the vision ‘to create a better everyday life for many people’ by ‘offering a wide range of well-designed, functional home furnishing products at prices so low that as many people as possible will be able to afford them’. For IKEA, their yearly Catalogue is one of the main channels of communication with existing and potential

customers globally: one Catalogue showcasing one product offering to serve ‘the many people’ around the globe. Every year, IKEA conducts a ‘global’ qualitative study to understand how people feel and think about the latest edition of the Catalogue and what they do with it (this was done before the actual launch).

A few weeks after the global distribution, a large quantitative study was conducted to evaluate the new edition on key performance indicators, to measure the impact it had in the market and to benchmark the performance with previous editions. The findings of these studies serve as input for the creative team that works on the latest edition.

For the 2013 edition, significant changes were made to the format (slightly bigger), content (offering more inspiration pages, including more storytelling and a different way of picturing the products) and structure of the Catalogue (Figure 1). Furthermore, a complementary mobile application was launched. These structural changes made it even more

important to deeply understand people’s emotional and rational reactions to the latest edition.

From good to great

3

2

1

Over the past years the feeling had grown within IKEA that the qualitative evaluation of the Catalogue, done through offline focus groups in different countries around the world, could potentially be done differently and better and in a fresher way. Here is a shortlist of the key issues that the business owners and the research team at IKEA were encountering:

Focus groups were only giving a snapshot of reality: the first reactions to the new Catalogue. No real insights were gained on people’s second thoughts and their behaviour afterwards (how they are using the Catalogue and how that use evolved over time).

Secondly, given the limited time spent with consumers and a limited portfolio of research techniques possible,

focus groups did not bring that much fresh and inspiring information to the table. Every year, a new edition of the Catalogue was only judged partially (due to the limits of the method) and in the same way, leading to conclusions that were very similar year after year.

Another issue flagged by internal stakeholders was the limited number of participants per session in a focus group and the dominance of certain individuals in the discussion. Communities give access to more opinions of people with a wide range of profiles. And due to the longer time period, all participants are given an equal chance to give their opinion.

5

4 Fourthly, the project owners experienced that the quality of a focus group was largely determined by the quality and experience of the moderator. There was probably interviewer bias as well. Country differences were possibly due to differences in moderation style, rather than real differences between cultures.

Finally, despite all the new technologies available to follow offline focus groups from home as a client,

internal stakeholders do not follow sessions that often.

Focus group transcripts are experienced as not that convenient and pleasant to go back to the real discussion for inspiration or to justify an element when creating or making decisions. The result is that internal stakeholders are less confronted with the ‘voice of the customers’ during and after the project.

Conclusion:

Given this situation, the research team at IKEA was on

the look-out for a method enabling them to understand both the emotional and rational reactions to the 2013 Catalogue and the mobile application. Furthermore, in order to really understand the Catalogue’s impact, it was also important to gain insights into how people use the Catalogue on a daily basis after it landed in their homes.

Given these objectives, a Market Research Online Community (MROC) (De Ruyck et al, 2010) was chosen as the backbone of this project in favour of traditional

focus groups (See Figure 2). Online closed platforms to have a dialogue and work together with consumers in five different countries around the world (Germany, Italy, Poland, the US and China)

seemed to be the answer to the above issues. But why was this really a better option?

A simple, but very useful framework to evaluate a new research method

and to demonstrate its effectiveness internally is to check if it is

providing ‘automatical’, ‘informational’ and/or

‘transformational’ benefits

1 Automational: doing things faster and more cost-efficient

A research community indeed takes more time to set up, but once created it can be reused without losing the time of a classical set-up phase. This is a plus in the process of crafting a new Catalogue: the platform

cannot only be used to evaluate the current issue, it can also be used as a source of feedback and inspiration during the process of creating the next

edition. For this project five communities with a 3-week duration were created. Eight weeks later the platforms were reopened for one week to get feedback on the next stage - the creation of the cover for the 2014 edition.

Secondly, the set-up cost of a community project is higher than that of an ad hoc qualitative project, but once

established and in use it becomes a cost-efficient tool: more and different research methods can be combined within the same budget.

The @Home Community

2 Informational: obtaining a better data quality and deeper insights

Communities are characterised by the fact that you can work with more people of different profiles - in this case with 50 participants in each of the five countries. We included three different profiles: potential, existing and lapsed customers. The result was five MROCs to hear the opinions of ‘the many people’ in one single project, on one platform.

The ‘research on research’ we did in the past taught us that for most participant profiles and research objectives

it is best to run a community in the mother tongue of the participants. Firstly, by doing so they will

discuss more and they will post in a more nuanced and emotional way. Secondly, our experience

showed that it is a must for the community to be moderated by a native who knows the language, the local culture and the local market. All of this will lead to more relevant and to-the-point customer dialogues. That is why we made the choice to set up a separate MROC in each of the five countries. This approach also allows us to run the whole project at once and in parallel. While being in contact with participants via a local moderator, you still grasp the advantages of having a central/global project team and content overview when ‘connecting the dots’ on a global level.

By working with a single master topic guide (which is adapted to the local reality and culture), interviewer bias is partly eliminated. The project is run by a team of different people who work together both on country and global level.

3 Transformational: doing things which were not possible before

Evaluating the Catalogue in an MROC

gives us the opportunity to work with the same participants over a longer period of time (in this case: three weeks of 24/7 contact) in the comfort and context of their own living environment (home). In this project, we

followed the evolution in perception and usage of the Catalogue over time (before the Catalogue arrived, the first flip-through moments and the different reading sessions). This is something that is hard to realise cost-efficiently with traditional methods.

Another advantage of having more time with participants is

‘consecutive learning’. One can build further on what one learned previously. Or stimuli material can be tested, adapted and tested again in only a matter of days with the same group of consumers.

To obtain a holistic view on a participant’s use and perception of the

Catalogue, a whole range of research techniques was plugged in into the community platform: observational tasks, an online diary, a collage tool, mini-surveys, creativeexercises, group discussions and even implicit/emotion measurement tools.

Communities allowed us to blend different research methods, giving us depth and breadth in terms of the insights we found.

The research team did not have a hard time convincing the

internal clients to make the switch, as it was clear that the time had arrived for something different and better than offline focus groups. The advantages of using a community in the evaluation of the Catalogue (and the IKEA Catalogue App) were very clear as well. The questions arising among the business owners were more about the practical side

of things, e.g. how to choose and optimally combine the different research options available on the platform, how to safely distribute large numbers of Catalogues to different parts of the world before the actual launch and how to make sense of a big pile of information.

A research community as the backbone for fusion research

In order to bring fresh and unique insights to the table and to get a valid, clear and holistic view on people’s

perception and use of the 2013 Catalogue, we used the principle of ‘triangulation’ (Guin et al, 2012) to create a

true Fusion Research tool: investigating the same subject or issue from different angles and by doing so

create a more adequate and deeper understanding. We applied ‘triangulation’ on five different levels:

1 Data triangulation: by including both existing, lapsed and non-customers in the evaluation, we get data on all three groups and a full view on how different receivers of the Catalogue react to it: what do they feel, think and do?

2 Method triangulation: by combining different observational, qualitative (both interviewing and discussions) and quantitative research techniques (both explicit and implicit measurement) we tackle the same issue from different angles, leading to a holistic view of it.

3 Environmental triangulation: the mobile application used on the community platform allows participants to not only take part on the main research platform from behind their PC, but to also provide us with more personal and contextual information - in this case on how they use the Catalogue on a day-to-day basis in their homes and over time.

4 Theory triangulation: as we know that ‘people think less than we think they think’, it is important to take into account both ‘System One’ and ‘System Two’ thinking when asking people to evaluate the Catalogue. That is why we included both an implicit and an explicit test of the 2014 cover during the one-week reactivation of the community.

5 Investigator triangulation: by asking the local community moderators, the global project team at agency side, the client-side researchers, the internal client and the research participants to analyse certain parts of the data, we made sure we got everything out of it and our final conclusions included different points of view from people with different backgrounds. The local moderators keep an eye on aspects specific to the local culture and market of their country, the global research team looks for the global consensus, the client-side researchers and internal client frame the results within the business context and participants help us to close the final blind spots we have. In this project we asked ambassadors from the five different country communities to join forces in an English-speaking Global Room where a discussion took place concerning the cover test during the reactivation, to see if and how it was possible to come to one cover for all countries. The view of the participants helped us to see which cultural differences could be won over.

Applying the principle of ‘triangulation’ asks for more time and man-hours. One needs to make the trade-off between cost and getting additional understanding. In this project, it was a must to get a 360° view on how the

Catalogue was perceived and used over time. It was only by making use of a ‘Research Community’ and applying the principles of ‘triangulation’ to it, that we gained a fresh and complete view on the evaluation of the 2013 Catalogue.

Next we give examples of each of the different tools

used to bring Fusion Research into practice

Phase 1: Evaluation of the 2013 Catalogue during three-week communities in five

different countries

During the first week the goal was to ‘meet the reader’

and understand the actual and aspirational behaviour of the participants. Who is the reader and what are his/her expectations towards the brand and the Catalogue? To get there we conducted

amongst other activities a mix of an ethnographic task to get a view of their house and insights into how they live, a mood board exercise in which we asked the members to map their feelings about the IKEA brand and creative tasks on the forum of the platform (e.g. Tell us ‘Your IKEA Catalogue story’).

After the Catalogue was dropped of at the participants’ homes (in preview, two weeks before the

real global launch), we assessed the perception, satisfaction and level of engagement the 2013 Catalogue evoked. First impressions and second thoughts on particular aspects of the Catalogue were researched (structure, pictures, stories, the Ikea Catalogue App, etc.). It was important to understand

how the Catalogue offers both inspiration and information to the reader and if that was done to the right extent. Moreover, we wanted to get a grip on the life cycle of the Catalogue. We did so by adding three specific exercises:

A photo safari, in which we asked the participants to spot the Catalogue in their house. We wanted them to imagine that the Catalogue could speak and tell the story of his past, current and future life.

During the last two weeks of the community, a calendar tool was added to the platform to get day-to-day insights in the usage and the emotions it evoked. This was done in a private part of the community (participants could not read the responses of others), which aimed at avoiding bias.

Finally the Ikea Catalogue App was tested by understanding the expectations of these types of apps, asking them to use the app and to evaluate it afterwards on its relevance, user-friendliness and discuss possible improvements for the next version.

During the final week, we investigated whether the Catalogue was meeting expectations, giving us first clues on the impact the latest Catalogue had on the brand perception and shopper behaviour. In other words, what impact did it have on the business: was it attracting people to the shop, raising interest in home furnishing and was there a positive impact on the perception of the brand, especially among lapsed customers?

Phase 2: Reactivation of the different communities (after eight weeks of inactivity) to

test the overall theme and the first cover ideas for the 2014 Catalogue

This was done in a two-step approach during one week:

A mini-survey (N=226, response rate > 70%) to test six options for the new cover (illustrating the theme)

Each participant randomly evaluated three different covers (resulting in at least 90 participants for each cover tested). The questionnaire was a mix of explicit questions (fit of the covers with brand and mission statements) and emotional/implicit measurement:

• Two-second test: each participant saw a cover for only two seconds. Afterwards, via an open-ended question, the members of the communities were to write down what they saw, which emotions they had felt and what they remembered. This was done three times for each participant in randomised order, to exclude order effect. We were measuring ‘stomach impact’ here explicitly, though spontaneously.

• Implicit measurement: each participant saw the same three cover pages for 10 seconds. Afterwards, the members had to execute an ‘implicit measurement’ assignment where 15 emotions were shown rapidly (one second) in randomised order. Participants had to press the space bar each time they associated the emotion with the cover they just saw. Here we measured unconscious emotional reactions to seeing the cover page.

By combining the breadth of the ‘stomach impact’ on the X axis (the share of people who associate a given cover with the emotion, via implicit

measurement) with the depth on the Y axis (for those who have associated the emotion with the cover, how strong is that association, i.e. how rapidly

was the association made). To make interpretation easier, the scale of the Y axis was reversed (= the higher, the faster) (see Figure 3). Results of these measurements were thrown back into the discussion during phase 2, in order to fully understand them.

Continuing with a discussion on the forum: in the survey, the participants were confronted

with different cover ideas

After two days they were asked which of the covers they still remembered. We used a tool in which you first have to answer yourself, before the others can see your answer (which ensures that answers are not biased). We did this to assess the ‘stomach impact’ on the mid-long term. After the spontaneous recall test,

we conducted the spontaneous associations test: upon showing the different covers, participants were asked to share their initial thoughts, without prompting. Via this exercise we got an understanding of the key associations made with each cover. Finally, by asking indirect questions (e.g. tell the story of the cover), participants were motivated to elaborate on the indirect impact of the cover. Finally, the most

remarkable results of the survey were shared and the participants were invited to comment on them in order to maximise our understanding.

‘Battle of the covers’ across the five countries in order to find creative ideas that would have

appeal across the globe

All participants of the five communities got the opportunity to take part in this discussion in English in a central room. This way, cultural differences and local preferences were unveiled.

Reactivating a research community

The previous paragraphs informed us about the fact that a community provides one with automational, informational and transformational benefits and that it is a great tool to apply ‘triangulation’ to. The examples provided in the previous paragraph on how we evaluated the 2013 Catalogue and pre-tested the themes and cover for the 2014 edition show that it can be a handy tool while creating a new Catalogue as well. A community does not need to be ‘always-on’. You can perfectly align it

with business planning. In this case the communities were used to draw learnings from the previous edition and to get feedback on first ideas for the new one. We can imagine situations later on in the creation process where we might want to reactivate the communities for a short period of time. Is that possible? Are participants indeed willing to participate again and what does it take to encourage them to do so?

The answer to the first question is ‘yes’: almost allparticipants from the communities in the US, Poland and Germany took part again. For Italy (78%) and China (70%), the reactivation was a bit less successful, although this was neither expected nor communicated at the start of the community.

Cultural differences in commitment are probably at the basis of the lower numbers in both countries. For all the questions on the different communities, we had at least 30 posts, which is what we needed to reach our saturation effect (Schillewaert et al, 2011). Across the five communities we had

2,807 interactions in a week’s time, which is 38% of the 7,261 we generated in the initial three weeks.

This is perfectly in line with what one may expect.

By making the participants feel part of the company as real ‘consultants’. Before the first phase of the project a kick-off was organised in a 30-minute chat session in smaller groups, during which we explained who the client was, what the goal of the project was and what was in it for them.

During the first phase, participants received weekly newsletters on the progress of the project and what the company was learning from the discussion.

How did we get to these results?

After the first three weeks and the workshop at the company side the participants got an update on how the debrief in Sweden went and pictures from the office were the Catalogue is

created and a word from the team behind it. It is by doing so and by keeping your promises in terms of incentives that you create a strong relationship with the members.

Global MROC projects: best practices

The fundamentals of the community approach work on a global scale. Just like the

brands we are working for, we need to localise our way of working from country to country. In order to fully understand to what extent localisation of our methodology is required, we

conducted several studies with moderators from our ‘Global Community Moderator Network’ (recruited in 30 different countries) and with local research participants. This way we

co-created best practices for the different markets we are operating in. We found that it is important to adapt your community to the local culture on five aspects. The direction the adaptations are made in can almost always be explained by the work of the Dutch academic researcher Geert Hofstede and his five dimensions to explain cultural differences between

countries. Next we explain the different dimensions in more detail and we also add some striking examples.

1 Reason to participate: intrinsic vs. extrinsic motivation

From the first e-mail invitation for joining the

community onwards, it needs to be clear what is in it for the participants. In almost all countries, the main reason to

participate is the possibility to have an influence on the future of a brand or a product.

We also noticed that some countries are more extrinsically motivated than others. This is especially the case in the US and in Eastern European countries, but for different reasons. Americans consider it normal that there is a payment to reward performance. In most Eastern European countries on the other hand an (monetary) incentive is perceived as a nice extra on top of their monthly income. Furthermore, in Poland, it is a must to gain the ‘trust’ of the members - trust in the fact that the agency or company behind the community will not harm them in any way and also in the fact that they will really

get their incentive. The preferred type of incentive differs from country to country. It is an illusion to think that ‘PayPal fits all’. In Asian countries such as China, the intrinsic part is important: they like to be connected with aspirational brands and sharetheir wisdom.

2 Conversation guide: empowering vs. directive

A different culture also means different attitudes and values, leading to a different way of reacting to certain questions, tasks and exercises that moderators want participants to perform. Some cultures, for instance, love to share a lot of details about themselves and their lives.

Others prefer talking about the group, which is considered to be a safer option. See it as a

projective technique to let people talk about their own situation, free of any pressure. The

same holds for co-creation exercises. It is not a given in every culture that people are used to taking initiative. They feel better when they are only asked to give feedback about what already

exists. It is important to map the country that one is working in on those two axes (‘me’ vs.

‘we’ and ‘feedback’ vs. ‘co-creation’) and to adapt the way of writing and (re)mixing topics for the conversation guide to it.

3 Role of the moderator: facilitator vs. authority

One does not only need to adapt the way of inviting and incentivising the members, the medium of data collection and the nature of the

topics in the conversation guide. The role of the moderator is also perceived differently from one country to the next. In Italy a moderator needs to facilitate and start the discussion. His/her role lies more in the background.

But it is also expected from the moderator that he/she is steering the discussion in the right direction when it is going off topic. In Poland on the other hand, the moderator needs to be strict and almost literally direct the members to the next question or task they need to look into. At the same time the moderator in Brazil is a social peer, he/she needs to be a formal professional in China and a like-minded person to

exchange wisdom with in India. It is crucial to know and manage all these different expectations when running (multi-country or global) ‘Consumer Consulting Boards’.

4 Gamification: playful vs. serious

Adding elements of ‘gamification’ to the community brings more richness to the table. In our research-on-research among our

moderators, we learned that the level of and the intensity with which you gamify your ‘Consumer Consulting Board’ need to differ between countries. In Germany, for example, it is wise to limit it to a minimal level as it is culturally less accepted.

The elements described on the previous slides show that in multi-country projects you need to start from a master conversation guide, which is important to make sure that there is a uniform

way of working and that you exclude the effects of interviewer bias. But adaptations both in content and style of the topics and in the way of moderating will be amongst other key elements in making the community a real success. Furthermore, it is wise to plan several debriefs between the different local moderators facilitated by the global research team: to challenge each other’s conclusions, let them go back to the results of their own country and come back with deeper and richer understanding in several iterative loops.

Creating a dialogue between the internal and external world

Only the business owners were following the

community closely, although it should be convenient as no travelling is required and one can take a look at the discussion on the community when one feels like doing so. As a researcher there is a clear need to give intermediate updates to the different stakeholders, as they do not follow the discussion spontaneously and there is a lot of information to digest. Moreover, this gives the business owners the opportunity to finalise the topic guide for the upcoming

days: going deeper into certain elements that are really interesting or pushing the discussion in a new direction.

Previous ‘research-on-research’ (De Ruyck, 2011) and our experience during this project have taught us that

confronting stakeholders with real stories by real people is very impactful as an illustration of the main conclusions. They become alive. It is also an advantage that stakeholders can go back to the community based on the final report and readexactly what and how customers put it during the

community. Finally, it is great that a community can be reopened when you need it, perfectly in line with the business planning.

The research results have led to significant changes to

the Ikea Catalogue App. We now know that the new concept of the Catalogue was a big step into a new and right direction and improvements will be made for the 2014 edition, based on the research.

The future for research communities

The power of ‘Research Communities’ as a methodology lies in the fact that you have the ability to work with more people, over a longer period of time, and that you get to know a lot from all kinds of different angles by combining different research tools and methods. The latter especially is still untapped potential in most communities. This case study describes how fusing observational, qualitative and quantitative research methods lead to a deeper understanding and new insights. The case also demonstrates that ‘Structural Collaboration’ with consumers over time is valuable and leads to more impactful communication tools.

References

De Ruyck, T. et al (2010). ‘How Fans Become Future Shapers of an Ice-cream Brand’, Proceedings ESOMAR Qualitative

Guin L., Diehl D. and McDonald D. (2012), Triangulation: Establishing the validity of qualitative studies, IFAS

Schillewaert, N. et al (2011). ‘The Darkside to Crowd-sourcing in Online Research Communities’, CASRO Journal

De Ruyck, T. et al (2011). ‘Engage, Inspire, Act: 3 Stepstones towards Developing more Impactful Products’, Proceedings ESOMAR Congress

Tom De RuyckHead of Consumer Consulting BoardsInSites Consulting

Pieter De VuystSenior Research ManagerInSites Consulting

Frédéric GennartGlobal Market Research ConsultantInter IKEA Systems

Frank NaessensSenior Research ConsultantInSites Consulting

tom@insites-consulting.com

+32 9 269 14 07

Tom De RuyckHead of Consumer Consulting Boards

Want to know more about Consumer Consulting Boards?

http://www.insites-consulting.com/

http://www.insites-consulting.com/