1

“Simulated Test Marketing in FMCG:some empirical evidence from the Russian market”

1. INTRODUCTION

According to “The Marketing Glossary”, a simulated test market is “a form of market testingwhere consumers are exposed to a simulated purchase situation to gauge the buyers’ reactionsto a product, advertising or marketing mix variations… It is used in marketing planning,estimating market demand and sales forecasting” (Clemente, 2002, p.391). Simulated TestMarketing (STM) is widely employed to predict new product sales in the U.S. and Europe.Clancy et al (1994, p. 46) pointed out that STM is ‘the single most validated tool in marketingresearch’. According to a recent study by Wherry (2006), the average accuracy claimed fordeveloped markets has reached ±9%. An on-going discussion between STM industry leadersat ESOMAR Congresses (Willke, 2002; Markowitz, 2010) reveals a number of areas forfurther improvement. Although Markowitz (2010) highlights the rising need for international“transferability”, the discussion revolves around mature “western” markets. The importanceof emerging markets is visibly underestimated in spite of their rapidly growing contributioninto the global FMCG sales.

Nowadays, there is a consensus among academics that international markets are vastlydifferent (e.g.: Usunier, 2000; Burgess & Steenkamp, 2006). Markowitz (2010, p.19) notesthat, “it is not uncommon for products to be successful in some markets but fail in others.Sometimes success and failure relate to local tastes but other times it is due to a lack ofunderstanding of certain characteristics of the market”. This therefore suggests that tools forresearch should also be adapted and customized, taking in account the unique features of agiven market, such as for example the Russian FMCG market (Kachalov, 2008; Malhotra,2007), on which this paper will focus.With reference to the use of STM in emerging markets and, particularly in Russia, there is avisible lack of academic knowledge. The observed accuracy for the Russian market is notreported and it is not available from independent reliable sources. Moreover, the effectivenessof Simulated Test Marketing in Russia hasn’t been discussed yet in the academic and businessliterature. This paper is therefore addressing and built around three key questions: (1) Whatare distinctive characteristics of the Russian FMCG market as compared to the majordeveloped markets? (2) Which forecasting techniques are used to predict new product sales inthe Russian FMCG market and why? (3) If traditional methods of Simulated Test Marketingare used, what are their advantages and disadvantages in the context of the local market?What is their observed accuracy?

The next paragraph reviews key theoretical and practical aspects of Simulated TestMarketing. Paragraph 3 provides an insight into the Russian FMCG market. Paragraph 4discusses the methodological approach taken to address study questions. Finally, Paragraphs 5and 6 present key findings and provide some managerial implications.

2. SIMULATED TEST MARKETING: THEORY AND PRACTICE

The history of STM starts with the rise of mass marketing and consolidation of retail in theU.S. in early 60-ies (Clancy et al, 2006; Kratt, 2009). STM’s theoretical foundations aregenerally built upon the fundamental works by Fourt & Woodlock (1960), Juster (1966),Charnes et al (1966), Parfitt & Collins (1968), Eskin & Malec (1976), Urban (1970, 1975),

2

Urban & Silk (1978) and Lin (1986,1997). The methods have significantly evolved over thepast decades and now “all major STM research models are similar enough that they exhibitsome common strengths and weaknesses” due to the use of few well-developed calculationprinciples (Clancy et al, 1994, p.49). The first principle is that future sales are typicallydecomposed following Fourt-Woodlock approach, i.e. “trial” and “repeat” components areestimated separately. This represents the major challenge for modern STMs to provideaccurate assessments for both “trial” and “repeat”. The second principle is concerned with thefundamental way of purchase probability estimation. According to available reviews (Shocker& Hall, 1986; Baldinger, 1988; Mahajan & Wind, 1988; Baldinger & Haley, 1991; Lilien etal, 1992; Clancy et al, 1994, 2006; Bemmaor, 1995; Bockenholt & Dillon,1997; Fader et al,2003; Wherry, 2006; Peng & Finn, 2007), currently existing models are based either on“purchase intent” or “preference share” approaches. The first method implies monadicevaluation of the tested offer using “purchase intent” scale (e.g. Juster’s scale), while“preference share” measures probability of choice in a competitive context, very often in astore-like environment. According to Wherry (2006), there are two world leading models,Nielsen’s BASES (over 50% market share) and Ipsos’ DESIGNOR (around 20% share). Bothmodels claim accuracy of ±9% in developed markets (85% of validations) and differentiatebetween various types of new product launches (Booz et al, 1988; Griffin, 1997; Schneider,2004). They are well-fit in the standardized NPD process, such as “stage-gate” (Baldinger,1988; Baldinger & Haley, 1991; Urban & Hauser, 1993; Tidd & Bessant, 2009; Cooper &Edgett, 2010) and perfectly integrated with other research tools (Malhotra, 2007). However,they attract a lot of criticism in various aspects, mostly due to their tendency to penalize“true” innovations, poor performance on entirely new or fuzzy markets, difficulties withmodeling new media and retail effects, relative complexity and inflexibility (Gundee, 1982;Watkins, 1984; Belinson, 1986; Armstrong & Scott, 1987; Mahajan & Wind, 1988;Baldinger, 1988; Schlossberg, 1989; Wilson, 1990; Baldinger & Haley, 1991; Prince, 1992;Hamel & Prahalad, 1994; Francis, 1994; Martin, 1995; Christensen, 1997; Mahajan & Muller,1998; Von Hippel et al, 1988, 1999, 2002, 2005; Hart et al, 1999; Trott, 2001,2008; Bilgramet al, 2002; Belliveau et al, 2002; Veryzer, 2003; Fader & Hardie, 2005; Wherry, 2006;Morwitz et al, 2007; Hoffman, 2007; Markowitz, 2007; Buur & Matthews, 2008; Hassan,2008). These findings have triggered some of the recent discussions at ESOMAR conferencesabout the future of Simulated Test Marketing (Willke, 2002; Markowitz, 2010). Thosediscussions seem still to have paid, as mentioned above, relatively little attention to futuredevelopments of STM in emerging markets, such as Russia.

3. INSIGHTS INTO THE RUSSIAN FMCG MARKET

Unlike developed “western” FMCG markets, where “whatever real growth there is, comesfrom population increases, which never exceed 1 to 2% a year,… and new product mustwrench market share from other, established brands” (Clancy et al, 1994, p.7), manyemerging markets exhibit sustainable double-digit growth every year. Therefore, they havebecome very attractive for FMCG multinationals, which generate 30-50% of their revenuesthere (Colgate, 2009; Unilever, 2009; TCCC, 2009; Nestle, 2009; ABInBev, 2009; Danone,2009; Henkel, 2009; Pepsico, 2009; L’Oreal, 2009; P&G, 2009; Kraft, 2009). Thus, in Russia,for example, savory snacks is to grow by + 36% by 2013 in terms of sales volume per capita,soft drinks by +29%, hair care by +23%, make up by +21%, confectionery by +17%, meat by+17%, fish by + 17% etc., while growth in the U.S., Europe and Japan is estimated +4% onaverage (Datamonitor, 2010; Euromonitor, 2010). The size of the Russian FMCG market isalready comparable to that in the U.K. or Germany and is still far from saturation(Datamonitor, 2010; Euromonitor, 2010). As discussed earlier, Russia has become an arena

3

for expansion of the biggest manufacturers that pursue “penetration” strategies probing themarket with their “new-to-the-country” products. While in the case of “developed” marketsthe main risk is mainly due to stiff competition, in Russia it is firmly related to a dynamiceconomic environment, lack of reliable information, different consumer culture, poor salesinfrastructure, tough regulations and fragmented underdeveloped markets (Usunier, 2000;Schorsch, 2009; Economist Intelligence Unit, 2010).The process of new product development is also radically different in Russia as compared to“developed” counties, driven by the sense of business urgency and desire to capture emergingmarket opportunities. Due to the “import-oriented” nature of operations, in most cases, theprocess of new product introduction is characterized by a very short R&D stage, transferringproduct ideas (or products) from the other markets, inadequate use of marketing researchtools, high emphasis on building sales infrastructure, short-term horizon of planning, focus onimmediate profit skimming. That is in line with what several authors pointed out as being thekey features of the Russian market: (1) lower conservatism of consumers than that in thedeveloped markets; (2) high importance of social networking; (3) lack of reliable statisticsand benchmarks, that undermines general trust in marketing research, including STM; (4)rising need for trustworthy research techniques applicable to the local market, including STM;(5) confidentiality issues (Burdey et al, 1999; Belotserkovskaya et al, 2005;PriceWaterhouseCoopers, 2006; Agaeva, 2008; Kachalov, 2008).The above discussion already suggests that STM methods require some careful adaptations tothe Russian market and many caveats need to be considered - in particular, limitations on useat early stages of market development and the ultimate need for local customization.However, two key questions still remain unanswered: (2) Which forecasting techniques areused to predict new product sales in the Russian FMCG market and why? (3) If traditionalmethods of Simulated Test Marketing are used, what are their advantages and disadvantagesin the context of the local market? What is their observed accuracy? In order to be able toaddress those questions and identify present features of STM in the FMCG Russian context,key users of forecasting techniques in Russia have been involved in a B2B Usage andAttitude survey (Parameswaran, 2005).

4. RESEARCH METHODOLOGY

As this study is aiming at identifying most commonly used forecasting techniques amongFMCG companies in Russia and understanding perceived advantages and disadvantages,including accuracy, of traditional STM, a quantitative survey has been undertaken. Accordingto Sekaran (2003) and McNeil (2005), the proposed methodological approach falls under thegeneral category of “descriptive studies”. The target population can be defined as NPDmanagers, business and consumer insight managers, category and brand managers, strategicplanning managers, i.e. in-house experts engaged in sales forecasting for new products in theFMCG market. With that, according to 20/80 rule, a skew towards the biggest advertisers isrequired in order to achieve a representative sample of marketing research users, particularly,the users of STM. Therefore, a “non-probability quota sampling” is required (Malhotra, 2007,p. 340), i.e. 80% of interviews should be conducted with employees of FMCG companiesfrom the Top-100 advertisers list (Tab.1). As the entire universe of client-side experts isestimated to be about 300 people, a sample size of n~30 respondents can be considered as aminimum number to achieve some fair conclusions. In this case, the margin of error variesfrom ±5% to ±14%, assuming finite universe population N=300 and confidence level 90%(Malhotra, 2007). Evidence coming from a similar study by the U.S. Advertising ResearchFoundation - which was able to survey n=42 experts - (Baldinger, 1988) suggests that it isextremely difficult and unlikely to collect data from a large sample in such studies,

4

particularly considering that confidentiality issues are one of the above mentioned features ofthe FMCG Russian market. During a two months period in 2010, 72 questionnaires were sentout, with a response rate around 65%, as 48 questionnaires were returned, 33 of which fullycompleted, 14 partially completed and 1 was fully completed, but excluded from the analysisupon request of the participant. Further details are shown in Fig. 1.The questionnaire consisted of 25 questions specifically designed to address researchquestions and identify use and perceptions of forecasting techniques and particularly STM.The estimated time to complete the questionnaire was 25-30 minutes. The online datacollection was aided by Sawtooth CiW software and data were processed using SPSS.The outcomes of the study were sent to all respondents, who in most cases provided feedbackby email or phone conversations appreciating the usefulness and in-depth insight of the study.

5. KEY FINDINGS AND DISCUSSION

The following paragraph integrates key findings with some discussion. According to the data,the frequency of use of sales forecasting per category (Fig.2) highlighted that the number of“line extensions / brand stretching” and “brand re-launches” is in line with that in “western”markets - 23% and 4% respectively, but relatively higher for “product/price changes”compared to a previous US study by Griffin (1997). Data, apart from changes occurred since1997, were most likely affected by the global economic downturn, which peaked in 2009. Thestudy results indicate that respondents are quite familiar with the fundamental methods ofmarketing research (Tab.2). A majority of respondents participated in a Simulated TestMarketing project at least once during the year 2009 (62% of surveyed experts). Thisconfirms theoretical prediction that STM has further potential to expand in the RussianFMCG market. The data analysis performed indicates that STM is more often used in the caseof true product innovation, while in the case of ”product/price changes” simpler quantitativediagnostic tests are frequently applied. Not surprisingly, reliable information is key to salesforecasting success in Russia (36% respondents) (Tab.3). Respondents point out the followingactual problems in sales forecasting, that need to be addressed: (1) accuracy of market data(39%), (2) tailoring existing approaches to the local market reality - in terms of design,accuracy, flexibility, cost (36%), (3) validations, acquiring historical data, benchmarks,working out evaluation criteria (29%), (4) improvement of business- and media- planningprocesses (25% and 21% respectively), (5) lack of knowledge about forecasting and researchtechniques among end users (e.g. brand teams, sales teams etc.) (Tab.4). Concerningsuccessful new product launches, over 70% of respondents reported that, in their experience,more than 25% of launches can be considered successful (i.e. in line or above business plan).This is significantly above that in “western” markets – 5%. Also, as per experience of experts,the accuracy of forecasts does not exceed 20% on average (Fig.3). More than a half of expertsindicated that they had previous experience with the most popular “western” approaches –ACNielsen BASES and Ipsos DESIGNOR (Fig.4). About 75% of respondents stated thataverage budget of STM studies they had recently participated in was above $30.000. Thisindicates that general price level for Simulated Test Marketing in Russia is comparable to thatin developed markets. However, despite this considerable price barrier, more than 60% ofrespondents believe that the market of STM in Russia will grow in the mid-term perspective -i.e. in the next 3-5 years (Fig.5). Study results suggest that Simulated Test Marketingprovides acceptable level of accuracy in key Russian FMCG markets. However, the numberof precise forecasts is slightly below to that in “western” markets (Fig.6). To certain extent,this suggests that the hypothesis about lower accuracy of STM in the Russian market is likelytrue, even though the significance of that difference cannot be technically tested within thecurrent study and requires additional exploration. Answering the question on why particular

5

STM model can (or cannot) be recommended, respondents mentioned such factors as: (1)reliable and accurate forecast, proven accuracy in the local market, (2) professionalperformance, quality service and consulting, individual approach and (3) simplicity (orcomplexity) of modeling approach (Tab.5). At the same time, ranking key factors in order oftheir importance had helped to reveal some other drivers, which determine selection ofSimulated Test Marketing model, in particular: (1) suitability for the Russian market, (2)recommendations from management or requirements by company research protocols (Fig.7).Comparative analysis uncovered perceived positioning of the leading STM approaches fromexperts’ point of view. Thus, BASES is often recommended by management or globalprotocols, while DESIGNOR produces a lot of diagnostic information. Locally developedSTMs are suitable for the Russian market and offer individual service at affordable price(Fig.8 & Tab.6). Analysis of correlation between particular image attributes and generalcustomer satisfaction revealed significant correlation between satisfaction and such factors as:(1) individual approach for every project, (2) high quality service and project management,(3) positive previous experience and (4) professionalism (Fig.9). This allowed exploring keystrengths and weaknesses of each particular STM model (Nielsen, Ipsos, Local STMs and‘Western STMs’). The main strengths of ACNielsen BASES (Fig.10) resulted to be:recommended by management or company protocols and well-known agency. The keyweaknesses that need to be addressed were: diagnostic information, speed of elaborating theresearch, transparent and flexible pricing. In the case of Ipsos DESIGNOR (Fig 11), the keystrengths were: a lot of diagnostic information, professional presentation of findings andrecommendations, well-known agency. However, the following perceived weaknesses wereidentified: model complexity, lack of management support and recommendations in companyprotocols. According to the respondents, the key areas for improvement and furtherdevelopment of Simulated Test Marketing in the Russian FMCG market at the moment are:1) more “client-oriented”, individual approach, greater focus on client’s business; 2) localR&D, i.e. gathering local historical information, benchmarks, knowledge and expertise,performing validations, developing evaluation criteria relevant to the Russian FMCG market;3) simplification of reporting and modeling approaches; 4) localized media models, gaugingeffectiveness of various marketing instruments in the local market; 5) reducing project timing;6) openness in explaining modeling principles, client friendly guidelines on terms of use,inputs, outputs, research implications, limitations, requirements to study materials etc.; 7)flexibility; 8) pricing (Tab.7)

6. CONCLUSIONS AND MANAGERIAL IMPLICATIONS

The research undertaken leads to a general conclusion that although traditional STM modelshave attained relatively high awareness among FMCG in Russia, their use is still limited.Instead, simpler alternatives are often employed, i.e. concept product tests. Although the mostcommonly acknowledged advantages of STM are well understood in Russia, there are keysome key barriers to its widespread adoption: poor quality or insufficient market data, lack oflocal market experience and validations, lower forecast accuracy as compared to “western”markets, low flexibility in terms of design and cost. Therefore, the future development ofSTM models in Russia and emerging markets with similar very dynamic environment shouldincorporate a greater degree of adaptation to local market insights and provide a higher levelof perceived value and services. In view of the findings discussed, FMCG companies willappreciate flexible, proactive, “client-oriented” approach as opposed to conservative, “model-centered” services based on “global” execution standards. This should lead to the co-creationof STM models that would achieve more accurate forecasts in emerging markets and achievea greater level of confidence in the use of STM among Russian FMCG companies

6

REFERENCES

- AB InBev (2009) Annual Report 2009 [online] Available at: http://www.ab-inbev.com/pdf/AB_InBev_AR09.pdf [Accessed 12.06.2010]

- Agaeva, M. (2008) ‘Marketing guesstimates: does the manufacturer need testmarketing?’, Sales Business, No. 05/11, [online] Available at:http://www.sostav.ru/articles/2008/11/05/ko1/# [Accessed 12.11.2009]

- Armstrong, G. & Scott, J. (1987) ’Pretest market models: A critical evaluation’,International Journal of Forecasting, Vol. 3, Issue 3/4, pp. 542-543

- Baldinger, A. (1988) ‘Trends and Issues in STMs: Results of an ARF pilot project’,Journal of Advertising Research, Vol. 28 Issue 5, pp. 50-56

- Baldinger, A. & Haley, R. (1991) ‘The ARF copy research validity project’, Journal ofAdvertising Research, Vol. 31, Issue 2, p. 11-32

- Bell, D., Bonfrer, A. & Chintagunta, P. (2005) ‘Recovering SKU-Level Preferences andResponse Sensitivities from Market Share Models Estimated on Item Aggregates’,Journal of Marketing Research, Vol. 42, pp. 169-182

- Belinson M. (1986) ‘Gap widens in testing approaches for different consumer products’,Marketing News, Vol. 20 Issue 22, p. 18

- Belotserkovskaya, O. & Shuranov, A. (2005) ‘Is marketing research required forlaunching a new product in the Russian market?’, Company – A weekly businessmagazine, No. 11/04 [online] Available at: http://www.ko.ru/document.php?id=11660[Accessed 12.11.2009]

- Bemmaor, A. (1995) ‘Predicting Behavior from Intention-to-Buy Measures’, Journal ofMarketing Research, Vol. 32, No. 2, pp. 176-191

- Bilgram, V., Brem, A. & Voigt, K. (2008) ‘User-centric innovations in new productdevelopment', International Journal of Innovation Management, Vol. 12, No. 3, pp. 419-458

- Belliveau, P., Griffin, A. & Somermeyer, S. (2002) The PDMA toolbook for NewProduct Development, New York: John Wiley and Sons

- Bockenholt, U. & Dillon, W. (1997) ‘Some New Methods for an Old Problem: ModelingPreference Changes and Competitive Market Structures in Pretest Market Data’, Journalof Marketing Research, Vol. XXXIV, pp. 130-142

- Booz, Allen & Hamilton (1988), ‘New Products Management for the 1980s’, New York:Booz, Allen & Hamilton

- Boughton, P., Nowak, L., & Washburn J. (1996) ‘A decision model for marketingresearch relationship choices’, Journal of Services Marketing, Vol. 10 Issue 1, pp.56 - 69

- Brody, A. & Lord, J. (2000) Developing new food products for a changing marketplace,Boca Raton: CRC Press

- Burdey K. & Troyan N. (1999) ‘New product launches in the Russian market: stages andmarketing support’, Advertising Ideas, No. 1/99 [online] Available at:http://www.advi.ru/archive/article.php3?pid=163 [Accessed 08.07.2010]

- Burdey K. (2010) How not to embarrass yourself when ordering a market research study[online] Available at: http://www.okresearch.ru/article5.html [Accessed 08.07.2010]

- Burgess, S. & Steenkamp J. (2006) ‘Marketing renaissance: How research in emergingmarkets advances marketing science and practice’, International Journal of Research inMarketing, Vol. 23, Issue 4, Pages 337-356

- Buur, J., & Matthews, B. (2008) ‘Participatory innovation’, International Journal ofInnovation Management, Vol. 12, No. 3, pp. 255–273

7

- Charnes, A., Cooper, W., DeVoe, J. & Learner, D. (1966) ‘Demon: Decision Mappingvia Optimum Go-No Networks-a Model for Marketing New Products’, ManagementScience, Vol. 12, No. 11, pp. 865-887

- Chambers, J., Mullick, S. & Smith, D.(1971) ‘How to choose the Right ForecastingTechnique’, Harvard Business Review, Vol. 49, No. 4, pp. 45-74

- Christensen, C. (1997) The innovator’s dilemma: when new technologies cause greatfirms to fail, Boston: Harvard Business School Press

- Clancy, K. & Krieg, P. (2003) ‘Surviving Innovation’, Marketing Management, No. 2,March-April, p.112-121

- Clancy, K., Krieg, P. & Wolf M. (2006) Market new products successfully, Oxford:Lexington Books

- Clancy, K., Shulman, R. & Wolf, M. (1994) Simulated Test Marketing: Technology forLaunching Successful New Products, Oxford: Lexington Books

- Clemente, M. (2002) The Marketing Glossary: Key terms, Concepts and Applications,New Jersey: Clemente Communications Group

- Colgate (2009) Annual Report 2009 [online] Available at: http://investor.colgate.com[Accessed 12.06.2010]

- Cooper, R. & Edgett, S. (2010) Stage-Gate® - Your Roadmap for New ProductDevelopment [online] Available at: http://www.prod-dev.com/stage-gate.php[Accessed01.07.2010]

- Crawford, C. (1989) ‘How Product Innovators Can Foreclose the Options of AdaptiveFollowers', Journal of Marketing Management, Vol. 4, No. 3, pp. 277-287

- Danone (2009) Annual Report 2009 [online] Available at:http://danone09.danone.com/en/fb/data/catalogue.pdf [Accessed 12.06.2010]

- Datamonitor (2006) ‘Targeting Profitable Consumer Trends In Brazil, Russia, India andChina Insights into Emerging Groups & Behaviors in BRIC’, London: Datamonitor

- Datamonitor (2005) ‘Evolution of Global Consumer Trends’, London: Datamonitor- Datamonitor (2010) ‘Savory snacks industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Savory snacks industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Savory snacks industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Savory snacks industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Savory snacks industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Make up industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Make up industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Make up industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Make up industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Make up industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Dairy industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Dairy industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Dairy industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Dairy industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Dairy industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Soft Drinks industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Soft Drinks industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Soft Drinks industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Soft Drinks industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Soft Drinks industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Hair care industry in Russia’, London: Datamonitor

8

- Datamonitor (2010) ‘Hair care industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Hair care industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Hair care industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Hair care industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Household products industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Household products industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Household products industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Household products industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Household products industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Confectionery industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Confectionery industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Confectionery industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Confectionery industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Confectionery industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Alcoholic Drinks in Russia’, London: Datamonitor- Datamonitor (2010) ‘Alcoholic Drinks in the US’, London: Datamonitor- Datamonitor (2010) ‘Alcoholic Drinks in the UK’, London: Datamonitor- Datamonitor (2010) ‘Alcoholic Drinks in Germany’, London: Datamonitor- Datamonitor (2010) ‘Alcoholic Drinks in Japan’, London: Datamonitor- Datamonitor (2010) ‘Hot Drinks in Russia’, London: Datamonitor- Datamonitor (2010) ‘Hot Drinks in the US’, London: Datamonitor- Datamonitor (2010) ‘Hot Drinks in the UK’, London: Datamonitor- Datamonitor (2010) ‘Hot Drinks in Germany’, London: Datamonitor- Datamonitor (2010) ‘Hot Drinks in Japan’, London: Datamonitor- Datamonitor (2010) ‘Tobacco industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Tobacco industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Tobacco industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Tobacco industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Tobacco industry in Japan’, London: Datamonitor- Datamonitor (2010) ‘Beer industry in Russia’, London: Datamonitor- Datamonitor (2010) ‘Beer industry in the US’, London: Datamonitor- Datamonitor (2010) ‘Beer industry in the UK’, London: Datamonitor- Datamonitor (2010) ‘Beer industry in Germany’, London: Datamonitor- Datamonitor (2010) ‘Beer industry in Japan’, London: Datamonitor- Economist Intelligence Unit (2010) ‘Industry Report: Consumer goods and retail January

2010 - Russia’ , London: EUI- Economist Intelligence Unit (2010) ‘Consumer goods and retail January 2010 – The

United States’ , London: EUI- Economist Intelligence Unit (2010) ‘Consumer goods and retail January 2010 – The

United Kingdom’ , London: EUI- Economist Intelligence Unit (2010) ‘Consumer goods and retail January 2010 -

Germany’ , London: EUI- Economist Intelligence Unit (2010) ‘Consumer goods and retail January 2010 - Japan’ ,

London: EUI- Economist Intelligence Unit (2010) ‘Country Profile – Russia’, London: The Economist

Intelligence Unit- Ehrenberg, A. (1988) Repeat-Buying (2nd Edition), Oxford: Oxford University Press

9

- Eskin, G. & Malec, J. (1976) ’A model for estimating sales potential prior to the testmarket’, American Marketing Association Educators’ Conference, Series No. 39

- Fader, P., Hardie, B., & Zeithammer, R. (2003) ‘Forecasting New Product Trial in aControlled Test Market Environment’, Journal of Forecasting, Vol.4, p. 391–410

- Fader, P. & Hardie, B. (2005) ‘The value of simple models in new product forecastingand customer-base analysis’, Applied stochastic models in business and industry, Vol.21,p. 461–473

- Fourt, L. & Woodlock, J. (1960) 'Early prediction of market success for new groceryproducts.' Journal of Marketing, 25, pp. 31–38

- Griffin, A. (1997) ‘PDMA research on new product development practices: updatingtrends and benchmarking best practices’, Journal of Product Innovation Management,Vol. 14, p.429

- Gundee, H. (1982) ‘Points to check when planning simulated test marketing study’,Marketing News, Vol. 15, Issue 17, p. 3

- Hart, S., Tzokas, N. & Saren, M. (1999) ‘The effectiveness of market information in- enhancing new product success rates’, European Journal of Innovation Management,

Vol. 2, No. 1, pp. 20–35- Hassan, S. (2008) ‘Bringing Lead-User Innovations to the Market: Research and

Management Implications’, SAM advanced management journal, Vol. 3- Henkel (2009) Annual Report 2009 [online] Available at:

http://www.henkel.com/com/content_data/165334_2010.02.25_FY_2009_annualreport_EN.pdf [Accessed 12.06.2010]

- Juster, F. (1966). Consumer Buying Intentions and Purchase Probability: AnExperiment in Survey Design, Cambridge: National Bureau of Economic Research

- Kachalov, I. (2008) Sales planning with accuracy exceeding 90%, Moscow: Piter Press- Kondratiev, N. (1993), Selected texts, Moscow: Russian Academy of Sciences- Kraft Foods (2009) Annual Report 2009 [online] Available at:

http://thomson.mobular.net/thomson/7/2956/3950/document_2/KFT_FactSheet10.pdf[Accessed 12.06.2010]

- Kratt, M. (2009) A History of the Supermarket in America [online] Available athttp://www.groceteria.com/about/history.html [Accessed 20.11.2009]

- Lilien, G., Kotler, P. & Moorthy K. (1992) Marketing Models, London: Prentice-HallInc.

- Lilien, G., Rangaswamy, A. & De Bruyn, A. (2007) Principles of MarketingEngineering, Bloomington: Trafford Publishing

- L’Oreal (2009) Annual Report 2009 [online] Available at: http://www.loreal-finance.com/eng/world-presence [Accessed 12.06.2010]

- Lin, L. (1986), ‘Estimating sales volume potential for new innovative products with casehistories’, Proceedings of the 36th ESOMAR Conference, Monte Carlo: ESOMAR, pp.159 – 174

- Lin, L. (1997), BASES New Product Sales Forecasting Model – A Collection of EarlierBASES & BASES Related Papers, Taichung: National Chung-Hsing University

- Mahajan, V. & Muller, E. (1998) ‘When Is It Worthwhile Targeting the Majority Insteadof the Innovators in a New Product Launch?’, Journal of Marketing Research, Vol.XXXV, p. 488-495

- Mahajan, V. & Wind, Y. (1988) ‘New Product Forecasting Models: Directions forResearch and Implementation’, International Journal of Forecasting, Vol. 4, p. 341-358

- Malhotra, N. (2007) Marketing Research: An Applied Orientation (5th Edition), NewJersey: Pearson Education Limited

10

- Markowitz, L. (2007) Concept Screening: Why Purchase Intent Is Losing Its Appeal:Reliance on Purchase Intent Can Lead to Lower Success Rates for Consumer PackagedGoods Products [online] Available at: http://www.ipsos.fr /InsideIpsos/attachment/2449-0-Ipsos_PI_LosingAppeal_POV-145.pdf [Accessed 12.10.2008]

- Markowitz, L. (2010) ‘The future of Forecasting is here: Did Simulated Test Marketsevolve as anticipated and what are the new expectations?’. Amserdam: ESOMARConference Proceedings

- McNeil, R. (2005) Business-to-Business Market Research: Understanding andmeasuring business markets, London: Kogan Page Limited

- Mentzer, J. & Moon, M. (2005) Sales Forecasting Management: A DemandManagement Approach (2nd Edition), London: SAGE Publications

- Morwitz, W., Steckel, J. & Gupta, A. (2007) ‘When do purchase intentions predictsales?’, International Journal of Forecasting, 23,pp. 347–364

- Nestle (2009) Annual Report 2009 [online] Available at: http://www.nestle.com/InvestorRelations/Reports/ManagementReports/2009.htm [Accessed 12.06.2010]

- Parameswaran, M. (2005) Building powerful brands using consumer research, Delhi:McGraw Hill Publishing

- Parfitt, J. & Collins, B. (1968) ‘Use of Consumer Panels for Brand Share Prediction’,Journal of Marketing Research, 5, pp.13-46.

- Peng, L. & Finn, A. (2008) ‘Concept testing: the state of contemporary practice’,Marketing Intelligence & Planning, Vol. 26, No. 6, pp. 649-674

- Pepsico (2009) Annual Report 2009 [online] Available at:http://www.pepsico.com/Investors/Annual-Reports.html [Accessed 12.06.2010]

- PriceWaterhouseCoopers (2006) 'From Beijing to Budapest (4th edition) – Russia’,London: PriceWaterhouseCoopers

- PriceWaterhouseCoopers (2010) 'Innovation by Large Companies in Russia:Mechanisms, barriers, perspectives’, Moscow: PriceWaterhouseCoopers

- Prince, M. (1992) ‘Choosing simulated test marketing systems’, Marketing Research,Vol. 4, Issue 3, p. 14-16

- P&G (2009) Annual Report 2009 [online] Available at:www.annualreport.pg.com/annualreport2009 [Accessed 12.06.2010]

- Reckitt Benckiser (2009) Annual Report 2009 [online] Available at:http://www.rb.com/Investors-media/Investor-information [Accessed 12.06.2010]

- Rosstat,Federal State Statistics Service (2010) Russia in figures [online] Available at:http://www.gks.ru/wps/PA_1_0_S5/Documents/jsp/Detail_default.jsp?category=1112178611292&elementId=1135075100641 [Accessed 04.07.2010]

- Schneider, J. (2004) New product launch: 10 proven strategies, Deerfield: StagnitoCommunications Inc.

- Schorsch, M. (2009) Market Entry Strategies for Russia: A comprehensive survey basedon expert interviews, Hamburg: Diplomica Verlag

- Shocker, A. & Hall, W. (1986) ‘Pretest Market Models: A Critical Evaluation’, Journalof Product Innovation Management, Vol. 3 ,Issue 2, pp86-107

- Sekaran, U. (2003) Research Methods for Business: A Skill-Building Approach (4th

Edition), NJ: John Wiley & Sons Inc.- Schlossberg, H. (1989) ‘Simulated vs. Traditional Test Marketing: the first one’s

improving while the second one is hardly dead’, Marketing News, Vol. 23, Issue 22, p.1-11

11

- The Coca-Cola Company (2009) Annual Report 2009 [online] Available at:http://www.thecoca-colacompany.com/ourcompany/ar/downloads.html [Accessed12.06.2010]

- Tidd, J. & Bessant, J. (2009) Managing Innovation (4th Edition) , Chichester: JohnWiley & Sons

- TNS (2009) Russia Ad Index 2009 [online] Available athttp://www.adindex.ru/rating/2009/marketing/company [Accessed 20.10.2009]

- Trott, P. (2008) Innovation management and new product development (4th Edition) ,Harlow: Pearson Education

- Trott, P. (2001) ‘The role of market research in the development of discontinuous newproducts’, European Journal of Innovation Management, Vol. 4, No 3, pp. 117-125

- Unilever (2009) Annual Report 2009 [online] Available at:http://annualreport09.unilever.com/ [Accessed 12.06.2010]

- Urban, G. & Hauser, J. (1993) Design and Marketing of New products (2d Edition),London: Prentice-Hall

- Urban, G., Hauser, J. & Dholakia, N. (1987) Essentials of New Product Management,NJ: Prentice Hall

- Urban, G. (1975) ‘Perceptor: A Model for Product Positioning’,Management Science,Vol. 21, No. 8, pp. 858-871.

- Urban, G. (1970) ‘Sprinter Mod III: A Model for the Analysis of New FrequentlyPurchased Consumer Products’, Operations Research, Vol. 18, No. 5., pp. 805-854.

- Urban, G. & Silk, A. (1978) ‘Pre-Test-Market Evaluation of New Packaged Goods: AModel and Measurement Methodology’, Journal of Marketing Research, Vol. 15, No. 2,pp. 171-191.

- Urban, G. & Katz, G. (1983) ‘Pre-Test-Market Models: Validation and ManagerialImplications’, Journal of Marketing Research, Vol. 20, No. 3, pp. 221-234.

- Usunier, J. (2000) Marketing across cultures (3rd Edition), Harlow: Pearson Education- Von Hippel, E. (2005) Democratizing innovation, Cambridge: MIT Press- Von Hippel, E. (1988) The sources of innovation, Oxford: Oxford University Press- Von Hippel, E., Thomke, S., & Sonnak, M. (1999) ‘Creating breakthroughs at 3M’,

Harvard Business Review, September-October, pp. 47-56.- Von Hippel, E. & Thomke, S. (2002) ‘Customers as innovators: A New Way to Create

Value’, Harvard Business Review, April, pp. 74-81.- Von Hippel, E. and Katz, R. (2002) ‘Shifting Innovation to Users via Toolkits’,

Management Science, Vol. 48, No. 7, pp. 821–833- Watkins, T. (1984) ‘Do STM models work?’, Journal Of The Market Research Society,

Vol. 26, Issue 3, pp. 255-256- Wherry, J. (2006) ‘Simulated Test Marketing: Its Evolution and Current State in the

Industry’, MIT Sloan Business School [Online]. Available athttp://dspace.mit.edu/bitstream/handle/1721.1/37225/85813336.pdf?sequence=1

- Willke, J. (2002) ‘The future of Simulated Test Markets: Obsolescence of current modelsand the characteristics of models of the future’. Amsterdam: ESOMAR ConferenceProceedings

- Wilson G. (1990) ‘Sometimes it's a sin to use a concept test model’, Marketing News,Vol. 24, Issue 14, p.20

12

APPENDICES

FMCG companies among Top-100 advertisers in RussiaFigure 1.9Tab. 1

Source: TNS (2009)

Rating Company Total ad budget ,mln $USD Rating Company Total ad budget ,

mln $USD

1 PROCTER & GAMBLE 301,59 36 BEIERSDORF AG (BDF) 40,352 L`OREAL 190,51 37 HEINEKEN 38,823 UNILEVER 144,11 38 LEBEDYANSKY 37,596 DANONE 123,92 46 DIROL CADBURY LLC 32,767 NESTLE 123,01 55 EFES BREWERY 27,018 MARS-RUSSIA 122,94 69 S.C.JOHNSON 22,97

10 RECKITT BENCKISER 116,62 70 TRANSMARK (SAB MILLER) 22,9611 HENKEL GROUP 111,91 75 ROLLTON 21,6512 COCA-COLA 106,72 76 ORIFLAME COSMETIC 20,6513 WIMM-BILL-DANN 96,42 77 KIMBERLY CLARK 19,6416 BALTICA 81,22 78 CAMPINA 19,4317 COLGATE-PALMOLIVE 76,70 79 DOUWE EGBERTS 19,0019 PEPSI CO 68,43 81 FABERLIC 18,7421 WRIGLEY`S 59,20 83 SCA HYGIENE PRODUCTS 18,3323 KRAFT FOODS 55,49 88 UNIMILK 16,0625 SUN INBEV 53,33 89 ORIMI TRADE 15,2027 NEFIS COSMETICS 51,85 90 HOCHLAND 15,0430 JOHNSON & JOHNSON 47,49 91 PEPSI LIPTON INTERNATIONAL 14,9231 KALINA 45,91 93 KAZANSKY FAT FACTORY 14,3132 FERRERO 44,82 100 UNITED CONECTIONERS 12,18

35 AVON BEAUTY PRODUCTSCOMPANY 41,11

53%

12%

35%

SURVEY PARTICIPANTS

NON-PARTICIPANTS:PROCTER & GAMBLE

NON-PARTICIPANTS:OTHER FMCGCOMPANIES

Sample profile (Base: N= 48 questionnaires)Figure 1.9Fig. 1

Number of questionnaires:Total returned (out of 72)..…........…...............................48Completed…….……..……………………………..………..33Incomplete (stopped at some point)………………..…….14Dropped from the analysis by respondent’s request..…..1

By company:

Companies ranked among Top-100 TV advertisers ...60%

Total number of companies surveyed…………….…...27Average number of respondents per company……...1.77

Television advertising expenditures in 2008(FMCG companies in the Top-100 group, see Figure 5.2):

SOV among the biggest FMCG TV advertisers...........53%

Category profile of surveyed companies (% of total) :

52%

21%

20%

13%

Food

Non-food (inc. tobacco)

Non alcoholic beverages

Alcoholic beverages

Experience:

Q3a. How long do you work in FMCG sector?Less than 2 years …..………........…...............................3%2-5 years…….……..……………………………..……….23%6-10 years………………………………………...……….45%More than 10 years…………………………………...….29%

Department / Position :

Q2. In which department do you currently work?Marketing research / Insights…..…...............................60%Marketing (Category or Brand management) …………20%Strategic planning and forecasting …………………….12%Marketing (NPD / Innovations )……………………...….4%Trade marketing / Sales……………………….……...….4%Q3. What is your role (position) in the department?Department director / Senior manager .........................36%Team/Working group manager …………………….……29%Specialist / Expert ………………………..……………….35%

13

New product launches by type (Base: N=47 questionnaires)Figure 1.9Fig. 2

Q5. Concerning these categories (or brands), howoften did the task of sales forecasting arise in thefollowing situations in your department in 2009 ?

N=870 responses(Answering to Q5)

Source: Griffin (1997)

New-to-the-Country or

World;

4%

New-to-the-Firm;

6%Line

Extensions,Brand

Stretching;

23%

Brandrelaunch /

Re-positioning;

4%

Product /Price change;

62%

Russia,FMCG,2009 (as reported by respondents): Western markets in the end of 90-ies:

New-to-the-Country or

World;

10%

New-to-the-Firm;

20%

LineExtensions,

BrandStretching;

23%

Brandrelaunch /

Re-positioning;

4%

Product /Price change;

43%

Types of new product launches according to resultsof the similar studies in the U.S. market

Marketing research tools in NPD (Base: N= 39 questionnaires)Figure 1.9Tab. 2

Q7.Which research techniques or information sources did you or your colleagues use while forecasting in each ofthe following business situations in 2009?

MARKET RESEARCH METHODS EMPLOYED IN THE RUSSIAN FMCG MARKET FOR NPD:

BASE:N=39 questionnaires

* ADJUSTED STANDARDIZED RESIDUALS, i.e. differencefrom expected average measured in Standard DeviationsShaded boxes indicate differences exceeding 2 Std.Dev.

BASE: N=418 responses on Q7 from 39 questionnairesSignificant at 5% error level (p<0.05, p=0.024 ,Chi-Square 23.465)

TOTAL(any type of new

product)

New-to-the-Country / World

/Company

Line Extensions /Brand Stretching

Changes in Marketing mix(Positioning / Product /

Price change

(% of Total Answering*)

Any expert analysis (time series based onavailable statistics, internal, external)

79% +0.1 -0.5 +0.4

Qualitative studies 72% -0.1 -0.2 +0.4

Quantitative diagnostic tests - w/o salesforecasting

64% -2.5 +0.4 +2.2

Simulated Test Marketing 62% +3.0 +0.5 -3.7

Quantitative tests with elements salesforecasting

55% +0.1 +0.3 -0.5

Strategic quantitative exploratory studies 47% -0.2 +0.3 -0.1

Traditional Test Marketing 42% +0.1 +0.7 -0.8

Deviation **

14

Factors of success in forecasting (Base: N=36 questionnaires)Figure 1.9Tab. 3

7a. From your experience, what are the factors that need to be taken into account when forecasting in the RussianFMCG market? /OPEN-ENDED/

FACTORS TO CONSIDER WHEN FORECASTING IN THE RUSSIAN FMCG MARKET

BASE: N=36 responses (Answering to Q7a)

Q7a (% from answering)

Reliable information about market size, trends, drivers of development, white spaces andsales opportunities 39%Business objectives, action standards (e.g. incremental sales, cannibalization etc) 22%Competitors and their likely response 19%Targeting - defining the audience and sourcing product categories 17%Similar cases, benchmarks, historical data 17%Forecasting approach (structure, components, inputs, outputs, limitations) 17%Various business scenarios 17%Consumer and shopper insights (barriers to adpotion, attractiveness of offer etc) 17%Business and Organizational insights (execution capbilities, sales infrastructure,distribution, retail, media etc) 14%Realistic marketing plans 11%Readiness of new product offer (communication, formula, marketing plans etc) 11%Proven method 11%Forecasting ROI, ratio between investments (time, efforts, budget) and project importance 11%Price context, price elasticity and dynamics 8%Monitoring performance during the launch / Validation / Calibration / Correction upon in-market results 8%Strategy and long-term vision 6%End - users (i.e. who and how will use the result) 6%Internal communication 6%New product type (new-to-the-firm, new-to-the-country etc) 3%Horizon of forecasting (short-, mid, long-term) 3%Using several alternative approaches to increase accuracy 3%

Difficulties to consider (Base: N = 36 questionnaires)Figure 1.9Tab. 4

7b. What are the difficulties that you usually face when forecasting sales for a new product? What issues requirefurther exploration? /OPEN-ENDED/

DIFFICULTIES AND ISSUES THAT NEED TO BE ADDRESSED

BASE: N=36 questionnaires

Q7b (% from answering)

Reliable, accurate and up-to-date information about the market (size, structure, trends andfactors of development, macro environment and sales infrastructure, e.g. sales channels)

43%

Tailoring western models to local market conditions - improving forecast accuracy (especiallyshort-term), speed of delivery, cost, simplification , developing alternative approaches

36%

Normative database / Developed evaluation criteria / Historical information / Validationsdatabase / Knowledge base for comparative analysis

29%

Effective methods of business and marketing planning in the local market (setting realistic goals,effective marketing approaches)

25%

Local media-model (relationship between ad investments and consumer response, detailing bychannel)

21%

Methods of forecasting in highly volatile markets 18%Education of internal clients (to overcome senior managers' lack of trust and avoid difficultieswith brand teams)

18%

Competitors - competitive environment, competitors response 14%Targeting - defining the market and target group 14%Consumer and Shopper insights in the local market 14%High cost and duration of forecasting, especially STM 14%Approaches to cannibalization and source of volume assessment 11%Manegerial and infrastructure aspects of NPD and launch in the local market (best practices ofNPD, infrastructure setup: distribution, media, retail etc)

11%

No problems if marketing research tools are used correctly 7%Methods of forecasting for absolutely new products, non existing markets 4%Accurate and clear description of modeling approaches ,their limitations and terms of application 4%Flexibility of the model (quick reruns of revised scenarios, adaptability etc ) 4%Methods of forecasting for incomplete of not final marketing mix at early stage of NPD 4%

15

100%

83%

70%

39%

25%

0% 50% 100% 150%

More than 0%

More than 10%

More than 25%

More than 50%

More than 75%

100%

95%

68%

23%

10%

0% 50% 100% 150%

More than 5%

More than 10%

More than 20%

More than 40%

More than 50%

Success rate of launches and accuracy of forecasting(Base: N=27 and N=29 questionnaires respectively)Figure 1.9Fig. 3

Q7d . In your experience, how many product launcheswould you consider successful

(i.e. actual sales were as planned or above) ?(% OF TOTAL ANSWERING, CUMULATIVE)

PERCEIVED SUCCESS RATE OF LAUNCHES:

BASE: N=27 q’res (N=36 excluding DK/NA),

Q7c. In your experience, what is the average error ofsales forecasting for a new product ?

(% OF TOTAL ANSWERING, CUMULATIVE)

BASE: N=29 q’res (N=36 excluding DK/NA),

PERCEIVED ERROR OF FORECASTING (NPD):

Hypotheses tested with Z-Test with correction for finite population , assuming same sample for Western markets – ( 70% vs. 50% ,Significant at 10%error level (p<0.01) 2-tailed Z-Score =1.66)

Med

ian

Med

ian

Familiarity with STM techniques (Base: N=36 questionnaires)Figure 1.9Fig. 4

Q9 . Which of the following “simulated test marketing”techniques have you ever heard of, at least by the

name? (% OF TOTAL ANSWERING)

AWARENESS OF STM MODELS

BASE: N=36 questionnaires

Q10. Which of the following “simulated test marketing”techniques have you or your colleagues have ever

used? (% OF TOTAL ANSWERING)

EXPERIENCE WITH STM MODELS

BASE: N=36 questionnaires

92%

86%

71%

90%

85%

54%

64%

58%

55%

40%

29%

28%

19%

14%

13%

5%

4%

4%

IPSOS DESIGNOR (ANY)

Designor Shelf, Desighor D'Light

NextGen, Innoscreen Forecast

ACNielsen BASES (ANY)

BASES I или BASES II

Snapshot или Pre-BASES

A/R/M/I Marketing - STM

GfK - MarketingLab / TESI

Comcon Sales Vision

TNS / RI - Microtest

MASMI - Simulated Test Market

TNS / RI - eValuate

Synovate - MarketQuest MVP

InVivo - MarketMind

TNS / RI - FYI

Other

Aegis Copernicus - Discovery

M/A/R/C - Assessor

65%

65%

21%

56%

56%

21%

19%

16%

15%

14%

9%

7%

6%

5%

4%

IPSOS DESIGNOR (ANY)

Designor Shelf, Desighor D'Light

NextGen, Innoscreen Forecast

BASES (ANY)

BASES I или BASES II

Snapshot или Pre-BASES

Comcon Sales Vision

TNS / RI - Microtest

A/R/M/I Marketing - STM

GfK - MarketingLab / TESI

TNS / RI - eValuate

MASMI - Simulated Test Market

InVivo - MarketMind

Other

Synovate - MarketQuest MVP

16

Willincrease;

65%

Will remainon the same

level;

25%

Will decline;

5%

Not going toconduct;

4%

STM – budgets, trends and barriers(Base: N=23, N=30, N=7 questionnaires respectively)Figure 1.9Fig. 5

Q14 . Concerning the last few STM studies, that you haveparticipated recently, would you please estimate the

average budget per study ?(% OF TOTAL ANSWERING, CUMULATIVE)

BUDGET PER STUDY STM MARKET TRENDQ15. Speaking about mid-term perspective (3-5 years), canyou please estimate the change in number of STM studies

conducted p/a with your participation? (% OF ANSWERING)

BASE: N=23 questionnaires

100%

93%

75%

64%

43%

25%

0% 50% 100% 150%

Above $ 20 000

Above $ 20 000

Above $ 30 000

Above $ 40 000

Above $ 50 000

Above $ 60 000

Med

ian

BASE: N=30 q’res

KEY REASONS FOR NOT-CONDUCTING STM

BASE: N=7 q’res

Q8. Why didn't you conduct STM in 2009? Q16. Why areyou not going to conduct STM in the next 3-5 years?(COUNT)The cost does not match its quality / Poor value for money /There are more cost efficient solutions.............................6Used simpler approaches……..…………………….………4Prefer traditional test marketing……..………………….......2Other……………………………..……..………………….……...1

Perceived accuracy of STM (Base: N=33 questionnaires)Figure 1.9Fig. 6

Q25 . From your personal experience, would youplease evaluate relative accuracy of <STM >forecasts

observed in the Russian market?(% OF ALL STM ASSESSMENTS)

PERCEIVED ACCURACY OF STM (RUSSIA, 2009):

BASE: N=72 responses to Q25 - by STM model type

PERCEIVED ACCURACY OF STM (U.S., early 90-ies):

Source: Baldinger (1988),Baldinger and Haley (1991)

STM Perceived Validity in the U.S. market (1988-1991)

Sales resultsare in line

with claimedglobal

accuracy;

46%

Sales arehigher or

lower thenpredicted

(belowclaimedglobal

accuracy) ;

54%

Sales resultsconfirmedthe STM;

52%

Sales arehigher or

lower thenpredicted;

48%

17

STM: Factors of choice (Base: N=30 questionnaires)Figure 1.9Tab. 5

Q19-21. Why some STM models can (or cannot) be recommended ? /OPEN-ENDED/

FACTORS OF CHOICE FOR SIMULATED TEST MARKETING IN THE RUSSIAN FMCG MARKET

BASE: N=30 questionnaires

Q19-21 (% fromanswering)

Reliable forecast, accurate in the Russian market 36%Level of expertise, quality service, professional consulting, individual approach 32%Simplicity of model 21%Previous experience 21%

Number of Validations / Benchmarks / Cases / Learnings - particularly, in thespecific FMCG category

18%

Cost 14%Rich diagnostics, a lot of parameters 11%Duration 11%Project importance, business risks 7%International recognition 7%Types of market and new product (i.e. saturated market, absolutely new product) 4%Ability to control the process 4%

Key Drivers of Choice – Ranking (Base: N=28 questionnaires)Figure 1.9Fig. 7

Q22 . You have mentioned that you are going to conductSTM in the next 3-5 years. Which of the following 5 (five)factors are more important for you, when you are makingyour decision about technique or agency? (% OF TOTAL)

KEY DRIVERS OF CHOICE – TOP 5 KEY DRIVERS OF CHOICE – TOP 1Q22. Which single factor out of these five is of a top

importance? (% OF TOTAL)

BASE: N=28 questionnaires BASE: N=28 questionnaires

88%

53%

53%

53%

53%

45%

44%

31%

22%

19%

17%

8%

6%

5%

4%

Highly accurate forecast

Suitable for Russian market

Professional presentation of findings

Positive previous experience

High quality of data collection

High speed of research

Individual approach for every project

A lot of diagnostic information

High quality service and project management

Recommended by management / protocols

Simple and easy-to-understand

Transparent and flexible pricing

Well-known agency

Affordable price

Use of modern technologies

53%

15%

14%

8%

8%

3%

Highly accurate forecast

Suitable for Russian market

Recommended by management or companyprotocols

Professional presentation of findings andrecommendations

Simple and easy-to-understand

Positive previous experience

18

Highly accurate forecast

Professional presentation offindings and recommendations

Recommended bymanagement or company

protocols

Positive previous experience

Well-known agency

Simple and easy-to-understand

Suitable for Russian market

A lot of diagnostic information

Publications in professionalliterature

High quality service andproject management

Individual approach for everyproject

Affordable price

Transparent and flexiblepricing

High speed of research

High quality of data collection

Use of modern technologies

ACNielsen BASES

Ipsos DESIGNOR

Local STMs

Other Western STMs

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8

--ax

is F

2 (

28.9

1 %

)-->

-- axis F1 (60.95 %) -->

Symmetric Plot (axes F1 and F2: 89.86 %)

Perception of various Simulated Test Marketing approaches(Base: N= 28 questionnaires)Figure 1.9Fig. 8

Q23. We would like to know your opinion about practical use of STM techniques. Would you please evaluate to whatextend you agree or disagree with each of the following statements regarding <STM> , using 5-point scale below

(Correspondence Analysis based on TOP2 scores)

PERCEIVED IMAGE OF STM APPROACHES / CORRESPONDENCE ANALYSIS – PERCEPTUAL MAP *

Correspondence analysis measures the distance between nominal variables on a map, where each variable isassociated with each other.

BASE: N=570 responses to Q23, Significant at 10% error level (p<0.1, p=0.064 ,Chi-Square 56.77)

Perception of various Simulated Test Marketing approaches(Base: N= 28 questionnaires)Figure 1.9Tab. 6

Q23. We would like to know your opinion about practical use of STM techniques. Would you please evaluate to whatextend you agree or disagree with each of the following statements regarding <STM> , using 5-point scale below

(Adjusted standardized residuals based on TOP2 scores)

PERCEIVED IMAGE OF STM APPROACHES / CORRESPONDENCE ANALYSIS – RESIDUALS *

BASE: N=570 responses to Q23, Significant at 10% error level (p<0.1, p=0.064 ,Chi-Square 56.77)* RESIDUALS ANALYSIS, is a descriptive technique designed to analyze two-way and multi-way tables measuring

correspondence between the rows and columns. Shows deviation between observed and expected (average) valuesmeasured in Standard Deviations

Shaded boxes indicate differences exceeding1 Std.Dev.

ACNielsen BASES Ipsos DESIGNOR Local STMsOther

Western STMs

Highly accurate forecast +0.5 -0.8 -0.6 -0.1

Professional presentation of findings and recommendations +0.7 +0.8 -0.8 -1.1

Recommended by management or company protocols +3.2 -1.3 -2.2 +0.9

Positive previous experience +0.8 +0.4 -1.0 -0.2

Well-known agency +1.4 +0.4 -1.4 -0.5

Simple and easy-to-understand -0.3 -1.8 +0.8 +2.0

Suitable for Russian market -1.1 -0.2 +1.3 -0.1

A lot of diagnostic information -1.3 +1.7 -0.1 -0.9

Publications in professional literature +0.1 -0.5 +0.2 +0.4

High quality service and project management +0.0 +0.4 -0.6 +0.1

Individual approach for every project +0.3 -0.1 +1.1 -1.6

Affordable price -1.8 -1.4 +3.1 +0.3

Transparent and flexible pricing -0.6 -0.7 +1.6 -0.4

High speed of research -2.1 +0.6 +1.4 -0.2

High quality of data collection -1.5 +0.4 +0.5 -0.2

Use of modern technologies -0.0 +0.9 -0.7 -0.4

19

Customer satisfaction and factors that influence it(Base: N= 28 questionnaires)Figure 1.9Fig. 9

24. For each STM model that you have ever used, would you please tell us, how satisfied were you with the quality ofservice provided by the agency. For your answers, please use 10-point scale, where 1 means “COMPLETELYDISSATISFIED” and 10 means “COMPLETELY SATISFIED”

CORRELATION BETWEEN IMAGE ATTRIBUTES AND CUSTOMER SATISFACTION

BASE: N=70 responses to Q24,** Correlation is significant at the 0.01 level (2-tailed)* Correlation is significant at the 0.05 level (2-tailed).

CustomerSatisfaction

(Mean 6.17,Std.Dev. 1.40)

Correlation

Individual approach for every project 0.333 **High quality service and project management 0.319 **Positive previous experience 0.301 **Professional presentation of findings andrecommendations

0.297 **

Well-known agency 0.275 *A lot of diagnostic information 0.222 *

Recommended by management or company protocols 0.187 *

Highly accurate forecast 0.175Simple and easy-to-understand 0.169Publications in professional literature 0.143Use of modern technologies 0.119Transparent and flexible pricing 0.103High speed of research 0.055High quality of data collection 0.053Suitable for Russian market 0.028Affordable price -0.059

Highly accurate forecast

Professional presentation offindings and

recommendations

Recommended bymanagement or company

protocols

Positive previous experience

Well-known agency

Simple and easy-to-understand

Suitable for Russian market

A lot of diagnostic information

Publications in professionalliterature

High quality service andproject management

Individual approach for everyproject

Affordable price

Transparent and flexiblepricing

High speed of research

High quality of data collection

Use of modern technologies

-0.10

-0.05

-

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

-3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00

Principalweaknesses

ACNielsen BASES: Perceived strengths and weaknesses(Base: N= 28 questionnaires)Figure 1.9Fig. 10

Cor

rela

tion

with

Sat

isfa

ctio

n**

Principalstrengths

Secondaryweaknesses

Level of Associations** Residuals, see Tab. 6

**Se

eFi

g.9

Secondarystrengths

20

Highly accurate forecast

Professional presentation offindings and

recommendations

Recommended bymanagement or company

protocols

Positive previous experienceWell-known agency

Simple and easy-to-understand

Suitable for Russian market

A lot of diagnostic information

Publications in professionalliterature

High quality service andproject managementIndividual approach for every

project

Affordable price

Transparent and flexiblepricing

High speed of research

High quality of data collection

Use of modern technologies

-0.10

-0.05

-

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

-2.00 -1.50 -1.00 -0.50 0.00 0.50 1.00 1.50 2.00

Principalweaknesses

Ipsos DESIGNOR: Perceived strengths and weaknesses(Base: N= 28 questionnaires)Figure 1.9Fig. 11

Cor

rela

tion

with

Sat

isfa

ctio

n**

Principalstrengths

Secondarystrengths

Level of Associations** Residuals, see Tab.6

** S

eeFi

g.9

Secondaryweaknesses

STM: Key areas for improvement (Base: N=33 questionnaires)Figure 1.9Tab. 7

Q17. How, in your opinion, STM services provided by local agencies can be improved? /OPEN-ENDED/

AREAS FOR IMPROVEMENT OF SIMULATED TEST MARKETING IN THE RUSSIAN FMCG MARKET

BASE: N=33 questionnaires

Q17 (% from answering)

More agency involvement into particular business issue / market analysis 33%

Gathering local benchmarks / evalution criteria / validations / learnings / historicalinformation

33%

Individual approach - customization to particular markets and particular projects 27%Model simplicity - more clear reports,concise and accurate set of terms 17%Media and marketing mix model. Local assessment of marketing impact on sales(ATL, BTL, distribution etc)

17%Timing (should be reduced) 13%Description of model, terms of application and limitations to use, requirements tomodel inputs, formats and use of results

13%

Flexibility - forecasting multiple scenarios, making adjustments 13%Success stories and cases obtained from the local market 13%Cost 13%Thoughtful application of global practices / guidelines 10%Qualification of agency specialists 10%Accuracy of forecast (including components such as inctremental sales,cannibalization etc)

7%Software (simulators) 3%Fieldwork quality 3%Approaches to targeting, i.e. market definition, source of volume 3%