nUSAera

Triadic preference testPlacebo pairsExtraneous factorsPlacebo test screening

as dativce telye r

under assessment. As an attempt to minimize the possibility of responses to extraneous factors when

s areg the lf prefe10; Re

ing/disliking for each product should be included, to give furtherinsight into the reasons for the measured preferences. For exam-ple: whether a consumer liked both products but happened to likeone more than the other or whether a consumer had a preferencebecause one product was liked while the other was disliked, etc.

t betweematically

to the properties of the products in the test that are of releit is a matter for concern.

The question becomes why consumers should responputatively identical pair of stimuli with a preference response. Itmay be assumed that the response was not the result of the assess-ment of the sensory properties of the putatively identical stimuli.The sensory input elicited by the attributes of the putatively iden-tical pair, would be close enough to identical to be deemed asbeing elicited by the same product. In which case, it would appearto be due to something else, which here will be called extraneous

Corresponding author. Address: Sensory Innovation Lab, Department of FoodScience and Technology, University of California, Davis, CA 95616, USA. Tel.: +1 530756 5493; fax: +1 530 756 7320.

E-mail address: maomahony@ucdavis.edu (M. OMahony).

Food Quality and Preference 39 (2015) 815

Contents lists availab

Food Quality an

l seRodriguez, OMahony, & Angulo, 2005; Angulo & OMahony,2005). Accompanying the preference test, questions regarding lik-

488 (total 1787), giving remarkable agreemenbrand. Because these preferences are not systehttp://dx.doi.org/10.1016/j.foodqual.2014.05.0160950-3293/ 2014 Elsevier Ltd. All rights reserved.n eachrelatedvance,

d to a& Sidel, 2004). With the No Preference option included, the dataobtained from such a test specify the proportion of consumerswho prefer each product and the proportion who have no prefer-ence, along with the sample size, to give an idea of the power ofthe test. Sometimes, a hedonic d0 is used as a single overall measureto indicate the strength of preference for one product over theother (Alfaro-Rodriguez, Angulo, & OMahony, 2007; Alfaro-

a variety of attributes like: better avor, easier draw, better after-taste, and slower burning. Finally, they were asked for their prefer-ences and 40% reported preference for cigarette X, 20% reportedNo Preference and 40% preference for Y. Yet, X had been takenfrom the same production run as Y; they were essentially thesame cigarette. This experiment was repeated for four differentbrands of cigarettes with consumer sample sizes ranging 412Introduction

Preference and acceptance testdevelopment and decisions regardinucts on to the market. A simple test oerence test (Lawless & Heymann, 20assessing the test pair, it has been suggested to only use those consumers who chose the No Preferenceoption for the placebo pair. However, this form of screening is not viable because the resulting screenedsample size is greatly reduced to approximately one third. However, in the present study, with the triadicpreference test, the resulting screened sample size ranged 76.594% of the total. Thus, this form ofscreening against consumers who demonstrated response to extraneous factors for the placebo pair,was now feasible.

2014 Elsevier Ltd. All rights reserved.

important for productaunching of new prod-rence is the paired pref-surreccion, 1998; Stone

As a casual test, the paired preference test is a simple tool forpreliminary experimentation. Yet, for more formal testing, thereare problems that need to be addressed. One such problem is a ten-dency to report preferences when the stimuli are putatively iden-tical. Ennis and Collins (1980) mailed two cigarettes (call them Xand Y) to a large number of consumers homes for comparison onKeywords:Preference testing

preference response. It is generally assumed that these consumers are responding to extraneous factors:those factors that elicit a preference response that are different from the sensory attributes of the foodThe triadic preference test

Eduardo Caldern a,b, Alondra Rivera-Quintero a, YixuaDepartment of Food Science and Technology, University of California, Davis, CA 95616,bUnidad de Investigacin y Desarrollo de Alimentos, Instituto Tecnolgico de Veracruz, V

a r t i c l e i n f o

Article history:Received 25 July 2013Received in revised form 7 May 2014Accepted 7 May 2014Available online 12 June 2014

a b s t r a c t

A triadic preference test win the test comprised a putthe regular paired preferenpairs of stimuli: a putativrequires one triad. With th

journal homepage: www.eXia a, Ofelia Angulo b, Michael OMahony a,

cruz 91860, Mexico

eveloped as an alternative to the paired preference test. The three stimuliely identical placebo pair and a different stimulus. This was in contrast toest that utilizes a placebo pair. Such a test requires the presentation of twoidentical placebo pair and a test pair. The triadic preference test onlyegular test, the majority of consumers respond to the placebo pair with a

le at ScienceDirect

d Preference

vier .com/locate / foodqual

y anfactors: those factors that elicit a preference response that are nota result of input from the sensory attributes of the foods underassessment. Such extraneous factors could have consequences forthe main preference tests that are being performed with the pairof different products that are of interest. This will be called thetest pair. If eighty percent of consumers can respond to extrane-ous factors when presented with the putatively identical pair, howmany of these consumers might respond only to extraneous factorswhen assessing the test pair? The problem is that there is no obvi-ous way of knowing. The potential for misinformation is not to beignored.

Ennis and Collinss (1980) 402040 frequency distribution forputatively identical stimuli does not seem to be general. For rea-sons as yet unresolved, other authors (Alfaro-Rodriguez, Angulo& OMahony, 2005, 2007, 2008; Alvarez-Coureaux, Aguilar,OMahony, & Angulo, 2010; Angulo, Okayama, Nakamura, Yuen,& OMahony, 2009; Chapman, Grace-Martin, & Lawless, 2006;Chapman & Lawless, 2005; Kim, Lee, OMahony, & Kim, 2008;Marchisano, Lim, Cho, & Suh, 2003; Sung, Lee, OMahony, & Kim,2011) found different frequencies, the numbers varying with theproducts being tested, the experimental conditions, the types ofconsumers tested and the types and numbers of response optionsallowed in the test. The frequencies vary a great deal but mostappear to be in the 2035% range. Yet, in nearly all cases, themajority of judges indicate preferences rather than no preference.

The safest approach at the present time would appear to be torequire consumers to assess the test (different) pair and also theputatively identical pair, to give a measure of the effect of theextraneous factors. This was adopted and required giving consum-ers two pairs of products to assess in a given test (Alfaro-Rodriguez, Angulo, & OMahony, 2007, 2008; Alvarez-Coureaux,Aguilar, OMahony, & Angulo, 2010; Kim, Lee, OMahony, & Kim,2008; Sung, Lee, OMahony, & Kim, 2011). The response frequen-cies elicited by the test pair of stimuli could be compared withthe responses frequencies elicited by the putatively identical pair,later called the identicality norm (Christensen, Ennis, Ennis, &Brockhoff, 2014) to determine whether they were signicantly dif-ferent. If they were, it could be concluded that the preferenceresponses elicited by the sensory input from the attributes of thetest pair, were not solely a response to extraneous factors in thetesting situation. Because the putatively identical pair was beingused in a way similar to a placebo in drug testing, it was calledthe placebo pair (Alfaro-Rodriguez, Angulo, & OMahony, 2007).

Two lines of research emerged from the potential problems ofextraneous factors present in the paired preference test, rstreported by Ennis and Collins (1980). The rst we will call statisti-cal. The safest approach, as mentioned above, was for consumers totaste both a placebo pair and a test pair of stimuli, so that compar-isons could be made between the two sets of response frequencies.Chi-squared comparisons were used to determine whether therewas a signicant preference indicated in the test pair. In thisway, consumers were their own control. There was some investiga-tion (Sung, Lee, OMahony, & Kim, 2011) regarding the use of therelated samples Bowker test (Bowker, 1948) in this analysis,although it was not adopted.

Comparison between preference tests using different measure-ment protocols (e.g. with or without the No Preference responseoption) is problematical when the data are represented as frequen-cies of response. Instead, it is convenient to use a hedonic variant ofthe fundamental measure, d0, derived from Thurstonian modeling(OMahony, Masuoka, & Ishii, 1994; OMahony & Rousseau, 2002;Lee and OMahony, 2004) and from Signal Detection Theory(Green & Swets, 1966; Macmillan & Creelman, 1991). Values of d0

E. Caldern et al. / Food Qualitwere used by Alfaro-Rodriguez, OMahony, and Angulo (2005) tocompare protocols which did and did not use No Preferenceoptions. For d0 values using tests without a response option, tablesfor the 2-AFC test (Ennis, 1993) can be used, while for tests with aresponse option, the computation for the 2-AC test is used (Braun,Rogeaux, Schneid, OMahony, & Rousseau, 2004). More impor-tantly, for tests with a No Preference option, values of d0 for thetest pair and the placebo pair have been compared to determinewhether they were signicantly different (Alfaro-Rodriguez,Angulo & OMahony, 2007; Sung, Lee, OMahony, & Kim, 2011).Recently, there has been a renewed interest in the statistics associ-ated with this computation (Christensen, Lee, & Brockhoff, 2012;Christensen et al., 2014; Ennis & Ennis, 2012a, 2012b; Jesionka,Rousseau, & Ennis, 2014).

The second line of research was based in experimental psychol-ogy and it is this line of research with which this paper is con-cerned. With this line of research it is important to keep in mindthe goal of preference testing. The preferences measured in sensorytests are called test preferences. The goal of a test preference isnot to predict preference behavior under test conditions but to pre-dict preference behavior under everyday real life conditions.These are called operational preferences (Wichchukit &OMahony, 2010).

Whereas the statistical line of research examined how better tointerpret the data from a test pair of products by statistical com-parison with the data from placebo pairs, the goal of the experi-mental psychology approach was different. It was to investigatethe variables involved with the preference responses elicited withthe placebo pair, with the long term goal of increasing the propor-tion of No Preference responses. The goal was to design a prefer-ence testing protocol which did not elicit responses based onextraneous factors. Freed of this tendency, it is reasonable tohypothesize that consumers would give a more valid picture ofconsumer preference. Test preference would be better at predict-ing operational preferences.

The variables associatedwith the various experimental protocolsthat tended tovary theproportionof NoPreference responses to theplacebo pair were investigated. For example, Marchisano et al.(2003) noted that ifmore than one level of No Preferencewas avail-able as a response, a larger proportion of No Preferences was elic-ited. They also noted that Koreans were particularly reluctant togive No Preference responses. Angulo, Okayama, Nakamura, Yuen,and OMahony (2009) and Sung, Lee, OMahony, and Kim (2011)noted that buying preferences induced more No Preferenceresponses than liking preferences. Alfaro-Rodriguez, Angulo, andOMahony (2008) found that the tendency to give No Preferenceresponses to the placebo pairwas not stable; sometimes a consumerhad it while at other times it might be absent.

Yet, none of these studies solved the problem of how to elimi-nate the possibility of response to extraneous factors in the testpair. Alfaro-Rodriguez et al. (2007) and Sung et al. (2011)approached the problem by not only considering the data fromall the consumers tested but also by examining the response fre-quencies in the test pair from the sub-set of consumers, who hadreported No Preference with the placebo pair. The advantage herewas that only the responses of consumers, who had demonstrateda tendency not to report preferences elicited by extraneous factors,were considered. In other words, the placebo pair acted as a formof screening tool by not selecting consumers who had demon-strated a tendency to report preferences elicited by extraneous fac-tors. Naturally, it was not expected that giving a No Preferenceresponse to a placebo pair would be a stable attribute of a con-sumer; it would be expected to come and go. The disadvantageof this approach was the small number of No Preferenceresponses to the placebo pair. The sample size of these screenedconsumers was reduced, often to around one third of the total. This

d Preference 39 (2015) 815 9is unacceptably small and renders this form of screening impracti-cal. Yet, if a high proportion of consumers passed the placeboscreening tool, this approach would be feasible.

y anThere is a second point. Even though a consumer might exhibita preference induced by extraneous factors while assessing thestimuli in the placebo pair, the sensory characteristics of the targetstimuli might be sufciently different to entice consumers to payattention to the sensory properties of the target stimuli and ignoreany extraneous factors. In other words, real preferences might beelicited when these consumers were confronted with the test pair.Therefore, only using data from the small sample of consumerswho had chosen the No Preference option for the placebo pair,could result in the loss of potentially many consumers, who wouldnow be responding to the sensory properties of the stimuli in thetest pair. These consumers would have provided useful data(Sung et al., 2011). Thus, using the placebo pair as a screening toolcan be viewed as cautious and perhaps too conservative. However,should the proportion of consumers passing the placebo testscreening be large, this effect would not be so serious; potentiallyonly a minimal amount of data would be lost.

The problem is that consumers have a predisposition to thinkthat a preference response must be given in a preference test. Itmay be hypothesized that the predisposition is triggered by thevery appearance and structure of the situation in which a prefer-ence is requested. This could be because the situation had oftenbeen encountered casually in everyday life. The problem is tochange the appearance and structure of a preference test, so thatit no longer appears like the familiar task of reporting a preferencefor one of a pair of stimuli. If the appearance and structure of thistask were to be changed, it may be hypothesized that the predispo-sition may be abandoned and the consumer may be comfortablegiving a No Preference response for the placebo pair.

Accordingly, it is hypothesized that instead of giving the usualfamiliar pair of stimuli in a preference test, whose appearancemightelicit the presupposition that a preference is required, giving threestimuli might be sufciently different to change the conditions thatelicit the presupposition. Such a triadic testwould contain a placebopair with the third product being different. Should this approach besuccessful, itwouldhave theadvantage that testpair and theplacebopair were being assessed as one test rather than two.

The goal of these experiments was to give triadic preferencetests to consumers to determine whether the changed circum-stances of the test elicited a high proportion of No Preferenceresponses to the embedded placebo pair. Should the proportionbe sufciently high, it would allow the screening task to be a fea-sible tool for eliminating consumers with a tendency to respondto extraneous factors. As a control, performance was compared tothe proportion of consumers who passed the screening test whenapplied to what will be called the regular test, having a placeboand a test pair. A further goal was to investigate the importanceof using the correct instructions. Instead of performing these testsunder controlled laboratory conditions, they were performed in anarea set up in a university dining room, to get a little closer to theconditions of ordinary dining.

Experiment 1

The goal of this experiment was to perform triadic preferencetests to determine whether the proportion of consumers whoresponded to the placebo pair with the No Preference option,was sufciently large for it to be used as a screening tool. Regulartests with separate placebo and test pairs were used as acomparison.

Materials and methods

10 E. Caldern et al. / Food QualitConsumersConsumers of fruit juice and fruit juice cocktails (N = 200, 99 M,

101F, age range 1845 yrs) were selected. They were students andstaff at the University of California, Davis. They were intercepted ina dining room on the campus. They were tested in an accessiblearea, by the side of the dining room, set up for the experiment.

StimuliThe stimuli were Cranberry Juice Cocktail (20% Juice, Langer

Juice Company Inc., City of Industry, CA) and Pomegranate JuiceCocktail (20% Juice, Langer Juice Company Inc., City of Industry,CA). The juice samples were served at room temperature (1820 C) in 2 oz (59 mL) black plastic cups (Fabri-Kal Portion Cups,Kalamazoo, MI).

ProcedureConsumers were tested individually. After establishing rapport

and collecting demographic details, the experimenter instructedthe consumers in the experimental procedure and only began theexperiment when the consumers had thoroughly understood theirtask. The experimenter observed the consumers unobtrusively, toensure that they followed the instructions correctly and to beavailable for answering any questions and recording any of theconsumers comments.

Consumers performed two preference tests. One test was a reg-ular paired preference test using a No Preference option, with atest pair and a placebo pair. The other test was a triadic preferencetest.

For the regular paired preference test, the placebo pair con-sisted of two putatively identical samples. For half the consumers,they were cranberry juice while for the other half, they werepomegranate juice. For the test pair, one sample was cranberryjuice while the other was pomegranate juice. These were presentedin order counterbalanced over all the consumers. However, to givea closer correspondence with the triadic preference test, the pla-cebo pair was always presented before the test pair.

The experimenter told the consumers that they were going toperform a set of preference tests. On presenting the placebo pairof juices to be assessed, the consumers were told by the experi-menter, who indicated each sample by pointing, that they mightprefer the sample on the left, they might prefer the sample onthe right or they might have no preference. They tasted and swal-lowed each stimulus in the pair and gave their response verbally.After this, the test pair was presented; the instructions wererepeated and the consumer tasted the two samples and respondedin the same way.

After a short break, the consumer performed the triadic prefer-ence test. The consumers were told that they were being giventhree products and to report howmuch they liked them. They weretold their response options. They were told that they might liketwo of the products (the placebo pair) roughly about the sameand like the third product more or like it less. Either of theseresponses would be seen as appropriate for passing the screening,as long as it was the placebo pair that was liked the same. This wasalways checked by the experimenter. They were also told that theymight like all three products roughly about the same. Again, thisresponse is appropriate because the placebo pair would be selectedas being liked roughly about the same. The nal response optionwas liking all three products differently. This response would beinappropriate for them to pass the screening, because it impliesa preference for the placebo pair. As a reminder, these four possibleresponses were presented on a sheet of paper within the con-sumers view. The order of presentation of the three stimuli inthe triad was counterbalanced so that all six orders of presentationfor the triad were used and spread equally among the consumers.

Before both the regular and triadic preference tests, no inter-

d Preference 39 (2015) 815stimulus water rinses were used, but a primer was given. Thisensured that when the rst stimulus in the test was tasted, themouth was in a similar condition as it was for the other stimuli.

This helped control adaptation effects. For half the consumers, theprimer was pomegranate juice while for the other half it wascranberry juice. Stimuli were presented at constant room temper-ature (1820 C). Session lengths ranged 58 min.

Results

The results of Experiment are given in Table 1. The data for theregular paired preference test are given above the data for the tri-adic preference test. For the regular paired preference test, the rstline of data gives the results for the total sample of consumers

Experiment 2

For the unspecied tetrad test, it is important to give the correctinstructions. Delwiche and OMahony (1995) demonstrated thisexperimentally. They used confusable chocolate pudding stimuliof different sweetness and gave two sets of instructions to theirjudges, one set correct and the other set incorrect. For the correctinstructions, judges were told to sort the four stimuli into two sep-arate pairs, the stimuli within each pair being identical. For thewrong instructions they were told to nd the two most similarsamples. Signicantly lower d0 values were obtained using thewrong instructions. These lower values were comparable to values

d pl

No preference Prefe

32

16(54.5%) (5.0%) (32.5%)

E. Caldern et al. / Food Quality and Preference 39 (2015) 815 11Regular Total (200) 68(34%)

1

Screened (68) 68(34%)

Triadic Total (200) 184(92%)(N = 200). It can be seen that for the placebo pair, 68 respondedwith the No Preference option, thus passing the placebo screeningtest, while 132 failed by responding with a preference. The lastthree columns give their responses (N = 200) to the test pair andit can be seen that there was a greater preference for pomegranatejuice than cranberry juice. The second line gives the preferences foronly those 68 consumers who responded with no preference forthe placebo pair. The last three columns indicate that once againthe pomegranate juice was preferred.

The third line of the data indicates that 184/200 consumerschose the No Preference response for the placebo pair and conse-quently passed the placebo screening. The remaining 16 liked eachjuice differently, thus, indicating a preference for the placebo pairand were eliminated from the test; their preferences were notrecorded. Again the last three columns indicate that for the 184consumers who passed the screening test, the pomegranate juicewas preferred to the cranberry juice. These columns give the num-ber of consumers who chose each response option while the per-centage given below were percentages based on the total 200consumers. The fourth line gives the same data except the percent-ages were based on the 184 consumers who passed the placeboscreening.

The proportion of consumers that were screened by reportingNo Preference for the placebo pairs, was considerably greaterusing the triadic protocol. The high percentage of consumers(92%) made this form of screening feasible for such tests. Consid-ering the test pair, it can be seen that in all cases, pomegranatejuice was preferred to cranberry juice. Yet, the degree of prefer-ence for pomegranate over cranberry was non-signicantly stron-ger (binomial, pP 0.12) using the triadic protocol than using theregular test (ratio of triadic consumers choosing pomegranate/cranberry = 1.68; ratio for regular preference test = 1.17, all 200consumers; 1.21, 68 screened consumers). As mentioned above,for the regular and triadic tests, the preference for pomegranatejuice was the same, only the effect was stronger with the triadictest. Also, the larger sample of screened consumers lends greaterpower to the triadic protocol, which also has the convenience ofcombining both test and placebo procedures into a singleprotocol.

Table 1Frequencies of consumers reporting preference and no preference responses for test anthe triadic protocol.

Experimental protocol Screened or total Placebo PairScreened (184) 184(92%)for the triangle test, which had also been performed by the judges.These results were re-iterated recently, using simulations, by Rous-seau and Ennis (2013).

In the same spirit, an experiment was performed to demon-strate how important it was to give the correct instructions forthe triadic preference test. Two sets of stimuli were used for thisexperiment: diluted and undiluted orange juice, and diluted andundiluted apple juice.

Materials and methods

Part AConsumers. Consumers of orange juice (N = 200, 100 M, 100F, agerange 1846 yrs) were selected. They were students and staff atthe University of California, Davis. They were intercepted in a din-ing room on the campus. They were tested in an accessible area, bythe side of the dining room, set up for the experiment.

Stimuli. The stimuli used were orange juice (100% juice, BayviewFarms Inc., Lathrop, CA) and the same orange juice diluted withpuried water to 60% strength. The water was puried by a PurelabPrima Reverse Osmosis System (ELGA LabWater, High Wycombe,Bucks, England) in series with a Milli-Q Advantage A10 system,using ion exchange and activated charcoal (Millipore Corp., Bed-ford, MA), yielding water with conductivity < 106 mho/cm, withTOC 6 5 ppb. The juice samples were served at constant room tem-perature (1820 C) in 2 oz (59 mL) black plastic cups (Fabri-KalPortion Cups, Kalamazoo, MI).

Procedure. The experimental location and the procedure were thesame as for Experiment 1. Except that different stimuli were used.

Consumers were presented with three cups. In two of the cupsthe stimuli were putatively identical while in the third cup a differ-ent stimulus was presented. For half the consumers the identicalpair consisted of the undiluted juice and the odd stimulus wasthe dilute juice. For the other half, the reverse was the case.

The instructions were as for Experiment 1. Before proceedingwith the test, the consumer tasted and swallowed the more dilute

acebo pairs, either presented separately using the regular protocol or embedded using

Test pair

rence Pomegranate juice No preference Cranberry juice

102(51%)

11(6%)

87(43%)

35(51.5%)

4(5.9%)

29(42.6%)

109 10 65109(59.2%)

10(5.4%)

65(35.3%)

juice as a primer. The consumer tasted and swallowed the threestimuli in the triad and gave her response. No water interstimulusrinses were taken during the test.

The order of presentation of the three stimuli in the triad wascounterbalanced so that all six orders of presentation for the triadwere used and spread equally among the consumers. For half thejudges, the undiluted stimulus furnished the putatively identicalpair, while for the other half it was the dilute stimulus. The orangejuice stimuli were presented at constant room temperature (1820 C). Session lengths ranged 12 min.

tetrad test, the effect would occur so rarely as to be unimportant.

the University of California, Davis. Again, they were intercepted

also told they might like one more or less than the other two.

more than the other two, while not liking the other two equally.

juice stimuli instead of orange juice.

Stimuli. The stimuli were apple juice (100% juice, Safeway KitchensApple Juice, Safeway Inc., Pleasanton, CA) and the same apple juicediluted with puried water, giving a strength of 25%. The puriedwater was the same as that used in Part A and the juice sampleswere also served as in Part A.

Procedure. The procedure was the same as Part A except that theapple juice stimuli were used instead of orange juice stimuli.

Part DThis experiment was the same as Part B except that apple juice

stimuli were used instead of orange juice stimuli. Again, the

ing

ree

12 E. Caldern et al. / Food Quality and Preference 39 (2015) 815Table 2Frequencies of the different preference options for undiluted and dilute orange juice, usA and B, respectively.

Odd sample in triad I like all three juices roughly about the same I like all th

Full instructions (N = 200), Part AUndiluted 1 29Dilute 3 18Totals 4 47Percentages (%) 2 23.5

Partial instructions (N = 250), Part BUndiluted 0 29Dilute 1 10Consumers. Consumers of apple juice (N = 200, 102 M, 98F, agerange 1859 yrs) were selected. They were students and staff atthe University of California, Davis. They were tested as in Part A.By comparing the data in this experiment with the data in PartA, some evidence for the latter case occurring might be obtained.

Part CThis part of the experiment was repetition of Part A, using appleThe instructions in Part A, about saying the other two being likedthe same, was omitted. Thus, logically it would be possible for con-sumers to pass the screening by reporting that they liked onein a dining room on the campus and were tested in an accessiblearea, by the side of the dining room, set up for the experiment.

Stimuli. The same orange juice stimuli used in Part A were used inthis experiment.

Procedure. Consumers were tested in the same way as for Part A,with one exception. The instructions were the same as in Part Aexcept one item was left out. As before, the consumers were toldthat they were going to perform a preference test and were toldthat they would taste three stimuli to see whether they liked themor not. They were told that they might like them roughly about thesame or they might like them all to a different degree. They wereConsumers. Consumers of orange juice (N = 250, 122 M, 127F, agerange 1858 yrs) were selected. They were students and staff atPart BThis experiment was the same as Part A except that only partial

instructions were given. The experiment investigated the effect ofusing partial instructions. It was hypothesized that unlike with theTotal 1 39Percentages (%) 0.4 15.6hypothesis was that the effect of incomplete instructions wouldbe unimportant.

Consumers. Consumers of apple juice (N = 200, 100 M, 100F, agerange 1757 yrs) were selected. They were students and staff atthe University of California, Davis. They were tested as in part B.

Stimuli. The same apple juice stimuli were used as in Part C.

Procedure. The procedure was the same as that used in Part B,except that apple juice stimuli were used instead of orange juicestimuli.

Results

Table 2 indicates the number of times a given response optionwas chosen by the consumers for the orange juice stimuli testedin Parts A (full instructions) and B (partial instructions). The sam-ple sizes varying according to availability. There are four rows ofnumbers for each part. The upper four rows are for Part A andthe lower for Part B. For the upper rows of numbers, the undilutedjuice was the odd sample in the triad. In the second row, the dilutejuice was the odd sample, while the third row comprised the totalsand the fourth row, the percentages. Table 3 indicates the numberof times a given response option was chosen by the consumers forthe apple juice stimuli tasted in Parts C and D. It is organized in thesame manner as Table 2.

From Tables 2 and 3, considering the case where the fullinstructions were given, the percentage of consumers expressinga preference for the placebo pair embedded in the triad was23.5% and 15%. Comparing with these values with the 6580%reported in the literature, it can be concluded that the triadic pref-erence test elicits a far higher proportion of consumers who givethe No Preference response to a placebo pair. The values for thepartial instructions tell the same story, but as discussed below,their values are partially artifactual.

With the full instructions, the response that a consumer maylike two products just about the same and the third product either

the triadic preference test, with the full and partial instructions in Experiment 1, Parts

juices differently I prefer the undiluted juice I prefer the dilute juice Total

59 11 10067 12 100126 23 20063 11.5 100

88 9 12688 25 124

176 34 25070.4 13.6 100

ng th

ree

y anmore or less, was stated explicitly in the instructions. Such aresponse, along with liking all the products just about the same,was deemed to indicate the correct response to the placebo pairembedded in the triad. However, with the partial instructions, thiscondition was not stated explicitly; it was assumed. Accordingly,this condition includes the possibility of liking all three juices dif-ferently, which would result in failure to give an appropriateresponse to the placebo pair embedded in the triad. It was hypoth-esized that this latter condition would occur so rarely as to beunimportant.

The data from Tables 2 and 3 rejects this hypothesis. Comparedwith the full instructions, the percentage of consumers respondingthat they like all three juices differently, with the partial instruc-tions, was noticeably smaller, yet not signicantly so (binomialcomparison of proportions, p = 0.095, 0.071, resp.). In other words,the percentage of consumers who were accepted as giving theappropriate response to the placebo pair in the triad, was notice-ably larger. Therefore, a number of consumers who reported thatthey liked/disliked one stimulus more than the other two, couldactually have liked all three stimuli to a different degree. Thiscan be seen in the higher total percentages recorded for likingone stimulus more than the other two. It would not have beenapparent that they had given an inappropriate response to the pla-cebo pair embedded in the triad. Therefore, the option of havingconsumers say that two juices were liked just about the samewhile the third was liked either more or less, is vital for thismethod. Furthermore, it can also be seen from Tables 2 and 3 thatwith the full instructions, the degree of preference for undilutedjuice over the diluted juice is comparable, when the odd sampleis undiluted or dilute. With the partial instructions, this consis-tency was lost. For the above reasons, it was concluded that it isimportant to give the full instructions.

Table 3Frequencies of the different preference options for undiluted and dilute apple juice, usiand D, respectively.

Odd sample in triad I like all three juices roughly about the same I like all th

Full instructions (N = 200), Part CUndiluted 1 17Dilute 9 13Totals 10 30Percentages (%) 5 15

Partial instructions (N = 200), Part DUndiluted 5 9Dilute 3 3Total 8 12Percentages (%) 4 6

E. Caldern et al. / Food QualitDiscussion

Various reasons for expressing preferences when assessing theplacebo pair, as well as the relationship between test preferencesand operational preferences have been discussed (Marchisanoet al., 2003; Xia, Rivera-Quintero, Calderon, Zhong and OMahony(2014). It is worth discussing some of these here.

Marchisano et al. (2003) after testing, asked their consumerswhy they had expressed a preference when assessing putativelyidentical stimuli in a placebo pair. Some American consumers indi-cated that they felt they ought to have a preference even thoughthe stimuli taste practically the same; they felt it was weak notto have a preference. Many Korean consumers felt sorry if theydid not have a preference because they felt they were not beinghelpful. Another consumer reported that it was that it felt likebeing asked by a polite friend to have rst choice of one of twoidentical coffees he had bought, so the friend would take the oneleft over. This response is discussed in more detail by Ullman-Margalit and Morganbesser (1977) in terms of the differencebetween picking and choosing.

A common explanation is that the stimuli are perceived as dif-ferent because of the varying effects of neural or oral noise in thesensory system. The application of Thurstonian modeling to differ-ence testing is based on such ideas. Thus, consumers may detect ormay imagine they detect a very slight difference and use this onwhich to base their preference (Chapman, Lovelace, Cardello, &Lawless, 2010). This is an example of what Kahneman (2011) callsfast thinking. Such test preferences would not be expected to pre-dict operational preferences. It is well known that operationally,consumers can detect small variations among samples of the sameproduct without initiating a preference, for example: fruit. Differ-ences generally have to be relatively large to initiate a preferenceor rejection. The rejection threshold is higher than the detectionthreshold (Prescott, Norris, Kunst, & Kim, 2005) althoughHarwood, Ziegler, and Hayes (2012) produced contrary results.But these might have been due to their use of 2-AFC protocolsfor preference and less powerful triangles for detection.

This argument can be expressed in various ways. For preferencetests, the consumers should be assessing whether the stimuli aredifferent products, not whether there are slight differences in theirsensory characteristics, as with difference testing. Another way ofexpressing this is that preference tests require a longer s-criterionthan difference tests (Sung et al., 2011).

Perhaps the most common explanation for preferenceresponses being given when consumers assess the placebo pair,is described in many ways. There is a strong predisposition to givepreference responses. There is a response bias towards giving apreference. Yet, it can be argued that such an explanation is merelya renaming of the behavior and not a true explanation. Why doconsumers respond to a placebo pair with a preference? Becauseof response bias. Dene response bias in this situation. It is

e triadic preference test, with the full and partial instructions in Experiment 1, Parts C

juices differently I prefer the undiluted juice I prefer the dilute juice Total

69 13 10062 16 100131 29 20065.5 14.5 100

65 21 10052 42 100117 63 20058.5 31.5 100

d Preference 39 (2015) 815 13responding to the placebo pair with preference.None of these explanations of why consumers have a strong

tendency to respond to a placebo pair with a preference, gaveany hints as to how the effect can be eliminated or at least reduced.Accordingly, Xia et al. (2014) used a different approach and deviseda protocol that reversed the hidden demand characteristics of thepaired preference test. This was achieved by telling the consumersthat she knew that they would like the stimuli in the placebo pairjust about the same, because everybody does. The result was that8090% of consumers passed the screening, they responded tothe placebo pair with a No Preference response. However, whenconfronted with the test pair, consumers readily had preferences.Prior to presentation of a placebo pair, Chapman et al. (2010) toldtheir consumers that the stimuli would be identical 50% of thetime. Yet, this had no effect on the proportion of preferences thatwere reported.

Xia et al. (2014) considered what is known about cognitive pro-cessing in the brain. Conscious thinking which requires active

studies (two in one paper) have been published that test the valid-

y anengagement, Kahnemans (2011) slow thinking, can be seen as thedomain of central processing. Yet, to protect the central processorfrom what could be called information overload, the brain hasseveral strategies. Such strategies regarding sensory input areselective attention and sensory adaptation. The brain also appearsto have protective output strategies. Certain motor behaviors liketying a necktie, riding a bike, selecting the right foot and handmovements when changing gear in a car, require conscious atten-tion at rst. With practice they become automatic. To draw ananalogy, they can be seen as having been relegated from centralprocessing to a motor subroutine. To extend the analogy further,there can be cognitive subroutines. An example is the choice ofwords when learning a second language. At rst it requires con-scious attention but later it becomes automatic. It can be arguedthat another example of a cognitive subroutine is Kahnemans fastthinking. For example, consider the response to the question that ashepherd has 27 sheep, all but 9 die, how many are left? Peoplewill tend to subtract 9 from 27 (=16) because that is what theyhave nearly always done before, when set similar problems. Theyreach a conclusion without properly engaging cognitively, withoutpaying sufcient attention to the task. It can loosely be describedas working on auto pilot. The important property here is thehabitual or stereotypic nature of this behavior. In the same way,it can be hypothesized that because of preference questions beingoften asked in everyday life, consumers develop a predisposition toanswer automatically with a preference response, without muchconscious engagement. Accordingly, they will respond in this pre-disposed manner to a preference test, even with a placebo pair.Motivationally, such behavior is described as responding to hiddendemand characteristics. To highlight its attentional aspects, it canalso be described as an example of fast thinking: responding with-out paying sufcient attention to the task. The problem for prefer-ence testing becomes one of how to change a consumerssubroutine-fast thinking to central processing-slow thinking. Con-sidering the difculty that is encountered in teaching a person howto tie a necktie, essentially trying to use conscious central process-ing for a task that has been relegated to a motor subroutine, itwould appear to be difcult.

As luck would have it, Neal, Wood, Wu and Kurlanders (2011)popcorn experiment indicated that it was comparatively simple.The experiment provided an example of how a seemingly trivialchange in an experimental protocol, like changing to using onessub-dominant hand, can change a consumers automatic popcorneating behavior. Subjects had been eating popcorn without payingsufcient attention to the avor, whereby stale popcorn was eatenas much as fresh popcorn. This could be described as eating in afast thinking mode. Changing to eating with the sub-dominanthand, elicited rejection of the stale popcorn. This could bedescribed as changing to eating in a slow thinking mode, wherebygreater attention was paid to the popcorns sensory properties.Thus, acting on auto pilot was changed to active consciousengagement by a small change in experimental conditions. Thusfor a preference test, a small change, like unusual instructionsand questions, might be different enough to separate consumersfrom the usual testing conditions and cause a switch from auto-matic fast thinking to more consciously slow thinking. Xia et al.(2014) were not the rst sensory scientists to use Kahnemans(2011) ideas. Ares et al. (2014) used his approach in the consider-ation of how to motivate consumers to cognitively engage and payfull attention to the check-all-that-apply method.

Xia et al. (2014) used a small change in the instructions toreverse the response bias and caused the consumers to use centralprocessing rather than acting on auto pilot. Consumers actively

14 E. Caldern et al. / Food Qualitengaged the sensory characteristics of the products and respondedthat they had no preference with the placebo pair. In the same way,the present authors used the same central processing/subroutineity of the 9-point hedonic scale. Peryam, Polemis, Kamen,Eindhoven, and Pilgrim (1960) correlated the choice behavior ofsoldiers in their canteens with scores on the 9-point hedonic scaleover a one-month period. Rosas-Nexticapa, Angulo, and OMahony(2005) monitored consumers buying behavior for a year after test-ing them with hedonic and purchase intent scales. No such validitystudies have yet been published for preference tests.

References

Alfaro-Rodriguez, H., Angulo, O., & OMahony, M. (2007). Be your own placebo: Adouble paired preference test approach for establishing expected frequencies.Food Quality and Preference, 18, 353361.

Alfaro-Rodriguez, H., Angulo, O., & OMahony, M. (2008). Paired preference tests:50:50 and alternating no preferences. Journal of Sensory Studies, 23, 765779.

Alfaro-Rodriguez, H., OMahony, M., & Angulo, O. (2005). Paired preference tests: d0

Values from Mexican consumers with various response options. Journal ofSensory Studies, 20, 275281.

Alvarez-Coureaux, Y., Aguilar, P., OMahony, M., & Angulo, O. (2010). Assessment ofpreference with controls for response bias operating in the test situation: Apractical example using omega-3 enriched wholegrain breads with Ecuadorianconsumers. Journal of Sensory Studies, 25, 659671.

Angulo, O., Okayama, T., Nakamura, T., Yuen, R., & OMahony, M. (2009). Use ofpurchase preference options to increase no preference frequencies in placebopreference tests. Journal of Sensory Studies, 24, 258268.

Angulo, O., & OMahony, M. (2005). The paired preference test and the NoPreference option: Was Odesky correct? Food Quality and Preference, 16,425434.

Ares, G., Etchemendy, E., Antnez, L., Vidal, L., Gimnez, A., & Jaeger, S. R. (2014).Visual attention by consumers to check-all-that-apply questions: Insights tosupport methodological development. Food Quality and Preference, 32, 210220.

Bowker, A. H. (1948). A test for symmetry in contingency tables. Journal of theAmerican Statistical Association, 43, 572574.

Braun, V., Rogeaux, N., Schneid, N., OMahony, M., & Rousseau, B. (2004).Corroborating the 2-AFC and 2-AC Thurstonian models using both a modelsystem and sparkling water. Food Quality and Preference, 15, 501506.

Chapman, K. W., Grace-Martin, K., & Lawless, H. T. (2006). Expectations and stabilityof preference choice. Journal of Sensory Studies, 21, 441455.

Chapman, K. W., & Lawless, H. T. (2005). Sources of error and the no-preferenceoption in dairy product testing. Journal of Sensory Studies, 20, 454468.

Chapman, K. W., Lovelace, E., Cardello, A., & Lawless, H. T. (2010). Preference for oneof two identical stimuli: Expectations, explicit instructions and personal traits.Journal of Sensory Studies, 25(Suppl. 1), 3553.model. This time they altered the regular preference testing situa-tion by substituting a triadic preference test.

The authors are not in the business of verifying or rejecting anymodels. They merely claim that thinking in another way gave theminsight into how to solve the problem of preference responsesbeing given to placebo pairs. Accordingly, sufciently differentconditions were established so that automatic behavior was notelicited. Consequently, sufcient numbers of consumers passedthe placebo screening test to make the new protocol viable. Itwould seem that the tendency to respond to extraneous factorshad been greatly reduced.

One shortcoming of the return to central processing is thatalthough it is a viable method, it does not include every consumer;some are resistant to the change. It provides an 8090% sample ofconsumers free of response bias but that leaves 1020% who arenot free. This minority may return to central processing with theirbias intact. Chapman et al. (2006) described a judge who, whenpresented with two clearly identical color swatches of wool, tojudge for color preference, with identical color names and numbersclearly visible, was suspicious enough to try to smell the swatchesto see whether there was a difference.

Thus, by thinking in a different way, the problem of placebopreferences is greatly reduced. The authors await conrmatorystudies. Yet, there is a serious deciency in research in this areaand that is an absence of validating studies. Namely, attentionhas not been paid to whether the experimental tests really predictreal life preference or choice behavior. Since the 1950s only three

d Preference 39 (2015) 815Christensen, R. H. B., Ennis, J. M., Ennis, D. M., & Brockhoff, P. B. (2014). Pairedpreference data with a no-preference option Statistical tests for comparisonwith placebo data. Food Quality and Preference, 32, 4855.

Christensen, B. B. H., Lee, H.-S., & Brockhoff, P. B. (2012). Estimation of theThurstonian model for the 2-AC protocol. Food Quality and Preference, 24,119128.

Delwiche, D., & OMahony, M. (1995). Flavor discrimination: An extension ofThurstonian paradoxes to the tetrad method. Food Quality and Preference, 7,15.

Ennis, D. M. (1993). The power of sensory discrimination methods. Journal ofSensory Studies, 8, 353370.

Ennis, D. M., & Collins, J. (1980). The distinction between discrimination andsplitting in paired testing. In Report # 80-233 (pp. 50). Richmond, Virginia:Philip Morris Research Center.

Ennis, D. M., & Ennis, J. M. (2012a). Accounting for no difference/preferenceresponses or ties in choice experiments. Food Quality and Preference, 23,1317.

Ennis, J. M., & Ennis, D. M. (2012b). A comparison of three commonly used methodsfor treating no preference votes. Journal of Sensory Studies, 27, 123129.

Green, D. M., & Swets, J. A. (1966). Signal Detection Theory and Psychophysics. NewYork: Wiley.

Harwood, M. L., Ziegler, G. R., & Hayes, J. E. (2012). Rejection thresholds in chocolatemilk: Evidence for segmentation. Food Quality and Preference, 29, 128133.

Jesionka, V., Rousseau, B., & Ennis, J. M. (2014). Transitioning from proportion ofdiscriminators to a more meaningful measure of sensory difference. FoodQuality and Preference, 32, 7782.

Kahneman, D. (2011). Thinking fast and slow. New York: Farrar, Straus & Giroux.Kim, H.-S., Lee, H.-S., OMahony, M., & Kim, K.-O. (2008). Paired preference tests

using placebo pairs and different response options for chips, orange juices andcookies. Journal of Sensory Studies, 23, 417438.

Lawless, H. T., & Heymann, H. (2010). Sensory evaluation of food. Principles andpractices (2nd ed.). New York: Springer.

Lee, H.-S., & OMahony, M. (2004). Sensory difference testing: Thurstonian models.Food Science and Biotechnology, 13, 841847.

Macmillan, N. A., & Creelman, C. D. (1991). Detection Theory: a Users Guide. NewYork: Cambridge University Press.

Marchisano, C., Lim, J., Cho, H.-S., Suh, D.-S., Jeon, S.-Y., Kim, K. O., & OMahony, M.(2003). Consumers report preference when they should not: A cross-culturalstudy. Journal of Sensory Studies, 18, 487516.

Neal, D. T., Wood, W., Wu, M., & Kurlander, D. (2011). The pull of the past: When dohabits persist despite conict with motives? Personality and Social PsychologyBulletin, 37, 14281437.

OMahony, M., Masuoka, S., & Ishii, R. (1994). A theoretical note on difference tests:models, paradoxes and cognitive strategies. Journal of Sensory Studies, 9,247272.

OMahony, M., & Rousseau, B. (2002). Discrimination testing: a few ideas, old andnew. Food Quality and Preference, 14, 157164.

Peryam, D. R., Polemis, B. W., Kamen, J. M., Eindhoven, J., & Pilgrim, F. J. (1960). Foodpreferences of men in the U.S. armed forces. Chicago: Department of the ArmyQuartermaster Research and Engineering Command.

Prescott, J., Norris, L., Kunst, M., & Kim, K. (2005). Estimating a consumer rejectionthreshold for cork taint in white wine. Food Quality and Preference, 16,345349.

Resurreccion, A. V. A. (1998). Consumer sensory testing for product development.Maryland: Aspen Publ. Inc..

Rosas-Nexticapa, M., Angulo, O., & OMahony, M. (2005). How well does the 9-pointscale predict purchase frequency. Journal of Sensory Studies, 20, 313331.

Stone, H., & Sidel, J. L. (2004). Sensory evaluation practices (3rd ed.). Oxford, UK:Elsevier.

Sung, Y. E., Lee, H.-S., OMahony, M., & Kim, K.-O. (2011). Paired preference tests:Use of placebo stimuli with liking and buying preference. Journal of SensoryStudies, 26, 106117.

Ullman-Margalit, E., & Morganbesser, S. (1977). Picking and choosing. SocialResearch, 44, 757785.

Wichchukit, S., & OMahony, M. (2010). Paired preference tests: Liking, Buyingand Take Away preferences. Food Quality and Preference, 21, 925929.

Xia, Y., Rivera-Quintero, A., Calderon, E., Zhong, F., & OMahony, M. (2014). Pairedpreference tests with reversed hidden demand characteristics. Journal of SensoryStudies, 29, 149158.

E. Caldern et al. / Food Quality and Preference 39 (2015) 815 15

The triadic preference testIntroductionExperiment 1Materials and methodsConsumersStimuliProcedure

Results

Experiment 2Materials and methodsPart AConsumersStimuliProcedure

Part BConsumersStimuliProcedure

Part CConsumersStimuliProcedure

Part DConsumersStimuliProcedure

Results

DiscussionReferences