Vol. 7 No.3 June 2005

HHiillddeeggaarrddee HHeeyymmaannnnProfessor, Sensory Science

Department of Viticulture and Enology

University of California - Davis, USA.

hheymann@ucdavis.edu

IInn 11997799,, RRoosseeMMaarriiee PPaannggbboorrnn wwrroottee anarticle1 with the secondary title 'The Crocodilesare coming.' In this article, written for a shortcourse, she described among other things aquiz that one should subject oneself to priorto doing any sensory testing. She alsodiscussed some aspects of sensory physiology,namely taste as defined by consumer and bythe physiologist, issues related to thresholdvalues and (mis)uses of these values,magnitude estimation and unresolvedproblems with suggested areas for futureresearch in sensory science. I was reminded ofthis article recently when I had three separateconversations that indicated to me that despitethe intervening 25 years - the crocodiles arestill coming.

The first conversation was with a professionalin industry who had been told that her well-trained and experienced panel was 'useless'because they did not perceive the odorassociated with a specific volatile compound inthe product. The panel was deemed 'useless'because instrumentally it was known that aspecific compound was in the product at alevel above threshold. The assumption wasthat, therefore the panel should be able toperceive it. However, upon further questioning

Musings on SensoryCrocodiles

E-Nose Pty Ltd

Just-About-Right Scales inConsumer Research

cont. pg 2

cont. pg 2

Graham Bell and Associates Pty LtdCentre for ChemoSensory ResearchISSN 1442-9098

RRiicchhaarrdd PPooppppeerr aanndd DDaanniieell RR.. KKrroollllPPeerryyaamm && KKrroollll RReesseeaarrcchh CCoorrppoorraattiioonn

6323 North Avondale Ave.Chicago, IL 60631, USA

Richard.Popper@pk-research.com

OPINION

NNeeww pprroodduucctt iinnttrroodduuccttiioonnss aarree ccrriittiiccaall ttoo tthhee ggrroowwtthh, continuing success andcompetitive strategies of packaged goods companies. In order to improve theodds of having a successful launch - whether of an innovative product or a lineextension - companies routinely incorporate consumer feedback in the productdevelopment process and obtain consumer reactions to product prototypes asthey emerge from R&D. Whether it's food, beverage or personal care, thestakes involved are so high that this investment in consumer research not onlymakes sense, it's probably a necessity, and it can spell the difference betweensuccess and failure.

The research tools employed in the pursuit ofa winning product formulation run from thesimple to the highly sophisticated. Theprimary objective, however, is the same:provide the product development team with direction on how to increase consumer appeal.

In product categories where sensoryproperties are important determinants ofconsumer appeal, one of the most simple anddirect ways to solicit feedback from aconsumer is to ask whether a product is justright with regard to a certain characteristic orhas too much or too little of that

What's Wrong with Sensory?

Wine Sense: New Tool

Program for Paradise

Student Assistance

INSIDE:

TM

ChemoenseS

Just-About-Right Scales in Consumer Research continued

4

characteristic. These "just-about-right"scales (Figure 1) can be effective inresearch on food and beverages, whereconsumers, in addition to rating theirliking of a product, are asked toevaluate a product on a number ofattributes using this question format.

For example, in a study of carbonatedsoft drinks, consumers might be askedto evaluate the prototypes with regardto sweetness, strength of flavor andcarbonation level (among othercharacteristics), indicating each timewhether the level is too low, too high orjust right. Based on consumers'answers to these questions, the softdrink manufacturer might adjust aprototype's sweetness, flavor andcarbonation in an effort to improve itsacceptability. Respondents tend toanswer just-about-right questions withease and researchers like the simplicityof the scale. Yet despite their intuitiveappeal to researchers and researchparticipants alike, these scales are notwithout limitations and potential pitfalls,and their results require carefulinterpretation.

In order to rate sweetness using a just-about-right (JAR) scale, respondentsmust decide how closely the sweetnessof the product they are tasting matchestheir ideal sweetness. There can oftenbe a disparity between the product

formulations consumers will rate as "justright" and those that they actually likethe most. Epler, Chambers, and Kemp(1998) asked to evaluate five lemonadesdiffering in the amount of added sugar.Consumers rated the product usingeither a JAR scale that ranged from "notsweet enough" to "much too sweet," orrated their liking of the sweetness on ascale that ranged from "dislikeextremely" to "like extremely." Theoptimal sugar concentration wasdetermined in two ways: by identifyingthe formulation whose average JARrating was closest to "just right" and byidentifying the formulation with thehighest average liking score. Using thedata reported in that study, Figure 2shows how the optimal levels can bedetermined using each measure.

For the JAR scale, the optimal sugarconcentration was about 9.5%; for theoverall liking scale about 10.5%. Whilethe difference may seem small, it waslarge enough to make a difference in apreference test. A separate group of

consumers, when presented with thetwo formulations, preferred the productoptimized on the basis of liking over theone optimized on the basis of the JARscale.

In another study the disparity betweenoptimizing a formulation based on aJAR scale versus a liking scale was evengreater. Optimizing the level ofaspartame in a fruit drink on the basisof a JAR scale for sweetness predictedan optimal level of aspartame 20%lower than that predicted on the basisof overall liking (Popper, Chaiton &Ennis, 1995).

When products vary in more than justone dimension a similar question arises:is the formulation that maximizes overallliking the same as the formulation forwhich the sensory characteristics are alljust right? In some instances(Moskowitz, Munoz & Gacula, 2003), aproduct that was just right on allattribute measures was not the same asthe product that was liked the most,although it was still an acceptableproduct; in another instance (Marketo &

cont. pg 5

Vol. 7 No.3 June 2005

ChemoSSense

Much too strong...........

Somewhat too strong ...

Just-about-right ............

Somewhat too weak.....

Much too weak ............

1

2

3

4

5

6

7

5 6 7 8 9 10 11 12 13 14 15

Percent Sucrose

Liki

ng S

cale

or J

AR

Sca

le

1%

JAR Scale Means

Liking Scale Means

FFiigguurree 11.. EExxaammppllee ooff aa ffiivvee--ppooiinntt jjuusstt--aabboouutt--rriigghhtt ssccaallee uusseedd iinn pprroodd--uucctt eevvaalluuaattiioonn.. OOtthheerr vveerrssiioonnss ooff tthhee ssccaallee eemmppllooyy tthhrreeee oorr sseevveennrreessppoonnssee ccaatteeggoorriieess,, wwiitthh tthhee mmiiddddllee ccaatteeggoorryy llaabbeelleedd ""jjuusstt--aabboouutt--rriigghhtt""..

FFiigguurree 22.. OOppttiimmaall ssuuccrroossee lleevveell iinn lleemmoonnaaddee ddeetteerrmmiinneedd uussiinngg aa 77--ppooiinntt lliikkiinngg ssccaallee ((ssoolliidd lliinnee)) aanndd aa 77--ppooiinntt JJAARR ssccaallee ((ddaasshheedd lliinnee)).. TThhee ooppttii--mmuumm ssuuccrroossee lleevveell iiss tthhee lleevveell aatt wwhhiicchh tthhee lliikkiinngg ccuurrvvee rreeaacchheess iittss mmaaxxiimmuumm oorr tthhee JJAARR lliinnee eeqquuaallss 44 ((jjuusstt--aabboouutt--rriigghhtt)).. TThhee ddiiffffeerreennccee iinn tthheeooppttiimmaall vvaalluueess iiss aapppprrooxxiimmaatteellyy 11%%.. SSoouurrccee:: EEpplleerr,, eett aall..,, 11999988..

Just-About-Right Scales in Consumer Research continued

Vol. 6 No.3 June 2004Vol. 7 No.3 June 2005

5

Moskowitz, 2004), the two methodsgave similar results.

One possible explanation, advanced byEpler et al., for the discrepancybetween JAR and liking scale optima isthat JAR scales induce a response biaswhen the attributes carry certainnegative health connotations. In thecase of sweetness respondents mayperceive a very sweet product as beingunhealthful. When tasting such aproduct, they may say it is "too sweet"because they are aware of thepotentially negative consequences ofconsuming such a product on a regularbasis. At the same time, they mayactually like the way it tastes, which isreflected in their hedonic ratings. Onlyone study (Bower & Baxter, 2003) hasattempted to confirm this hypothesis bycomparing JAR and liking ratings ofsweetness among two groups ofrespondents that differed in theirconcern with "healthy eating".Unfortunately, the results wereinconclusive.

While health concerns may or may notbe a source of response bias in the useof JAR scales, experienced researchersknow that there are certain attributesthat, by their nature, are likely to induce

a response bias. It is hard to imaginethat an orange juice could have toomuch "fresh orange flavor," and manyrespondents would rate the level offresh orange flavor as "not enough"regardless of the formulation. Similarbiases may exist in the case of theamount of chocolate chips in achocolate cookie, or the amount ofcheese on a pizza. In both instances,researchers can expect respondentsusing a JAR scale to express a desire for

more, even though their liking ratingsmay begin to decrease as the level ofchocolate chips or the amount ofcheese rises above a certain level.

The reverse skew can also occur, saywhen respondents are asked to rate thebitterness of coffee: the responses willskew towards too much, sincebitterness is considered "bad."However, a certain amount of bitternessmay actually be a positive in terms ofoverall liking, JAR ratings to thecontrary.

What direction to product developmentdo JAR ratings provide? Consider theresults for a hypothetical product shownin Table 1.

The results suggest that the sweetnessand citrus flavor of the product shouldbe increased; less clear is whether thecarbonation level should be raised. Theresults definitely do not tell the productdeveloper how mmuucchh of a change insweetness, flavor, or carbonation wouldbe required to increase the just rightpercent. It is tempting to conclude thata bigger increase is needed in the caseof citrus flavor than sweetness since thepercentage of too low responses isgreater for citrus flavor than for

sweetness. But that is not necessarilythe case. The sensitivity of the JARscale to formulation changes is usuallynot known and may differ by attribute(Moskowitz, 2004). It might requireonly a small increase in flavor, but alarge increase in sweetness to addressthe perception that these attributes aretoo low. And it is not known whethersuch increases would alienaterespondents who currently view the

levels of these attributes as just right,leading to a greater percentage offuture respondents rating the producttoo sweet or too high in citrus flavor.Finally, the possible interaction amongattributes needs to be considered whenmaking formulation adjustments(increasing the citrus flavor may changethe desired level of carbonation).

It is also tempting to conclude from theresults that an increase in citrus flavorhas the greatest potential to improvethe overall acceptability of the product,since this was the shortcoming noticedby the greatest percentage ofrespondents. But this conclusion couldalso be erroneous. Consumers mightbe more tolerant of deviations in flavorlevel than they are in sweetness,making sweetness the higher priority interms of reformulation. Furthermore,even though the overall level ofsatisfaction was greater for carbonationthan for the other two attributes, it ispossible that for those that consideredthe carbonation too high or too low,this shortcoming was a bigger detractorthan anything else.

Such interpretive difficulties underscorethe need for researchers to link the just-about-right ratings to the respondent'slevel of liking. Using one of severalanalysis techniques, it is possible to rankorder the shortcomings in terms of theirimportance to overall liking, therebyfocusing the attention of productdevelopment on the critical attributes.In some cases, an attribute garnering arelatively moderate percentage ofcomplaints (e.g. carbonation too low)may be shown to have a high impact onthe overall liking of some respondents.

While a more in-depth analysis canmake the results from JAR scales moreactionable for the product developer,including JAR scales may still beproblematic, as was demonstrated in astudy by Popper et al. (2004). In thestudy, respondents rated their overallliking for four dairy desserts. Some

cont. pg 6 ChemoSSense

Carbonated Soft Drink Ratings Too Low (%) Just Right (%) Too high (%) Sweetness 30 55 15 Citrus flavor 50 40 10 Carbonation 22 65 13

TTaabbllee 11.. HHyyppootthheettiiccaall rraattiinnggss ooff aa ccaarrbboonnaatteedd ssoofftt ddrriinnkk oonn tthhrreeee jjuusstt--aabboouutt--rriigghhtt ssccaalleess,, ssuummmmaarriizzeedd iinn tteerrmmss ooff tthhee ppeerrcceennttaaggee ooffrreessppoonnddeennttss rraattiinngg tthhee pprroodduucctt jjuusstt rriigghhtt,, ttoooo llooww,, oorr ttoooo hhiigghh..

Just-About-Right Scales in Consumer Research continued

6 ChemoSSense

Vol. 7 No.3 June 2005

respondents rated only overall liking.Other respondents, in addition to ratingtheir overall liking, rated the productson a series of JAR scales, such assweetness, thickness and flavorintensity. The study showed that therespondents that answered the JARscale questions rated their overall likingof the products differently than thosethat rated only overall liking.

If JAR scales are biasing respondents'overall evaluations of products, thenincluding them in studies designed tomeasure a product's overall liking maybe ill advised. Popper et al. (2004)found that intensity scales, which askrespondents to rate the level of sensoryintensity on a scale from low to high,did not have the same biasing effectsthat the JAR scales did, even though thesame attributes were being rated. Thedifference between the two scale typesis that in answering JAR questionsrespondents need to consider howproducts differ from an ideal, whichmay focus them on reasons why theylike or dislike a product, something thatintensity scales may not. Other research(Wilson & Schooler, 1991) has shownthat asking respondents to considerreasons for their preferences maysubsequently alter their preferencechoices.

With the difficulties surrounding the useof JAR scales*, why do they remain sopopular? One reason is that alternativeresearch methods can be more costly.Systematically varying a number of keyformulation parameters and inferringthe optimal formulation from the overallliking responses may require more

prototypes than product developmentthinks it has the time or money toproduce and test. Similarly, formulationdirection based on a correlation withintensity ratings, whether collected fromconsumers or from a trained sensorypanel, also requires a fair number ofprototypes or in-market products inorder to be robust. Compare thatapproach to one of testing only one ortwo prototypes (and maybe acompetitor) and using just-about-rightscales for formulation direction, and theappeal of just-about-right scales forproduct development is immediatelyapparent.

JAR scales do not give the specificity ofdirection that product developmentoften requests, which can lead toinefficient testing-and-retesting in orderto get the formulation right.Nevertheless, just-about-right scales, inthe hands of knowledgeable researchersand along with the appropriateanalyses, can do a just-about-right jobof serving as a score card forcomparing a number of products andindicating areas where there are majorproduct deficiencies.

* This article discusses some of the limitationsand caveats surrounding the use of just-about-right scales. A subcommittee of ASTMCommittee E-18 is drafting a detailed guideconcerning the benefits and risks associated withthe use of just-about-right scales. Thatdocument will also include examples of thestatistical analyses most appropriate for JARscales

RREEFFEERREENNCCEESS

Bower, J.A. & Baxter, I.A. (2003). Effects of health concern and consumption patterns on measures ofsweetness by hedonic and just-about-right scales.Journal of Sensory Studies, 18 (3), 235-248.

Epler, S., Chambers, E & Kemp, K. (1998). Hedonicscales are a better predictor than just-about-rightscales of optimal sweetness in lemonade. Journal ofSensory Studies, 13, 191-197.

Marketo, C. & Moskowitz, H. (2004). Sensory optimization and reverse engineering using JARscales. In Data analysis workshop: getting the mostout of just-about-right data, Food Quality andPreference, 15, 891-899.

Moskowitz, H.R. (2004). Just about right (JAR) directionality and the wandering sensory unit. InData analysis workshop: getting the most out of just-about-right data, Food Quality and Preference, 15,891-899.

Moskowitz, H.R., Munoz, A.M. & Gacula, M.C. (2003).Viewpoints and Controversies in Sensory Science andConsumer Product Testing. Food & Nutrition Press,Trumbull, CT, USA.

Popper, R., Chaiton, P. & Ennis, D. (1995). Taste test vs.ad-lib consumption based measures of productacceptability. Presented at the Second PangbornSensory Science Symposium, University of California,Davis, July 30 - August 3.

Popper, R., Rosenstock, W., Schraidt, M. & Kroll, B.J.(2004). The effect of attribute questions on overallliking ratings. Food Quality and Preference, 15, 853-858.

Wilson, T.D. & Schooler, J.W. (1991). Thinking toomuch: introspection can reduce the quality of preferences and decisions. Journal of Personality andSocial Psychology, 60(2), 181-192.

ReceiveChemoSense freeSSeenndd ""ssuubbssccrriibbee"" mmeessssaaggee ttoo

gg..bbeellll@@aattpp..ccoomm..aauu

((SSeenndd ""rreemmoovvee"" ttoo qquuiitt))