The Experimenter Effect

8/3/2019 The Experimenter Effect

1/10

Psychological Bulletin196S, Vol. 63, No. 4, 223-232

THE EXPERIMENTER EFFECTB. L. KINTZ, D. J. DELPRATO, D. R. METTEE, C. E. PERSONS, AN D R. H. SCHAPPE

Ohio UniversityRecent investigations in experimental psychology have shown somew hat s tr ik ingresults which indicate t ha t experimenters (Es) may and do influence their data.This problem w as considered in relation to the areas of: experimental, counsel-ing, and testing psychology. A historical review revealed t ha t although theinfluence of E has been generally acknowledged, an unconcerned attitudetowards this phenom enon has been taken, especially by experime ntal psy-chologists. Counseling psychology has been most concerned with th e experi-menter effect because this is closely related to the counseling situation. Workersin this area have not only recognized but have atempted to manipulate E in-fluence . In the field of testing, investigators h ave re porte d significan t vari-ability in Ss' per formances as a result of E characteristics.

It is significant that a problem which per-plexed some of the most influential scientistsof Germany in 1904 was resolved at thattime, yet should contaminate psychological in -vestigations of the present day, that is, theexperimenter 's influence on his subjects. Theamazing horse of Mr. von Osten caused anuproar throughout all of Germany which Pro-fessor Stumpf and his co-workers, throughmeticulous investigation, demonstrated to bethe result of the questioners' unintentional,involuntary cues utilized by the animal. Thisincident dramatically emphasized th e st imu-lus value of "unconscious" cues emitted byan experimenter to his animal subjects. Eventhough questioners of "Hans" were awarethat this might be the explanation for hisfeats and were most careful in attempting torefrain from allowing him this advantage, theunconscious cues were still emitted until th esituation w as carefully analyzed and the spe-cific variables controlled (Pfungs t , 1911) .McGuigan (1963) states:While w e have traditionally recognized that th e char-acteristics of an experimenter m ay indeed influencebehavior, it is impor tan t to observe that w e havenot seriously attem pted to stu dy him as an inde-pendent variable [p . 421],However, Stumpf with his careful, detailedmeasurements of questioners' cues began thestudy of the experimenter as an independentvariable in 1904, but not until recently hasthis problem been considered by experimentalpsychologists for study (Cordaro & Ison,1963; McGuigan, 1963; Rosenthal & Halas,

1962) . Clinical psychologists have long ledthe way in this aspect of investigation. Thepersonal effect of examiners upon patients 'performance in clinical tests was initiated asan object of study 35 years ago (Marine,1929) . Yet, psychologists working in thelaboratory have not been completely unawareof the implications of experimenter influenceupon subjects .Ebbinghaus (1913) in discussing th e effectsof early data returns upon psychological re-search stated:

it is unavo idab le tha t , af ter th e observation of thenumer ical results, suppositions should arise as to gen-eral principles which ar e concealed in them an dwhich occasionally give hints as to their presence.As th e investigations are carried fur ther , these sup-positions, as well as those present at the beginning,constitute a complicating factor which probably hasa definite influence up o n th e subsequent results [pp.28-29],Pavlov, noting the apparent increase in learn-ing ability of successive generations of micein experiments on the inheritance of acquiredcharacteristics, suggested tha t an increase inth e teaching ability of experimenters m ayhave, in fact, constituted th e critical variable(Gruenberg, 1929, p . 3 2 7 ) .The foregoing yields some indication of thescope inherent in this phenomenon. However,response-induced bias is not the only data-affected result . A study in which experiment-ers (Es) recorded the frequency of contrac-tions and head turns of planaria demonstratedthat E's expectancy can dramatically influ-ence th e da ta obta ined in this type of situa-

223


2/10

22 4 KINTZ, DELPRATO, METTEE, PERSONS, AND SCHAPPEt ion (Cordaro & Ison, 1963) . In this casesubje ct's (S's) responses were not affected,but highly statistically significant differencesin number of reported responses were ob-tained for E's expecting a low f requency ofresponse and for E's expecting a high fre-quency of response. In other words, Es saw( r epor ted) what they expected to see.Thus far we have seen that E may notonly bias S's responses but also that this in -terpretation of S's responses may be biased.Because these effects are dependent upon E 'sknowledge of the hypothesis to be tested orhis expectancy, one can readily propose tha tth e solution to this problem would be, as isoften th e case, a simple matter of having re-search assistants (As) who are unaw are ofth e hypothesis collect th e data for E. In test-in g this suggestion, i t was found tha t "a subtletransfer of cognitive events" existed, resultingin response bias (Rosenthal , Persinger, Vikan-Kline, & Mu lray, 1963, p . 313) . The authorsstate:Our finding of a subtle transmission of E's bias totheir 4s forces us to retract an earlier suggestion fo rthe reduct ion of E bias (Ro senth al, 1963c). Ourrecom m enda t ion had been to have E employ a sur-rogate data collector who was to be kept ignorantof th e hypothes is under tes t . The implication of thesuggestion w as simply not to have E tell A the hy-pothesis . I t now appears t ha t E 's simply no t tellingA th e hypothesis may not insure A's ignorance ofthat hypothesis [pp. 332-333].

It is the present authors ' contention tha twherever an experimenter-subject relationshipexists, th e possibility also exists for E to con-taminate his data by one or more of a multi-tude of conveyances. It appears that experi-mental psychology has too long neglected th eexperimenter as an independent var iable . B yrelating some of the f indings of clinical andsocial psychologists, as well as the few experi-mental studies to date , it is hoped that experi-mental psychologists will no longer accept onfaith tha t th e exper imenter is necessary butharmless. Implications for experimental, coun-seling, and testing psychology will also beconsidered.

R E S E A R C H F I N D I N G SNondifferentiated E f f e c t s

Research studies have been, on the whole,minimal in reporting differential results with

regard to individual Es. In particular , this istrue concerning careful discussion of the pos-sible reasons for the differing data. It is, how-ever, illustrative of the pervasiveness of ex-perimenter effect to examine several of thestudies which have shown a nondifferentiatedexperimenter influence.Lord (195 0) was interested in examiningRorschach responses in three different typeso f situations. Thirty-six Ss took th e Ror-schach three timesonce from each of threedifferent female examiners. Of the Rorschachresponses being considered for differenceswithin Ss, Lord found 48 to yield t tests sig-nificant at th e .10 level. Of these, 27 we redue to examiner differences.In an interesting study on learning withoutawareness (Postman & Jarre t t , 1 9 S 2 ) , 30 d i f -feren t Esall students in advanced experi-mental psychology classeswere employed.The Ss responded to 240 st imulus words withanother word which came to mind. Half th eSs were instruc ted to guess, an d half weretold, the "correct" principle of answering,which was to give common associations asfound in speaking and writ ing. Differencesamong Es were highly significant sources ofdata variance. Postman and Jarre t t suggestthat since complete universal uni formi ty ofexperimenter behavior is apparently impos-sible, th e difficulty exper ienced in at temptingto replicate results of other investigators is tobe expected.Using a verbal conditioning paradigm, Kan-fe r (195 8 ) reinforced verb responses with af lashing l ight, under three different re inforce-ment schedules . Two Es were employed, withapparent ly l i t t le difference be tween them. Inreinforcing Ss, E simply w as required to dis-t inguish between verbs and nonverbs. A sig-nificant interaction w as found between E andmethod. There was more frequent reinforce-me n t of words for one schedule than for theothers, the f requency varying with the E. Ap-parently, even th e ability of an E to per fo rmsuch a relatively e lem enta ry, objective taskas judging whether a word is, or is not, a verb,is highly subject to individual differences.A recent experiment (Severin & Rigby,1 9 6 3 ) investigated different patterns of digitgrouping. An inc identa l f inding from thiss tudy is pertinent here. In analyzing th e vari-


3/10

THE EXPERIMENTER EFFECT 225ance of perfec t memoriz ing, an E effect wasfound , significa nt at the .01 level. Fu rthe r,this effect w as largely due to only one of thefour Es, since repetition of the analysis with-ou t this E's data yielded no significant E d i f -ference.A n avoidance study using rats (Harris, Pic-colino, Roback, & Sommer, 1964) w as pri-marily concerned with th e effects of alcoholon learning th e avoidance response in a Miller-Mowrer shut t le box. Tw o teams, of two Eseach, were employed. The alcohol did not pro-duce differential results, but the different Eteams did. It is likely that this outcome is dueto differences in the handling of the animalsby E s.Personality

In discussing th e differential effect of Eupon 5 , Masling (1960) w as writ ing with par-ticular attention to projective testing. How-ever, it seems logical to postulate that if sexand aspects of the examiner 's personality(such as warmth or coldness) are causativeof differential results in projective testing, th einfluence of these personal variables m ay alsobe felt in other, even objective, situations.In attempting to assess effects of person-ality factors of exper imenters in the exper i-mental situation, McGuigan ( 1 9 6 0 ) comparedtrait scores of Es on personality tests withdependent variable scores of Ss. He did notobtain any significant correlations, but notedseveral quite high ones that m ay indicate di-rectional influence s. For example, the moreneurotic (Bl-N scale of the B e r n r e u t e r ) theE, the poorer the per fo rmance of 5.The effect of E's personal i ty upon Ss' per-formances had been invest iga ted ear l ie r (San-ders & Cleveland, 1953) using projective tech-niques. Nine E s took the Rorschach, whichwas scored blindly by two experienced clini-cal psychologists. The Es were then t ra ined inadministering th e Rorschach, and each E gaveit to 30 5s. An a t tempt w as made to deliber-ately standardize the quest ioning proceduresused. After taking the Rorschach, each 5f illed out a questionnaire designed to elicit hisat t i tudes about the par t icular E. Sanders andCleveland found that overtly anxious E s (a sindicated by their own Rorschach responses)tended to elicit more subject f lexibil i ty and

responsiveness, wh ile overtly hostile Es (againmeasured by their Rorschach responses) drewmore passive and stereotyped responses andless of the hostile responses. The 5s' quest ion-naires indicated that Es who were most l ikedwere those who had been ra ted low on anx-iety and hostility.

The research just mentioned has been pri-marily interested in the effect of the person-ality of E, per se, on 5s' per fo rmances . O nefur ther study is especially interesting, as ittries to answer the p ertinen t qu estion ofwhether E's personality and personal bias caninteract. Rosenthal , Persinger, and Fode(1962) used 10 naive Es, who were biased toexpect certain results. They found that agree-me n t of f inal data and E bias were related toEs' scores on the MMPI scales, L, K, and Pt,bu t not to age or grade-point average.The S-E personality interaction is depend-ent , of course, not only on the personality ofE, but also to some degree on that of 5. Inone of the few studies designed to investigatethis interaction, Spires (1960; cited in Mc-Guigan, 1963) used a 2 X 2 design in a ver-bal condit ioning paradigm, reinforcing a par-ticular class of pronouns wi th th e word"good." The 5s were divided into tw o groups,one of which had scored high on the Hy scaleof th e MMPI and one of which had scoredhigh on the Pt scale. Each group was sub-divided in half, receiving either a positive ora negative "set" ("this experimenter is warmand f r iend ly " o r "this experimenter is coldand un f r i e n d ly " ) . The high .ffy-positive se tgroup far surpassed the other three groups.The high ZTy-negative set group per fo rmedthe poorest. Thus, not only E's personality,but 5s' perception of this personality, cancontr ibute to the E effect.Investigation of 5s' perception of E has beenunder taken by two related studies (Rosenthal,Fode, Fr iedman, & Vikan-Kl ine , 1960;Rosenthal & Persinger, 1962). In the first ex-per iment , 5s were asked to rate E on a n u m -ber of variables. In the second study, the ex-per iment w as not ac tua l ly condu cted, but onlydescribed, and 5s were requested to imagina-tively rate their imaginary E. Yet a correla-tion which was calculated between the ratingsof the first and second studies yielded an rof .81. This wo uld appear to support the hy-


4/10

226 K1NTZ, DELPRATO, M E TTE E , PERSONS, AND SCHAPPEpothesis that naive Ss, in particular, m ay havea kind of predetermined "set" about what a"typical" E is likescientific, intelligent, etc.Experience

Inves t igators wi th widely variant amountsof experience are busily conducting studiesevery day. Cantri l (1944) stated that inter-viewers who are highly experienced show asmuch bias as those who are less experienced.In an experimental investigation, however,Brogden (1962) came to a different conclu-sion. Four s each trained a group of rabbitsand recorded the acquisit ion speed of a con-ditioned shock-avoidance response. The rab-bits of the three experienced Es reached thelearning criterion faster than the naive E'srabbits. To fu r the r study this result , th e naiveE w as required to run another group of rab-bits to see whether his practice w ould prod ucemore rapid conditioning in the second group.Anothe r experienced E t ra ined another groupof 5s to serve as a control measure. The datashow both a significant E practice effect ( f o rthe naive E only) and a significant differencebetween Es.Se x

Several studies have been concerned withinvestigating the manner in which results areinfluenced by E differences in sex. In a verbalcondi t ioning s tudy (Binder , McConnel l , &Sjoholm, 1 9 5 7 ) , Ss were reinforced for sayinghostile words. Tw o clearly distinguishable E swere employed: onea you ng, pet i te femininegirl; the othera mature, large masculinemale. Significantly more hosti le words wereemitted in the presence of the female E . It isconceivable that Ss perceived th e male E asbeing more hosti le than th e female E, in whichcase the results confirm Sanders and Cleve-land's (19S3) f indings that hostile E s elicitfewer hostile responses f rom their Ss than doother E s.Sarason and Minard (1963) a lso found thatsex and hosti l i ty significantly influenced Ss'performances. Degree of contact between Sand E and E 's prestige value (a s perceived byS) also contr ibuted significant effects. Sarasona nd Mi na rd w a rn that ignoring these situa-tional variables is hazardous research meth-odology.

In a very recent experiment investigatingthe sex variable, Stevenson and Allen (1964)show what is perhaps th e most clear-cut dem-onstration of S-E interaction. Eight male andeight female Es each tested eight male andeight female Ss in a simple sorting task. Themean number of responses was recorded at30-second intervals. With either male or fe-male JEs, female Ss made more responses thandid male Ss. However , all Ss performed rela-tively better under an opposite-sexed E.Expectancy E f f e c t

Perhaps th e component of experimenter ef-fect which is the cause of greatest concern isthat by which the E in some way influenceshis Ss to per fo rm as he has hypothesized. Thereasons for concern about expectancy effectare that so little is known abou t it and solittle research has been devoted to it. Onlyrecently have systematic studies been con-ducted in this area.Rosenthal and Fode ( 1 9 6 3 a ) demonstratedth e problem clearly in an experiment wi thtw o groups of randomly assigned animals. O negroup of six Es w as instructed that it sgroup of ra ts w as "maze-bright" and a secondgroup of s ix Es was instructed that i ts groupof rats w as "maze-dull ." In a simple T maze,the maze-bright ra ts per formed significantlybetter than the maze-dull rats.In a s imilar s tudy (Rosenthal & Lawson,in press) investigators divided 38 Es into 14research teams, each of which had one ratrandomly ass igned to it. Six of the teamswere told that their rats were bred for dull-ness and the other eight were told that theirrats had been bred for brightness. Seven ex-periments, including such tasks as operantacquisit ion, st imulus discrim inatio n, and chain-ing of responses, were conducted. In seven outof eight comparisons (overall p = .02) , d i f -ference in per fo rmance again favored Es w hobelieved their Ss to be bred for brightness.A factor which m ay have prompted th e differ-ence w as that Es w ho believed their rats werebred for brightness handled them more thanE s w ho believed their rats were bred for dull-ness.In both experiments c i ted, the questionarises as to the sensitivity of the animalsto a t t i tudinal differences in E s t ransmit ted


5/10

THE EXPERIMENTER EFFECTthrough the tactual and sensory modalities.Further research is required to clarify th eissue.Modeling E f f e c t

Modeling effect is denned (Rosenthal ,1963b) as a significant correlation betweenE 's per fo rmance and the p e r fo r ma n c e of ran-domly assigned 5s on the same task.Graham (1960) divided 10 psychotherapistsinto two groups on the basis of their percep-tion of movement in Rorschach inkblots. Inth e ensuing psychotherapeutic sessions, pa-tients of the group of psychotherapists thatperceived more movement in the inkblots sawa significantly greater amount of movementthan th e pat ients of the group of psycho-therapists that had perceived less movement.In the area of survey research the phe-nomenon of modeling has been reported instudies by Cantr i l (1944) and Blankenship(1940) who have found that interviewerselicit from their interviewees, at a probabilitygreater than chance, responses which reflectth e in terviewers ' ow n beliefs.

Rosenthal (196 3b) repor ted e ight exper i-ments conducted to assess the existence andmagnitude of experimenter modeling effect byem ploying the task of S s' rating a series ofphotographs of people on a scale of apparentsuccessfulness and unsuccessfulness rangingfrom 10 to +10. Prior to each experiment,Es had rated th e photos which were selectedbecause in earlier ratings on the same scalethey had yielded a mean value of zero. Theresult ing eight rank-order correlations be-tween Es ra t ings and their 5s' ra t ings rangedfrom -.49 to +.65. Only the rho of +.65w as significantly different f rom 0 (p < .001) ,but the hypothesis of equality among theeight rhos was rejected using a chi-square test(p < .005) .

H a mme r and Piotrowski (195 3 ) had threeclinical psychologists and three interns rate400 House-Tree-Person drawings on a 3-pointscale of aggression. The degree of hostilitywhich clinicians saw in the drawings corre-lated .9 4 with th e evaluations of their personalhostili ty made by one of the investigators.

Early Data Returns E f f e c tEarly data returns effect is the problem ofth e experimenter who is receiving feedbackfrom his experiment through early data re-

tu rns and who contaminates the subsequentdata . The reasons w hy this occurs are unclearbut some suggestions are that E 's mood m aychange if the data are contrary to his expecta-tions, or if the data are in agreement with hisexpectations, there is the possibility of height-ening an existing bias. There is evidence(Rosenthal, Persinger, Vikan-Kline, & Fode,1963) that this mood change in E , broughtabout by "good results," m ay lead him to beperceived by the 5s as more "likable," "per-sonal," and more "interested" in their workand thereby influence the ir per formance .

In the study by Rosenthal, Persinger,Vikan-Kline, and Fode ( 1963 ) , three groupsof four Es each had three groups of 5s ratethe apparent success of people in photographson a scale ranging f rom 1 0 to +10. The E swere instructed that 5s' mean ra t ing would beabout +5. In each of two experimental groups,two Ss were confedera tes of the investigatorwhile in a control group all of the 5s werenaive. One of the confederate pairs was in-structed to give "good data" (in accord withE 's expectations) and the other pair was in-structed to give "bad data" (contrary to E'sexpecta t ions) . Ratings of all 5s were learneda f te r several trials. It was hypothesized thatth e experience of having obtained good datawould lead those Es to obtain "better" sub-sequent data while the experience of havingobtained bad data would lead those E s to ob-tain "worse" data in relation to the control.Although nei ther exper imenta l group differedsignificantly from th e control group, the ex-per imenta l groups were significantly differentfrom each other. There was a further tendencyfo r th e effect of early returns to become morepronounced in the later stages of data gather-ing.Griffith ( 1 9 6 1 ) states clearly th e effect ofearly data returns in an autobiographicaldocumentary:Each record declared itself for or against . . . ( m e ). . . (and) . . . ( m y ) . . . spirit rose and fell al-most as wildly as does the gambler whose luck sup-posedly expresses to him a higher love or reflection[p. 309].


6/10

228 KINTZ, DELPRATO, METTEE, PERSONS, AND SCHAPPEOverview oj Cues and Their Transmission

After discussing at some length the variousexper imenter effects, th e question must cer-tainly arise as to how the experimenter con-taminates his data. W h a t a re these cues andhow are they transm itted? Some suggestionshave been made but it is necessary to look atevidence dealing directly with the problem. Itwas suggested earlier that in the case of labo-ratory animals it might be due to tactual andkinesthetic cues, but probably also involvedare all of the sensory processes of the organ-is m so that E inadvertently transmits cues bynearly everything that he does.In dealing with humans, because of theprobable lack of bodily contact, cues aretransmitted verbally and/or visually. B u t"verbally" implies not only the words , bu talso th e inflectional and dynamic processes ofspeaking.

The transmission of verbal cues was firstdramat ical ly demonstrated by Greenspoon(195 5 ) who, by reinforcing plural n ouns with" mmm- h mmm, " was able to increase the fre-quency of emission of such words.

In a similar experiment, Verplanck ( 1 9 5 5 )was able to control th e content of 5s' conver-sation by agreeing with some opinions anddisagreeing with others. The results showedthat every 5 increased in his ra te of speakingopinions wi th re inforcement by agreement ,and 21 out of the 24 5s decreased their rateof opinion s ta tements w i th non reinfo rcem ent .

Rosen thai and Fode ( 1 9 6 3 b ) conductedtw o experiments specifically designed to in-vestigate th e transmission of cues from E tohis hu ma n 5s. The 5s were to ra te the ap-parent success or failure of persons in photo-graphs on a scale ranging f rom 10 to +10.A ll Es received identical instructions exceptthat f ive of them w ere told that their 5s wouldprobably rate the pictures a t a bou t + 5 whileth e remaining Es were told their 5s wouldprobably rate the pictures at about 5 . Fur-ther, prior to the experimen t, each E rated thepictures on the same scale as the 5s. Resultsshowed that 5s for high-biased ( + 5) Es ob-tained significantly higher mean ra tings than5s of low-biased ( 5) Es. Since Es were notpermit ted to say anything to 5s other thanw h a t was on the instruction sheet, th e com-

municat ion of bias must have been done bytone, manner, gesture, or facial expression.The second experiment designed to investi-gate this nonverbal transmission of cues wasconducted in the same manner as the first withthe exception that now, instead of E showingeach photo to his 5s, each set of 10 photoswas mounted on cardboard and labeled sothat 5 could give his ra t ing without Es' han-dling the photos. The results showed thatelimination of visual cues f rom to 5 didsignificantly reduce th e effect of E 's bias. Itcan be hypothesized then that visual cuesplay an impor tant part in the phenomenonof bias, but probably to a lesser degreethan verbal cues.Wickes (1956) also showed th e impor t ofvisual cues by effectively using nodding, smil-ing, and leaning forward in his chair as rein-forcement for certain responses given to ink-blots by clients in psychotherapy sessions.Considerable research is required to learnwhat th e cues are, how they are t ransmi t t ed ,and how they can be control led.I M P L I C A T I O N S OF THE EXPERIMENTER EFFECT

The preceding survey of the l i terature hasrevealed th e existence of the exper imente reffect in all aspects of psychology. Al thoughth e experimenter effect is generally recognizedand perhaps paid lip service, it tends to bea forgotten skeleton in the research psy-chologist 's closet. Comparison of a study byPostman and Jarrett ( 1 9 52) with one bySpence (1964) provides an example.Postman and Jarrett ( 1 9 5 2 ) c omme nt e d :We have paid too l i t t le at tention to the con-t r ibu t ions m a d e by v a r i a t i o n s in Es ' behavior tothe experimental results. The difficulty whichmany psychologists experience in repeat ing th eresults of other invest iga tors may be due to ourfailure to attack systematically th e role of differ-ences a m o n g s [p . 25 3] .Spence (1964) , a f t e r examining variousaspects of variabil i ty occurring in experi-ments us ing th e Taylor Mani fes t Anxie tyscale, says the following in concluding hisdiscussion of the experimenter-subject inter-action:This is, nevertheless, a po ten t ia l ly importan t var i -able and should be invest igated f u r t h e r , possibly


7/10

THE EXPERIMENTER EFFECT 229by deliberately manipulating th e behavior o f E[p. 136].

It is clear from a comparison of these tw ostatements that during th e past 12 years th eprogress in examining and controlling theexperimenter effect has been something lessthan spectacular. Thus, th e objective of thisportion of the present paper is to attempt toalter fur ther research procedure by emphasiz-ing the implications of the experimentereffect as it relates to the individual psy-chologist engaged in his varied activities.Clinical Implications

Clinicians have long recognized the in-fluence of the experimenter (therapist) uponthe behavior of a subject (clien t). In fact , th ediffering views existing in the clinical realmas to the most effective therapeutic procedureto utilize seem to have their origin in theclinician's conception of the role of thetherapist in the therapeutic situation. Forexample, the psychoanalyst believes trans-ference is essential if the client is to be ledto adjustment, whereas th e nondirective thera-pist strives to accompany th e patient alongth e road to adjustment ra ther than to lead.Even though clinicians not only recognizebut argue over th e implementation of theexperimenter influence, they are not exemptfrom a thorough evaluation of the implica-tions (some of which are discussed below)that the experimenter variable holds for theclinical field.Perhaps a reevaluation of the experimentervariable will reveal that pseudodifferencesexist among the effects of various psycho-therapeutic techniques. Goldstein (1962)showed that clients who are rehabilitatedby a particular technique m a y be moreproducts of perceived therapist expectanciesthan of therapeutic techniques. A n essentiallydifferent technique employed by a therapistexpecting good results with his procedure canrehabilitate the client just as completely.Considering th e present state of sophistica-tion in the clinical realm, it is not unreason-able to assume that therapist expectancies arelikely to play a large part in client rehabilita-tion. Thus, th e pseudoproblem m ay tend tothwart intensive searches for valid, opera-tionally defined therapeutic procedures.

Other more specific clinical areas affectedby E-S interaction would include th e effectupon patients' Rorschach scores as a functionof experimenter differences (Lord, 1950), theonce-removed influence of the experiencedclinician's effect upon a neophyte therapist'sprognosis of a patient (Rosenthal , 1963c) ,the possibility that the therapist may be acontr ibut ing factor to the patient 's failure torecover as the result of perceived negativetherapist expectancy (Sanders & Cleveland,1953), and the not-so-alluring possibilitythat patients receiving th e stamp of rehabili-tation have only adjusted to the wishes of thetherapist and n ot necessarily to the em otionalproblems which brought them to the therapistoriginally (Verplanck, 1955) .These are important problems for prac-ticing clinicians, and it is to their credit thatthey have recognized this, as evidenced byth e increasing use of the team approach indiagnosing clients. Utilization of the teamapproach may be extended to therapy inorder to reduce th e negative aspects of theclinician-client interaction.Such a team approach might involve th eobjective assignment of patients to therapistsby means of a large-scale correlational deter-mination of what therapist-patient "person-ality types" interact most effectively in thetherapeutic situation. O f course, th e deter-mination of personality types still leaves uswith all the previously mentioned E -S inter-action problems, but nonetheless attempts atimprovement can be made even if imperfecttools must be used.Implications for the Field of Testing

The general field of testing which wouldinclude IQ tests, placement tests, readingreadiness tests, aptitude tests, etc., is alsobeset with th e problem of the experimentervariable. Even though there have been rigor-ous attempts at standardization of test itemsand procedures in this area, E or adminis-trator of the test still influences th e test takerin other subtle ways (Kanfer , 1958; Rosen-thai, 1963a).The implications of the experimenter effectin th e testing area have m any ramifications.It is questionable whether many tests havebeen proven sufficiently reliable and valid in


8/10

230 KINTZ, DELPRATO , METT EE, P ERSONS, AND SCHAPPEtheir own right, and this problem is fu r thercomplicated by the experimenter variable.Judgment of an individual 's score on specialabilities and IQ tests, etc., must not only beviewed in l ight of which test w as used, butmust also take into consideration the previ-ously ignored variable of the specific adminis-trator. In addition to knowing that a personachieved an IQ score of 105 on the Stan fo rd-Bine t and not the Wechsler, it is also neces-sary to know whether or not E was threaten-ing, docile, fr iendly, anxious , or expected th etest taker to be smart, dumb, score well, etc.(B inder , McConnell , & Sjoholm, 1 9 5 7 ; M c-Guigan, 1963; Rosenthal, 1963a) .

The administrator contamination problemm ay eventual ly be resolved by the applicationof machines to the administration of tests.A t this time a more judicious selection ofth e hundreds of available tests on the partof administrators, using test results to guidetheir decision-making process, is essential. Inaddition, test results should be viewed witha more sophisticated, cri t ical eye, with IQand aptitude scores being considered as butsome of many indices of pe r f o rma nc e . Allpersons using test scores must recognize th estrong inf luence of E and make decisionsaccordingly.Experimental Implications

The psychologist engaged in controlledexperimentation should realize that he hasfailed to provide a control for himsel f . Thatthis variable is disregarded is evidenced byWoods' (1961) investigation of 1,737 pub-lished experiments, of which 42%-45% in -volved multiple authorship. None of theseran an analysis of experimenter interaction.

O ne particular aspect of controlled experi-mental endeavor which has neglected th eexperimenter effect is learning-theory re -search. Much energy is expended on "crucial"experiments which ostensibly attempt to de-te rmine which of the conflicting theories ofHull, Tolmon, Guthr ie , and others are cor-rect. At the present t ime these crucial ex-periments have produced results which aregenerally inconclusive, except for establishinga high correlation between the theory an E'sresults support and his theoretical position.

The experiments already reviewed pro-vide a speculative base for partially explain-in g th e conflicting results obtained by thesupporters of various learning theories. AsRosenthal (19 63c) has shown, experimen terbias is a powerful influence in the experi-mental si tuation. The E has ma ny oppor t u n i-ties to influence, unintentionally, Ss whohave been brought into a very strange, highlyst ruc tured situation. In view of this, it isnot surprisingit should be expectedthatE s favoring a part icular learning theorywould tend to obtain results favoring thissame theory.Results reported recently (Cordaro & Ison,1963; Rosenthal & Fode, 1963a; Rosenthal& Halas, 1962) indicate that Es also affectthe results of studies using nonhuman Ss.These f indings f u r t he r emphasize the pos-sibility that the overlooked experimentervariable may have contaminated many cruciallearning experiments .

This is not to suggest that being able toreplicate studies and/or controll ing the ex-perimenter variable is the panacea for psy-chology's problems. But i t can not be over-emphasized that at the present t ime is apowerful , yet much ignored, variable. It isa s t range paradox that even many of themost adamantly scientific of psychologistshave failed to control for the experimentervariable.

CONCLUSIONSFuture experimentat ion might prove moreprofitable if more r igorous communicat ioncould be established between researchers of

differing points of view and theoretical orien-tations so that a system of research exchangemight be established. This suggestion ad-mittedly presents a mul t i tude of problems,not the least of which would be that of au-thorship c redi t . Al thoug h this and m any otherproblems would arise, they would not bein su rmoun tab le .If research exchange were implemented,it might prove an effective means of con-trolling the experimenter effect and, in ad-dition, bring scientific communication into theprepublication stage of research. This initself might prove to be the most importantcontr ibut ion of all.


9/10

TH E EXPERIMENTER EFFECT 231Other suggestions for control of the experi-menter variable have been given previouslyby Rosenthal (1963c) and McGuigan (1963) .

These suggestions included counterbalancingof Es and the use of factorial designs whichinclude the experimenter as a major inde-pendent variable. Fode (1960; cited inRosenthal , 1963), as reported by Rosenthal(1963c) , found that both visual and audi torycues influenced th e behavior of Ss. Thus,another suggestion involves th e elimination ofverbal and visual cues, including inflectionsof th e voice, speaking pecu liari ties, gestures,etc., as transmitted to 5s dur ing th e readingof instructions.This paper, which began with a discussionof a horse and the subtlety of experimentercues, has ranged far afield. W e have seen thatth e experimenter effect exerts an insidiousinfluence upon th e relationship betweencounselor and client. Indeed, th e more ob-jective and nondirective th e counselor, th egreater th e potential hidden effect. To be un-aware of the relationship between counselorand client expectations is to lose much of thecontrol that a counselor must maintain overthe counseling situation. In the sam e wayteachers must be aware that objective ap-praisal by their students is affected by thegoals which th e students believe their teachershave. And finally, but probably most im -portant at this time, directors of laboratoryresearch who use s tudent Es, must be awareof the extremely great effect of their personalbiases which can be perceived by the s tuden tEs and translated into practically any sig-

nificant experimental effect.REFERENCES

BINDER, A., MCCONNELL, D., & SJOHOLM, N. A.Verbal conditioning as a func t ion of exper imentercharacteristics . Journal of Abnormal and SocialPsychology, 1957, 55, 309-314.BLANKENSHIP, A. B. The effect of the in te rv iewerupon the response in a public opinion poll.Journal of Consulting Psychology, 1940, 4, 134-136.

B R O G D E N , W. J. The exper imenter as a f ac t o r inanimal condi t ioning. Psychological Reports, 1962,11, 239-242.CANTRIL, H., & R E S E A R C H A S S O C IA T E S . Gaugingpublic opinion. Princeton, N. J.: Pr ince ton Un iver .Press, 1944.

C O R D A R O , L., & ISON, J. R. Psychology of thescientist: X. Observer bias in classical conditioning

of the planarian. Psychological Reports, 1963, 13,787-789.EBBINGHAUS, H. Memory: A contribution to ex-perimental psychology. (Orig . publ. 1885; t rans ,by H. A. Ruger & Clara E. Bussenius) New Y ork :Teachers College, Columbia University, 1913.F O D E , K. L. The ef fect of non-visual and non-verbalinteraction on experimenter bias. Unpublishedmaster 's thesis , University of North Dakota, 1960.GOLDSTEIN, A. P. Therapist-patient expectancies inpsychotherapy. N e w Y o r k : Pergamon Press, 1962.G R A H A M , S. R. The influence of therapist characters t r uc tu r e u pon Rorschach changes in the courseof psycho therapy . American Psychologist, 1960,15, 415.G R E E N S P O O N , J. The reinforcing effect of two spokensounds on the frequency of two responses. Amer-ican Journal of Psychology, 1955, 68 , 409-416.GRIFFITH, R. M. Rorschach water precepts : A studyin conflicting results. American Psychologist, 1961,16, 307-311.G R U E N B E R G , B. C. The story of evolution. NewYork : Van Nostrand, 1929.HAMMER, E. F., & PIOTROWSKI, Z. A. Hostility as

a f ac to r in the clinician's personality as it affectshis interpretation of projective drawings. Journalo f Projective Techniques, 1953, 17 , 210-216.HARRIS, H. E., PICCOLINO, E. B ., R O B A C K , H. B ., &SOMMER, D. K. The effects of alcohol on cou n te rconditioning of an avoidance response. QuarterlyJournal of Alcoholic Studies, 1964, 25 , 490-497.

KANFER, F. H. Verbal condi t ioning: Reinforcementschedules and expe r imen tal inf luence. PsychologicalReports, 1958, 4, 443-452.L O R D , E. Exper imental ly induced var iat ions inRorschach per fo rm ance . Psychological Monographs,1950, 64(10, W hole No. 316) .McGuiGAN, F. J. Variat ion o f whole-par t methodsof learning. Journal of Educational Psychology,1960, 51, 213-216.McGuiGAN, F. J. The exper imenter : A neglecteds t imulus object . Psychological Bulletin, 1963, 60,421-428.MARINE, EDITH L. The effect of familiarity with th e

examiner upon Stanford-Binet tes t per formance.Teachers College Contributions in Education, 1929,381, 42 .MASLING, J. The influence of situational and inter-personal variables in projective testing. Psy-chological Bulletin, 1960, 57, 65-85.PFUNGST, O . Der Kluge Hans. (Orig. publ. 1905;t rans , by C. L. Ra hn) New Yo rk: Holt , 1911.POSTMAN, L., & JARRETT, R. F. An experimentalanalysis of learning without awareness . AmericanJournal of Psychology, 1952 , 65 , 244-255.R O S E N T H A L , R . Exper im en te r a t t r ibu tes as determi-nan t s of subjects ' responses. Journal of Projective

Techniques, 1963, 27, 324-331. (a)ROSENTHAL, R . Exper imenter model ing effects as de-t e rm inan t s of subjects ' responses. Journal of Pro-jective Techniques, 1963, 27, 467-471. (b )ROSENTHAL, R. On the social psychology of the psy-chological experiment: The exper imenter ' s hypothe-


10/10

232 KtNTZ, DELPRATO, METTEE, PERSONS, AND SCHAPPEsis as unintended determinant of experimental re-sults. American Scientist, 1963, 51, 268-283. (c )

R O S E N T H A L , R ., & F O D E , K. L. The e f f e c t of experi-menter bias on the p e r fo r m an ce of the albino rat.Behavioral Science, 1963, 8, 183-189. (a)R O S E N T H A L , R ., & F O D E , K. L. Psychology of the sci-entist : V. Three experiments in experimenter bias.Psychological Reports, 1963, 12 , 491-511. (b )R O S E N T H A L , R ., F O D E , K. L., F R I E D M A N , C. J., &

VIKAN-KLINE, L. L. Subjects ' perception of theirexper imenter under condit ions of experimenter bias.Perceptual and Motor Skills, 1960, 11 , 325-331.R O S E N T H A L , R ., & H A L A S , E. S. Experimenter e f f e c tin the s tudy of invertebra te behavior . Psychologi-cal Reports, 1962, 11, 251-256 .R O S E N T H A L , R ., & L A W S O N , R. A longi tud inal s tudy

of experimenter bias on the operant learning ofl abora tory ra t s . Journal of Psychiatric Research,in press.

R O S E N T H A L , R ., & P E R S I N G E R , G. W. Let's p re t en d :Subjects ' perception of imaginary experimenters .Perceptual and Motor Skills, 1962, 14, 407-409.R O S E N T H A L , R., P E R S I N O E R , G. H., & F O D E , K. L.Experimenter bias, anxiety, and social desirability.Perceptual and Motor Skills, 1962, IS , 73-74.R O S E N T H A L , R ., P E R S I N G E R , G. W ., VIKAN-KLINE,

L I N D A , L., & F O D E , K. L. The e f f e c t of early datare turns on data subsequently obtained by outcome-biased experimenters. Sociometry, 1963, 4, 487-498.R O S E N T H A L , R ., P E R S I N G E R , G. W ., VIKAN-KLINE,

L I N D A L., & M U L R A Y , R. C. The role of the re-

search assistant in the mediat ion of experim enterbias. Journal of Personality, 1963, 31, 313-335.

S A N D E R S , R., & C L E V E L A N D , S. E. The relation be -tween certain experimenter personality variablesan d subjects' Rorschach scores. Journal of Pro-jective Techniques, 1953, 17 , 34-50.S A R A S O N , I. G., & MINARD, J. Interrelat ionshipsamong subjects, experimenters, and si tuat ionalvariables. Journal of Abnormal an d Social Psy-chology, 1963, 67 , 87-91.S E V E R I N , F. T., & RIGBY, M. K. Inf luences of digitgrouping on m e m o r y fo r telephone n u m b e r s .Journal of Applied Psychology, 1963, 47 , 117-119.S P E N C E , K. W. Anxiety (drive) level and p e r fo r m -ance in eyelid condit ioning. Psychological Bul-

letin, 1964, 61 , 129-140.S P I R E S , A. M. Subjec t -experimenter in terac t ion inverbal condit ioning. Unpublished doctoral dis-sertation, New York Universi ty, 1960.S T E V E N S O N , H. W ., & A L L E N , S A R A . Adul t perform-ance as a funct ion of sex of experimenter and sexof subject . Journal of Abnormal an d Social Psy-chology, 1964, 68 , 214-216.V E R P L A N C K , W. S. The control of the conten t ofconversation. Journal of Abnormal an d SocialPsychology, 1955, 51, 668-676.WICKES, T. H. Examiner difference in a test si tua-tion. Journal of Consulting Psychology, 1956, 20 ,23-26.W O O D S , P. J. Some characteristics of journals andauthors . American Psychologist, 1961, 16 , 699-701.

(Received May 25, 1964)

The Experimenter Effect

Documents

Transcript of The Experimenter Effect