THE SUPERSTITION EXPERIMENT: A...

Psychological Review1971, Vol. 78, No. 1. 3-43

THE "SUPERSTITION" EXPERIMENT:

A REEXAMINATION OF ITS IMPLICATIONS FOR THEPRINCIPLES OF ADAPTIVE BEHAVIOR '

J. E. R. STADDON2 AND VIRGINIA L. SIMMELHAG3

Duke University Scarborough College, University of Toronto

Replication and extension of Skinner's "superstition" experiment showedthe development of two kinds of behavior at asymptote: interim activities(related to adjunctive behavior) occurred just after food delivery; theterminal response (a discriminated operant) occurred toward the end ofthe interval and continued until food delivery. These data suggest a viewof operant conditioning (the terminal response) in terms of two sets ofprinciples: principles of behavioral variation that describe the origins ofbehavior "appropriate" to a situation, in advance of reinforcement; andprinciples of reinforcement that describe the selective elimination of be-havior so produced. This approach was supported by (a) an account ofthe parallels between the Law of Effect and evolution by means of naturalselection, (fc) its ability to shed light on persistent problems in learning(e.g., continuity vs. noncontinuity, variability associated with extinction,the relationship between classical and instrumental conditioning, the con-troversy between behaviorist and cognitive approaches to learning), and(c) its ability to deal with a number of recent anomalies in the learningliterature ("instinctive drift," auto-shaping, and auto-maintenance). Theinterim activities were interpreted in terms of interactions among motiva-tional systems, and this view was supported by a review of the literatureon adjunctive behavior and by comparison with similar phenomena inethology (displacement, redirection, and "vacuum" activities). The pro-posed theoretical scheme represents a shift away from hypothetical "laws oflearning" toward an interpretation of behavioral change in terms of inter-action and competition among tendencies to action according to principlesevolved in phylogeny.

The field of learning has undergone in- At times, one senses a widespread feeling of dis-creasing fractionation in recent years. In- couragement about the prospects of ever getting, ! . • « • • , . i »> j * clear on the fundamentals of conditioning. Attemptsterest in miniature systems and exact to arrive at firm dedsions abouf alternatfve

theories of local effects has grown _ to the formulations rarely produce incisive results. Everydetriment of any attempt at overall integra- finding seems capable of many explanations. Issuestion. Consequently, as one perceptive ob- become old, shopworn, and disappear without aserver has noted: proper burial [Jenkins' 197°- ™ 107-108^

1 Research was supported by grants from the The present article outlines an attempt toNational Institute of Mental Health, United States redress this imbalance. It is organized

F±L£*£ STunlverS Tndte UnlT -ound the problem of "superstitious" be-sity of Toronto. The authors thank Nancy K. havior, originated by Skinner some yearsInnis for advice and assistance. Janice Frank, agO; which plays a crucial part in the em-Irving Diamond, Carl Erickson, Richard Gil- . . , , ., ,. , , . . . ,bert, and Peter Klopfer kindly commented on Pmcal .and theoretical foundations of CUr-earlier versions of this paper. A shorter version rent views of learning. Discussion of aof the experimental part of this work was pre- replication of Skinner's original experimentsented in partial fulfillment of the requirements for . . . , , i j . - t . - i .the MA degree at the University of Toronto leads to an account of the relationships be-(Simmelhag, 1968). tween evolution and learning, and a system

2 Requests for reprints should be sent to J. E. R. Of classification derived therefrom. TheStaddon, Department of Psychology, Duke Uni- , , • . < .1 . - 1 , rversity, Durham, North Carolina 27706. PaPer concludes with a theoretical account of

3 Now at Duke University. "superstition" and some related phenomena.

J. E. R. STADDON AND VIRGINIA L. SIMMELHAG

THE "SUPERSTITION" EXPERIMENT

In his classic experiment on "supersti-tious" behavior, Skinner (1948) showedthat the mere delivery of food to a hungryanimal is sufficient to produce operant con-ditioning. The pigeons in that experimentwere allowed 5-second access to food every15 seconds, and food delivery was independ-ent of their behavior. Nevertheless, nearlyevery pigeon developed a recognizable formof stereotyped, superstitious behavior thatbecame temporally correlated with food de-livery as training progressed.

Skinner's (1961) analysis of this phe-nomenon is a straightforward application ofthe Law of Effect:

The conditioning process is usually obvious. Thebird happens to be executing some response asthe hopper appears; as a result it tends to repeatthis response. If the interval before the nextpresentation is not so great that extinction takesplace, a second "contingency" is probable. Thisstrengthens the response still further and subse-quent reinforcement becomes more probable[p. 405].

Skinner's observations were quickly re-peated in a number of laboratories. Theapparent simplicity and reliability of thephenomenon, coupled with the plausibilityof Skinner's interpretation of it, and themore exciting attractions of work on re-inforcement schedules then developing, effec-tively stifled further study of this situation.However, both the experiment and his ex-plication played a crucial role in advancingSkinner's theoretical view of operant be-havior as the strengthening of unpredictablygenerated ("emitted") behavior by the auto-matic action of reinforcers.

Two kinds of data obtained in recent yearsraise new questions about "superstition"in this sense. First, experiments with time-related reinforcement schedules have shownthe development of so-called "mediating"behavior during the waiting period, whenthe animal is not making the reinforced re-sponse. Thus, on schedules which requirethe animal to space his responses a few sec-onds apart if they are to be effective in pro-ducing reinforcement (spaced-respondingschedules), pigeons often show activitiessuch as pacing and turning circles. Simi-

larly, on fixed-interval schedules, in whichthe first response t seconds after the pre-ceding reinforcement is effective in produc-ing reinforcement, pigeons may show a simi-lar behavior during the postreinforcement"pause" when they are not making the re-inforced response. Other species showactivities of this sort in the presence of ap-propriate environmental stimuli; for ex-ample, schedule-induced polydipsia, in whichrats reinforced with food on temporal rein-forcement schedules show excessive drinkingif water is continuously available (Falk,1969). None of these activities is rein-forced, in the sense of being contiguous withfood delivery, yet they are reliably producedin situations similar in many respects toSkinner's superstition procedure. Possibly,therefore, some of the activities labeledsuperstitious by Skinner, and attributed byhim to accidental reinforcement of spon-taneously occurring behavior, may insteadreflect the same causal factors as thesemediating activities.

Second, a number of experiments havedemonstrated the development of behaviorin operant conditioning situations by a proc-ess more reminiscent of Pavlovian (classi-cal) conditioning than Law of Effect learn-ing as commonly understood. Breland andBreland (1961) reported a series of observa-tions showing that with continued operanttraining, species-specific behavior will oftenemerge to disrupt an apparently well-learnedoperant response. In the cases they de-scribe, behavior closely linked to food (pre-sumably reflecting an instinctive mechanism)began to occur in advance of food delivery,in the presence of previously neutral stimuli("instinctive drift"). Since these "irrele-vant" activities interfered with food deliveryby delaying the occurrence of the reinforcedresponse, they cannot be explained by theLaw of Effect. A description in terms ofstimulus substitution—a principle usuallyassociated with Pavlovian conditioning—isbetter, although still not completely satis-factory. More recently, Brown and Jenkins(1968) have shown that hungry pigeons canbe trained to peck a lighted response keysimply by illuminating the key for a fewseconds before food delivery. Fewer than a

"SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR

hundred light-food pairings are usually suf-ficient to bring about key pecking. The rela-tionship between this "auto-shaping" pro-cedure and Pavlovian conditioning is furtheremphasized by an experiment reported byWilliams and Williams (1969). They foundthat auto-shaped key pecking is maintainedeven if the key peck turns off the light on thekey and thus prevents food delivery on thatoccasion. All these experiments show theoccurrence of food-related behaviors, inanticipation of food, under conditions moreor less incompatible with the Law of Effect.

The auto-shaping procedure is opera-tionally identical to Pavlovian conditioningwith short delay (the light-food interval inthese experiments is typically 8 seconds).Therefore the eventual emergence of food-related behavior, in anticipation of food de-livery, is not altogether surprising—althoughthe directed nature of key pecking has nocounterpart in principles of conditioning thattake salivation as a model response. Thesuperstition situation is also equivalent to aPavlovian procedure—in this case temporalconditioning, in which the UCS (food) issimply presented at regular intervals. Per-haps, therefore, prolonged exposure to thissituation will also lead to the emergence offood-related behavior in anticipation of food.Possibly the superstitious behavior describedby Skinner includes activities of this sort,that occur in anticipation of food, as well asmediating activities that occur just after fooddelivery.

The present experiment affords an op-portunity to test these ideas. It providescomparative data on the effect of fixed versusvariable interfood intervals on superstitiousresponding (Skinner used only fixed inter-vals), as well as allowing a comparison be-tween response-dependent and response-independent fixed-interval schedules. Theexperiment also extends Skinner's work byrecording in some detail both the kind andtime of occurrence of superstitious activities.The emphasis is on the steady-state adapta-tion to the procedures, but some data on thecourse of development of superstition arepresented.

We hope to show that careful study of thesuperstition situation makes necessary a re-

vision of Skinner's original interpretationand, by extension, requires a shift of empha-sis in our view of adaptive behavior.

MethodSubjects

Six pigeons were used: four white Carneaux,two with experimental experience (Birds 31 and29) and two experimentally naive (Birds 47 and49). Two other pigeons were of a local (Toronto,Ontario) breed and were experimentally naive(Birds 40 and 91). All the birds were main-tained at 80% of their free-feeding weightsthroughout.

ApparatusTwo standard Grason-Stadler operant condition-

ing chambers were used. The response keys werecovered with white cardboard except during theresponse-dependent condition when one key was ex-posed and transilluminated with white light. Datawere recorded by a clock, digital counters, and anevent recorder. Food delivery was controlledautomatically by relays and timers. Behaviorswere recorded via push buttons operated by anobserver. A tape recorder was used to recordcomments and corrections. Except for the pushbuttons and the tape recorder, all programmingand recording apparatus was located in a separateroom. White noise was present in the experi-mental chamber and, together with the noise ofthe ventilating fan, served to mask extraneoussounds.

ProcedureThree schedules of food delivery were used:

(a) A response-independent fixed-interval (FI)schedule in which the food magazine was pre-sented at 12-second intervals, (fc) A response-independent variable-interval (VI) schedule inwhich the food magazine was presented on theaverage every 8 seconds. The following sequenceof interreinforcement intervals was used, pro-grammed by a loop of 16-millimeter film withholes punched at appropriate intervals: 3, 6, 6,12, 9, 7, 3, 10, 21, 6, 5, 11, 8, 5, 3, 9, 7, 9, 5, 13,3, 8, 9, 4, 7, 12, 11, 3, 6, 5, and 9 seconds, (c) Aresponse-dependent FI schedule in which food wasdelivered (reinforcement occurred) for the firstkey peck 12 seconds or more after the precedingreinforcement.

Food delivery involved 2-second access to mixedgrain. Sessions ended after the sixty-fourth fooddelivery and the pigeons were run daily.

Habituation sessions. All the birds were givena number of daily 10-minute sessions when no foodwas delivered; the birds were simply placed in thechamber and their behavior observed and recorded.The birds received the following numbers of suchsessions: Bird 31, 3; Bird 29, 3; Bird 47, IS;Bird 49, 15; Bird 40, 7; Bird 91, 7. Note that the


experimentally naive birds received more habitua-tion exposure.

Response-independent training. Four of thepigeons were then given a number of sessions oneach of the two response-independent procedures,either FI followed by VI, or the reverse. Thebirds received the procedures in the indicatedorder (number of sessions in parentheses) : Bird31: FI 12 (26), VI 8 (111); Bird 29: VI 8 (26),FI 12 (109) ; Bird 47: VI 8 (36), FI 12 (36) ;Bird 49: FI 12 (37), VI 8 (36).

Response-dependent training. Two of the birdsalready trained on the response-independent pro-cedures were then switched to the response-dependent FI 12 for the following numbers ofsessions: Bird 31, 37; Bird 47, 52. The twonaive birds (40 and 91), after their habituationsessions, were given one session of response-inde-pendent FI 12 when the food magazine operatedevery 12 seconds, but contained no grain. Thiswas to habituate these birds to the sound of themechanism. The next day these two birds wereplaced in the experimental box with the food maga-zine continuously available for 10 minutes. Thefollowing day the two birds were introduced tothe response-dependent FI 12-second schedule. Asmall piece of black tape was placed on the lightedresponse key, as an inducement to pecking, for thisfirst session only. All the birds pecked the keyduring the first session of the response-dependentprocedure. This rather elaborate key-training pro-cedure was designed both to prevent the experi-menter from shaping the naive birds' behavior, andto avoid the possibility of "superstitious" condi-tioning, which might be entailed by some formof auto-shaping. Bird 40 received a total of 45sessions of the response-dependent FI 12-secondschedule, Bird 91 received 38.

Response description and scoring. Responsecategories were arrived at on the basis of initialobservation during the habituation sessions andwere altered as necessary to accommodate newbehaviors. The names, descriptions, and numbersof the categories appear in Table 1. Responseswere scored (by pushing the appropriate button)in two ways, either discretely or continuously.If a response tended to occur in discrete units(e.g., pecking), then the appropriate button waspushed each time an instance of the responseoccurred. The observer was the same throughout(VLS), and the maximum recordable rate fordiscrete responses was 3-4 per second. A con-tinuous response is one which took an indefiniteamount of time (e.g., facing magazine wall) ; theappropriate button was pressed throughout theduration of a continuous response. Discrete re-sponses were pecking (wall, key, or floor) andquarter circles, all the rest were continuous re-sponses. In general, the response categories weremutually exclusive. The only exception is facingMagazine wall (Ri), which at various times oc-curred with Flapping wings (Rs), Moving alongmagazine wall (Rs), and Pecking (R?).

ResultsThe data of interest in this experiment are

the kind and amount of behavior at differentpoints in time following the delivery of food.As training progressed, a systematic patternof behavior as a function of postfood timebegan to emerge. The properties of thissteady-state pattern for VI and FI schedulesis discussed first, followed by a descriptionof the changes that took place during acquisi-tion.

Steady-State BehaviorIn the steady state, the behavior developed

under both the FI and VI procedures fellreliably into two classes: (a) The terminalresponse was the behavior that consistentlyoccurred just before food delivery. It be-gan 6-8 seconds after food delivery on theFI procedures, and about 2 seconds afterfood on the VI procedure, and usually con-tinued until food delivery, (b) A numberof activities usually preceded the terminalresponse in the interval. These activities areprobably indistinguishable from what hasbeen termed mediating behavior, but we pre-fer the more descriptive term interim activi-ties. These activities were rarely con-tiguous with food.

Figure 1 shows the performance averagedacross three sessions of steady-state respond-ing under all conditions for all the pigeons.The left-hand panels show the response-de-pendent FI schedule; the middle panels, theresponse-independent (superstitious) FIschedule; and the right-hand panels, the re-sponse-independent VI procedure, for thefour birds exposed to each. The graphsshow the probability (relative frequency)with which each of the activities occurredduring each second of postfood time. Eachbird shows the clear division between termi-nal and interim activities already alluded to.Excluding R1; and the results for Bird 29on VI, Pecking (R7) was the terminal re-sponse for all the response-independent pro-cedures. For Bird 29, the terminal responseHead in magazine (Ru) became an interimactivity following the shift from VI to FIand was replaced as terminal response byPecking. Pecking remained the terminalresponse for this bird throughout a sequence

'SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR

TABLE 1

DESCRIPTION OF OBSERVED ACTIVITIES

Response no. Name Description

R3R4

Re

Rr

Rs

Rio

Rn

Rl2

Ris

Ru

Rl5

Rl6

Magazine wall

Pecking keyPecking floorJ circle

Flapping wingsWindow wall

Pecking

Moving along magazine wall

Preening

Beak to ceiling

Head in magazine

Head movements along magazinewall

Dizzy motion

Pecking window wall

Head to magazineLocomotion

An orientation response in which the bird's head andbody are directed toward the wall containing themagazine.

Pecking movements directed at the key.Pecking movements directed at the floor.A response in which a count of one \ circle would be

given for turning 90° away from facing the magazinewall, a count of two for turning 180° away, three for270°, and four for 360°.

A vigorous up and down movement of the bird's wings.An orientation response in which the bird's head and

body are directed toward the door of the experi-mental chamber containing the observation window.

Pecking movements directed toward some point on themagazine wall. This point generally varied betweenbirds and sometimes within the same bird at dif-ferent times.

A side-stepping motion with breastbone close to themagazine wall, a few steps to the left followed by afew steps to the right, etc. Sometimes accompaniedby (a) beak pointed up to ceiling, (6) hopping,(c) flapping wings.

Any movement in which the beak comes into contactwith the feathers on the bird's body.

The bird moves around the chamber in no particulardirection with its beak directed upward touching theceiling.

A response in which at least the beak or more of thebird's head is inserted into the magazine opening.

The bird faces the magazine wall and moves its headfrom left to right and/or up and down.

A response peeuliar to Bird 49 in which the head vi-brates rapidly from side to side. It was apparentlyrelated to, and alternated with, Pecking (Rr).

Pecking movements directed at the door with the ob-servation window in it.

The bird turns its head toward the magazine.The bird walks about in no particular direction.

of response-independent procedures after theones reported here, lasting for a total inexcess of 90 sessions. A curious idiosyn-cratic head movement accompanied peckingby Bird 49 (R13:Dizzy motion) on response-independent FI, although it disappeared fol-lowing the switch to VI. The locus of Peck-ing on the magazine wall differed from birdto bird and varied both across and withinsessions for some birds. The stable featuresof this response were its topography and itsrestriction to the general area of the maga-zine wall. A variety of interim activitiesoccupied the early parts of the FIs and theperiod within 2 or 3 seconds after food on

the VI schedule: Pecking floor (R3), icircles (R4), Flapping wings (RB), Movingalong magazine wall (R8), and Beak to ceil-ing (R10) were the most frequent. Theinterim activities were therefore more vari-able from bird to bird than was the terminalresponse.

The pattern of behavior characteristic ofeach interval was little affected by whetheror not food was dependent on key pecking.The similarity between the patterns duringresponse-dependent and response-independ-ent FI is particularly striking for Bird 47,who was switched directly from the re-sponse-independent to the response-depend-


RESR-DEP. Fl RESR-INDER Fl R E S R - I N D E R V I

» ' 12 0 2 4 ' 6 8 10 ' 12 ' 14 ' 16 ' 18

2 4 6 8 10 12 0 2 4 6 8 10 12 0 2 4 6 8 10 12 14 16 18

2 4 6 8 10 12 0 K> 12 14 16 18 20

SECOND IN INTERVALFIG. 1. Probability of each behavior as a function of postfood time for all birds

for all three experimental conditions, averaged over three sessions of steady-stateresponding under each condition. (Each point gives the probability that a givenbehavior occurred in that second of postfood time. Data for the response-dependentcondition are averaged across 2-second blocks. Behaviors (R») are identified inTable J,)

'SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR

ent procedure. Bird 31, for whom the re-sponse-independent and response-dependentFIs were separated by response-independentVI, shows more variation in the interimactivities under the two conditions. Birds91 and 40, who were exposed only to theresponse-dependent FI, show a similar pat-tern of interim activities to the response-independent birds, although again there issome variation as to details. This similaritycannot be attributed to the procedure usedto shape key pecking (see Procedure). Afterthe imposition of the key-pecking contin-gency, Bird 31 retained the old Pecking re-sponse as an interim activity restricted to aperiod in the interval just before key peck-ing. Bird 47 showed a similar effect of hisresponse-independent experience in that ahigh proportion of his key pecks failed todepress the key sufficiently to activate theautomatic recording circuitry, although theywere recorded as key pecks by the observer.Overall, these data provide no evidence forsubstantial changes in the pattern of terminalor interim activities traceable to the imposi-tion of a key-pecking requirement.

The general pattern of terminal and in-terim activities during the response-inde-pendent VI schedule was similar to the FIprocedures. Differences were restriction ofthe interim activities to the first 2 or 3 sec-onds of postfood time rather than to the first6 or 7 seconds (with one exception, to be dis-cussed), and the smaller number of interimactivities, in most cases. Under both fixedand variable procedures, once the terminalresponse began in an interval, it continueduntil food delivery. The exception is Bird47 who showed a drop in the probability ofthe terminal response (Pecking, R7) accom-panied by a transient increase in the interimactivity of i circles (R4) at the 14-15-second postfood time. A similar slight dropin the probability of Pecking, although notaccompanied by an interim activity, was alsoshown by Bird 31. These differences are re-lated to the properties of the VI schedule(see Discussion).

Sequential Structure

Figure 2 indicates something of the se-quential structure of the behavior occupying

each interfood interval for each bird duringthe response-independent procedures. Thefigure summarizes the two to five behaviorsequences that account for most of the inter-vals during the three steady-state sessions.For simplicity, no account is taken either ofinterbehavior times or of the duration ofeach activity in this method of representa-tion. The most striking characteristics ofthese sequences are: (a) that each birdshowed only a small number of typical se-quences (usually three or four); (&) thatthe sequencing was very rigid, so that al-though a given behavior might fail to occurduring a particular interval, it never occur-red out of sequence—this is indicated by theabsence of return arrows ("loops") inthese diagrams; and (c) that the variabilityof the sequences was greatest early in theinterval and least at the end, in the periodjust preceding food delivery—this is indi-cated by the absence of "forks" (ambiguoustransitions from one behavior to two or moreothers, as in the diagram for Bird 31 whereR6 —» R5 or R6 -» R7 with approximatelyequal probability) late in the sequence of be-haviors shown in the diagrams. This reg-ular sequencing did not occur early in train-ing, as indicated in Figure 4, discussedbelow.

An inviting possibility raised by these reg-ular sequences is that this behavior may bedescribed by some kind of Markov chain(cf. Cane, 1961). Although the argumentcannot be presented in full here, this assump-tion cannot be sustained for a number ofreasons, the most important of which are(a) that the duration of a bout of a givenactivity was shorter the later the activitybegan within an interval, and (&) that thetime between two successive activities wasshorter the later in the interval the firstactivity ended. These and other considera-tions suggest that postfood time was themost important factor controlling both theonset and offset of each activity in the se-quence.

Acquisition

Figure 3 shows acquisition data for naiveBird 49, through and beyond the periodwhen his behavior became stable on re-

10 J. E. R. STADDON AND VIRGINIA L. SIMMELHAG

FIXED-INTERVAL VARIABLE-INTERVAL

183/192 Intl.

f .16 x̂ l-° * i«mi«ne«i

2 ••qutoett

179/192 loti.

169/192 Int..

^-*-R3-.21

.32 /&*-Vt

^5~.io "

.17

Bird 49

" ;ssv.— •"— Cx.

.94**^ 1.0 3 l.qulnc.i-*-R10 »-R7 »-rSf .* 176/192 Intl.

FIG. 2. Steady-state sequences: Sequential relationships among behaviors during the lastthree sessions of each response-independent procedure for all four pigeons. (Fixed-interval ison the left, variable-interval on the right. Numbers give probabilities of the indicated transi-tions. Number of different sequences on which the diagrams are based and number of intervals(out of 192) accounted for by these are indicated. "F" is food delivery, and behaviors (Ri)are identified in Table 1. Note: probabilities at each "fork" do not always sum to 1.0because not every sequence is accounted for.)

spouse-independent FI. The graphs show(as a function of sessions) the probabilityof the various behaviors in each of thesix 2-second periods making up the FI.The most noteworthy characteristic of acqui-sition is the relatively sudden disappearanceof the behavior Head in magazine, which wasalmost the only behavior to occur duringthe first few sessions for Bird 49, in favorof the terminal response Pecking on maga-zine wall. For Bird 49 this transition tookplace between the seventh and eighth ses-sions of FI without any prior history ofpecking during earlier sessions. Once peck-ing became established as a terminal re-sponse, the only further change was a slowdecline in the probability of the responsedviring early parts of the interval (e.g., be-tween 3 and 8 seconds). The other birdsshowed similar results on their first exposureto the response-independent procedures;for each bird, pecking on the magazine wall

first occurred during the following sessions:for Bird 31, on the first FI session; Bird 29,twelfth session of FI following the switchfrom response-independent VI; Bird 47,twenty-eighth session of VI; Bird 49, eighthsession of FI. Thus, although one of the ex-perienced birds pecked during the first re-sponse-independent session (31), the otherdid not until being switched to a differentschedule (29). No bird shifted to a differentterminal response once he began pecking;the total number of sessions of response-independent experience (FI and VI) foreach of the four birds following the onsetof pecking was as follows: Bird 31, 140;Bird 29, 96; Bird 47, 45; Bird 49, 66. ForBird 29, the terminal response during theresponse-independent FI procedure (i.e., be-fore the onset of pecking) was Head inmagazine. As with the other birds, oncepecking appeared it was in full strengthalmost immediately and further experience

"SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR 11

_••,••« • ••••»•« i

o. p-o-" '̂ •. o-°-<*. .0-0-0V V

HEAD IN MAGo o PECKING MAG. WALL

1/4 CIRCLES

.10-12

i ' T I 4 " ' 'ib'' "i^' " '26 " 2^'' 30 35SESSIONS

FIG. 3. Development of the terminal response, (The graph shows, forBird 49 on the response-independent fixed-interval procedure, the transitionfrom Head in magazine (Ru) to Pecking (R7) as terminal response, andincludes one interim activity, i circles (R<), for comparison. Each panelcovers 2 seconds of the 12-second interval and indicates the number ofintervals (out of 64) in which a given response occurred in that 2-secondblock for each session over the first 36 sessions. Gaps indicate days forwhich data are not available. Bird 49 was not run for a 9-day periodbetween Sessions 21 and 22.)


FIXED - INTERVALBird 31

19 sequences

160/188 tuts.

Bird 49

3 sequences

161/192 Ints.

VARIABLE - INTERVALBird 29

25 sequences

179/192 Ints.

4 sequences

179/192 Ints.

FIG. 4. Sequence data, in the same form asFigure 2, for the first three sessions under thefirst response-independent procedure to which eachbird was exposed. (Details as in Figure 2.)

had the effect merely of restricting it moreto the later parts of the interval.

Figure 4 shows sequence data, in thesame form as Figure 2, for the first threesessions of the first response-independentschedule to which each bird was exposed.Approximately the same proportion of thetotal number of sequences observed within192 intervals is accounted for by these dia-

grams as in the diagrams of Figure 2 forthe last three sessions. The data for thefirst three sessions are much more variable,showing repetitions of a behavior within aninterval and reversals of sequence; threeout of four birds show no single terminalresponse. Thus, the regularities apparentin Figure 2 are evidently a real effect oftraining and not simply an artifact of thismethod of representation.

DiscussionThe results of this experiment confirm the

suggestion that the "superstition" situationgenerally produces two distinct kinds of ac-tivity: interim activities that occur at shortand intermediate postfood times, and theterminal response that begins later in theinterval and continues until food delivery. Itis not always clear from Skinner's originaldiscussion just which kind of behavior hewas observing. In one case, he briefly de-scribes an experiment in which the interfoodinterval was 60 seconds, and the "super-stitious" response (a "well-defined hoppingstep from the right to the left foot") wasautomatically recorded. In this case, thebehavior was evidently a terminal response,since it occurred with increasing frequencythrough the interval:The bird does not respond immediately after eating,but when 10 or IS or even 20 sec. have elapsedit begins to respond rapidly and continues untilthe reinforcement is received [Skinner, 1961, p.406].

On other occasions, however, Skinner mayhave been observing interim activities, as,in our experience, they are sometimes muchmore striking than the terminal response,especially early in training when they mayinclude actions like jumping in the air,vigorous wing flapping, and idiosyncratichead and limb movements. We have alsosometimes observed interim activities duringsessions when there was no obvious terminalresponse.

Nature of the Terminal ResponseThe data from both FI and VI schedules

of food presentation indicate that the prob-ability of the terminal response at differentpostfood times was a function of the prob-


ability of food delivery at those times: theprobability of the terminal response in-creased just in advance of increases in theprobability of food delivery, and decreasedjust after decreases in that probability.Thus, on the FI schedule, the terminal re-sponse began at postfood times greater thanabout 8 seconds, corresponding to a prob-ability of food delivery of zero at times lessthan 12 seconds, and one thereafter. Onthe VI schedule, probability of food deliverywas zero at postfood times less than 3 sec-onds (the shortest interfood interval in theVI sequence), and increased thereafter.Correspondingly, the birds began their ter-minal response after 2 seconds or so of post-food time and continued until food delivery.The data for Birds 29 and 47 on the VIschedule provide additional confirming evi-dence. Both birds showed a small declinein the probability of the terminal responsebetween Seconds 12 and 17 of postfood time,accompanied by a brief reappearance of aninterim activity (between Seconds 13 and16) in the case of Bird 47. This brief de-crease in the terminal response, and accom-panying reappearance of an interim activity,corresponds to the zero probability of foodin the region of postfood time between 13and 21 seconds (because the VI sequenceused had no interfood interval between 13and 21 seconds in length—see Method).Catania and Reynolds (1968) showed asimilar relationship between rate of keypecking and probability of reinforcement, asa function of postreinforcement time, on avariety of conventional (response-depen-dent) VI reinforcement schedules. Thissimilarity and the similarities between theresponse-independent and response-depen-dent conditions of this experiment emphasizethat the terminal response must be regardedas a discriminated operant in Skinner's(1958) sense.

Herrnstein (1966) reported an experi-ment in which the rate of key pecking onresponse-independent FI 11 seconds waslower than on response-dependent FI 11.However, in the present experiment, con-sidering just the terminal response of Peck-ing on magazine wall (RT), we found noevidence for a difference in either probability

or rate of response favoring the response-de-pendent procedure. Indeed, rate of keypecking (denned as switch operations)under the response-dependent FI was gen-erally lower than the observer-defined rateof pecking under the response-independentFI because many pecks failed to break theswitch contact. On the other hand, becausethe location of pecking was much morevariable under the response-independentcondition, a comparison between key peck-ing (in the response-dependent condition)with pecking on a comparable area of themagazine wall (in the response-independentcondition) would show more respondingunder the response-dependent condition.This result raises the possibility that theeffect of the response-dependency, inoperant conditioning experiments using in-terval reinforcement schedules, may belargely one of determining (perhaps imper-fectly) the location of pecking, rather thaneither its form or its frequency of occurrence.

Nature of the Interim Activities

Falk (1969, 1970) has coined the termadjunctive behavior for a variety of activitiesthat are induced in a number of differentanimal species (rats, pigeons, monkeys,chimpanzees) by intermittent schedules ofreinforcement. These activities generallyoccur just after reinforcement, when the re-inforced (i.e., terminal) response is notoccurring. However, the important variablein determining their temporal location seemsto be the low probability of further reinforce-ment in the immediate postreinforcementperiod (rather than the interruption of eat-ing), since they also occur following briefstimulus presentations on second-order inter-val schedules (Rosenblith, 1970), followingeach response on spaced-responding (dif-ferential reinforcement of low rates) sched-ules (Segal & Holloway, 1963), during time-out periods (Wiittke, 1970), only after thelast pellet when a number are delivered con-secutively at the end of each FI (Keehn,1970), and later in the interval during longFI schedules (Segal, Oden, & Deadwyler,1965). They are not elicited (e.g., byfrustration) in any obvious sense, since theytake some time to develop (Reynierse &


Spanier, 1968). They are related to moti-vational systems, since a number of activ-ities—for example, polydipsia (excessivedrinking), pica (eating nonfood material),wheel running, and schedule-induced aggres-sion—may occur more or less interchange-ably depending on the presence of appro-priate stimuli; and animals will learn anoperant response in order to obtain an op-portunity to engage in one of these activities(Falk, 1970).

The interim activities in the present ex-periment appear to reflect the same causalfactors as adjunctive behavior: they occurat times when reinforcement is not availableand the terminal response is not occurring;they occur on both response-dependent andresponse-independent schedules (cf. Azrin,Hutchinson, & Hake, 1966; Burks, 1970;Flory, 1969); and they occur on both FIand VI schedules. Adjunctive behavior re-quires appropriate stimuli (water for poly-dipsia, wood shavings for pica, etc.) to whichit is directed and by which it can be modified,within limits. This is unlikely to be a crucialdifference, however, since appropriate stim-uli were not available in the present experi-ment (had water been available, schedule-induced drinking would almost certainlyhave been observed in lieu of the interimactivities; cf. Shanab & Peterson, 1969),and behavior resembling interim activitieshas occasionally been reported under condi-tions when most animals show adjunctivebehavior, thus

[this rat] was atypical in that it did not developpolydipsia . . . exhibiting instead a number ofstereotyped behaviors, like rearing and runningto the corners of the experimental chamber, be-tween reinforcements [Keehn, 1970, pp. 164-167].

We return to a theoretical account of interimand adjunctive behavior in the concludingsection.

An analogy can be drawn between theterminal and interim activities here and theclassical dichotomy between consummatoryand appetitive behavior (Craig, 1918).Thus, pecking, the stable terminal response,is a food-elicited (consummatory) activityin pigeons, and the interim activities werequite variable, as might be expected of appe-titive behavior. Moreover, the variability of

the sequences (as measured by the numberof "forks" in the sequence) was greatest atthe beginning of the sequence and decreasedtoward the end (cf. Figure 2). More orless unlearned sequences terminating in con-summatory acts show a similar reductionin variability toward the end of the sequence(e.g., Morris, 1958).

In a similar vein, Falk (1970) has com-pared adjunctive behavior with displace-ment activities (Tinbergen, 1952) :

In both adjunctive behavior and displacement ac-tivity situations, the interruption of a consumma-tory behavior in an intensely motivated animal in-duces the occurrence of another behavior imme-diately following the interruption [p. 305].

At the present stage of knowledge, thesecomparisons do little more than group to-gether a number of puzzling phenomena thatcannot as yet be convincingly explained,either by ethological principles or by theLaw of Effect. We can choose either toaccept these behaviors as anomalies withinour present conceptual system, hoping thatfurther research will show how to reconcilethem, or revise the system in a way that willaccommodate them more naturally. Thefirst alternative is becoming increasinglyhard to maintain, as it becomes clear thatthese behaviors are of wide occurrence, andas they continue to resist attempts to explainthem in conventional terms. Polydipsia isthe most widely studied adjunctive behavior,and Falk (1969) summarizes research re-sults as follows:It is not explicable in terms of any altered stateof water balance initiated by the experimental con-ditions. It cannot be attributed to adventitiousreinforcing effects. Since it cannot be related toabnormal water losses, chronic internal stimulationarising from unusual states . . . or from injuryto the central nervous system, the overdrinkingwould be classified clinically as primary or psy-chogenic polydipsia [p. 587].

A revision of the conceptual foundations ofoperant behavior, which will deal naturallywith these behaviors, as well as guide re-search into more profitable channels, seemscalled for.

Development of the Terminal ResponseThe development of the terminal response

provides some clues toward an alternative


conception. Changes in the terminal re-sponse throughout acquisition rule out anunqualified application of the Law of Effectas a description of the process. For ex-ample, three of the four birds made the re-sponse Head in magazine at a higher fre-quency and for a larger fraction of the totaltime than any other response, for the firstfew sessions of exposure to the superstitionprocedure (either FI or VI, depending onthe bird). This behavior was also the onemost often contiguous with the delivery offood. Yet during later sessions, it droppedout abruptly and was replaced by Pecking.Thus the development of Pecking as terminalresponse resembles the findings of Williamsand Williams and of Breland and Breland,much more than it does the strengtheningof an emitted response, in Skinner's terms,by the automatic action of food as a rein-forcer. In all these cases, the presentationof food at a predictable time resulted, aftertraining, in the regular occurrence of a food-related behavior in anticipation of food de-livery, quite independently of the demandcharacteristics of the situation (i.e., thereinforcement schedule). Given that foodis a stimulus that elicits pecking by pigeons,the appearance of pecking, in anticipation offood, in the present experiment is an instanceof the principle of stimulus substitution (Hil-gard & Marquis, 1940). The stimulus is,of course, a temporal one (postfood time),and the situation is analogous, therefore, toPavlovian temporal conditioning (Pavlov,1927), both operationally and in its con-formity to the substitution principle. There-fore, it is tempting to attribute the appear-ance of pecking to a classical conditioningmechanism, and leave it at that.

A number of considerations suggest thatthis explanation will not suffice, however:(a) The behavior of one bird (29) is a par-tial exception. He showed a terminal re-sponse (Head in magazine) different fromPecking, although this response was meta-stable (Staddon, 1965), in the sense thatit was displaced by Pecking following aschedule shift. In addition, Skinner's resultsand informal observations in a number oflaboratories indicate that the superstitionprocedure may generate behaviors other than

pecking that persist for considerable periodsof time. Presumably, many of these be-haviors are terminal responses, in our sense,and, metastable or not, they cannot be dis-missed in favor of a Pavlovian account of allterminal responses in superstition situations.(b) The results of Rachlin (1969), who wasable to obtain spontaneous key pecking bymeans of the auto-shaping procedure ofBrown and Jenkins, but using electric shock-reduction rather than food as the reinforcer,also complicate the picture, since it is notclear that pecking is elicited by shock, as itis by food. The results of Sidman andFletcher (1968), who were able to autoshape key pushing in rhesus monkeys, arealso not readily explicable by stimulus sub-stitution, (c) Williams and Williams(1969) note that the directed nature of thekey peck in the auto-shaping situation is notreadily accommodated within a Pavlovianframework:

the directed quality of the induced pecking doesnot follow naturally from respondent principles(see also Brown and Jenkins, 1968). It is un-clear, for example, why pecking would be directedat the key rather than the feeder, or indeed whyit would be directed anywhere at all [p. 519],

A similar objection can be raised here, sincethe terminal response of pecking was alwaysdirected at the magazine wall, although it isimportant to notice that this objection ismore damaging to an interpretation in termsof classical conditioning than one couchedsimply in terms of stimulus substitution.(d) Finally, there is the problem of theskeletal nature of pecking. Ever sinceSkinner's (1938) original suggestion, it hasbecome increasingly common to restrict theconcept of classical conditioning to auto-nomically mediated responses. Some cri-ticisms of this convention are presented be-low, and Staddon (1970a) has arguedagainst the operant/respondent and emitted/elicited dichotomies. For the moment, letit be said that the objection to the skeletalnature of the response is only a problem foran interpretation of the development of peck-ing in terms of classical conditioning, astraditionally conceived. It does not conflictwith a stimulus substitution interpretation.


In summary, it is clear that while the prin-ciple of stimulus substitution describes thedevelopment of the terminal response in amajority of cases, and is more successfulthan an appeal to Pavlovian principlesmodeled on the salivation reference experi-ment, it is not adequate as a universalaccount. We turn now to the possibility ofa general scheme to deal with these anoma-lous facts.

EVOLUTION AND LEARNING

Objectively considered, the subject mat-ter of the psychology of animal learning-—•the behavior of animals—is a part of biology.This commonality does not extend to termsand concepts, however. Although theethologists have investigated unlearned be-haviors in a variety of species, learning re-mains almost exclusively the possession ofpsychologists. Consequently, the theoreticalfoundations of the study of learning, such asthey are, have evolved almost independentlyof biology. It is now 15 years since Ver-planck (1955) wrote: "the structure of thetheory of unlearned behavior and that oflearned behavior must prove to be similarif not identical [p. 140]," but little progresstoward a unified set of concepts has beenmade. Yet the facts discussed in the pre-vious section seem to demand an interpreta-tion within the context of adaptive behavioras a whole.

The principles of evolution by natural se-lection provide a unifying framework forbiology. Pfaffman (1970) has recentlycommented:

I am impressed by the extent to which evolution,the genetic machinery, and biochemistry providefor biologists a common language and unity oftheory that overrides the molecular versus orga-nismic debate within biology. In contrast, it isobvious that there is no unified theory of behaviorfor all students of behavior [p. 438].

These considerations—the commonality ofsubject matter between biology and animalpsychology, the probable common basis oflearned and unlearned behavior, the unifyingrole of evolutionary processes in biology—all suggest the application of evolutionaryprinciples to the psychology of learning inanimals. In recent years, several papers

have drawn attention to the similarities be-tween evolution and learning (e.g., Breland& Breland, 1966; Broadbent, 1961; Gilbert,1970; Herrnstein, 1964; Pringle, 1951;Skinner, 1966a, 1966b, 1969), but no ver-sion of the evolutionary approach has provedinfluential as yet. The main reasons forthis failure, perhaps, have been the compara-tive effectiveness of traditional versions ofthe Law of Effect in dealing with the limitedphenomena of laboratory learning experi-ments and the lack of a substantial body offacts clearly in conflict with accepted theory.To some extent, of course, the second factorreflects the first—although it would be cyni-cal to speculate that the kind of experimentspsychologists do are often such as to pre-clude data that might go beyond currenttheory. In any event, neither of these rea-sons now holds true.

The growing number of facts on auto-shaping, instinctive drift, adjunctive, andsuperstitious behaviors are not readilyaccommodated within traditional views. Inaddition, the history of the Law of Effectas a principle of acquisition (as opposed tothe steady state, i.e., asymptote) has not beena distinguished one. Little remains of theimpressive edifice erected by Hull and hisfollowers on this base. We are no longerconcerned with the production of learningcurves, nor with the measurement of habitstrength or reaction potential. Hulliantheory has proven effective neither in theelucidation of complex cases nor as an aidto the discovery of new phenomena. In-deed, the opposite has generally been true:new learning phenomena such as schedulesof reinforcement, learning, and reversal sets,etc. have typically been the result of unaidedcuriosity rather than the hypothetico-deduc-tive method, as exemplified by Hull and hisstudents. This represents a failure of Hul-lian theory rather than a general indictmentof the hypothetico-deductive method, whichhas proven every bit as powerful as Hullbelieved it to be—when used by others (e.g.,Darwin, cf. Ghiselin, 1969). A similar,although less sweeping, verdict must behanded down on stochastic learning theory,which represents perhaps the most directattempt to translate the Law of Effect into


a quantitative principle of acquisition. Withthe exception of predictions about the steadystate, such as Estes' ingenious deduction ofprobability matching (Estes, 1957), theearly promise of this approach as a way ofunderstanding the behavior of individualanimals has not been fulfilled. It seems fairto say that the rash of theoretical elaborationthat has occupied the years since Thorndikefirst stated the Law of Effect has told usalmost nothing more about the moment-by-moment behavior of a single organism ina learning situation. Although the alterna-tive we will offer may be no better thanearlier views, it does accommodate theanomalies we have discussed—and the needfor some alternative can hardly be ques-tioned.

The close analogy between the Law ofEffect and evolution suggests an approachthat may be a step toward the general frame-work which the psychology of learning soobviously lacks. At the present stage ofknowledge, this approach is simply an anal-ogy, although a compelling one, and cannotyet be called an evolutionary theory of learn-ing. However, it provides a beginning andmay lead to a truly evolutionary accountwhich can securely imbed the study of learn-ing in the broader field of biology.

Behavioral Variation and Reinforcement

The Law of Effect, suitably modified totake account of advances since Thorndike'soriginal formulation, can be stated so as toemphasize acquisition, or the steady state,and learning (e.g., S-R bonds), or per-formance. We will take as a point of de-parture a neutral version of the law thatemphasizes performance in the steady state,as follows: "If, in a given situation, a posi-tive correlation be imposed between someaspect of an animal's behavior and the de-livery of reinforcement, that behavior willgenerally come to predominate in that situa-tion." The term "correlation" is intended toinclude cases of delay of reinforcement andexperiments in which the response acts onthe rate of reinforcement directly (e.g.,Herrnstein & Hineline, 1966; Keehn, 1970).This formulation does not take account ofmore complex situations where more than

one behavior and more than one correla-tion are involved (i.e., choice situations), asthese complications do not affect the presentargument. A discussion of three aspects ofthis law—(a) the initial behavior in thesituation before reinforcement is introduced,(£>) the process whereby this behavior istransformed into the dominant reinforcedbehavior, and (c) reinforcement—follows:

(a) The behavior in a situation beforethe occurrence of reinforcement reflects anumber of factors, including past experiencein similar situations (transfer), motivation,stimulus factors (e.g., novel or sign stimuli),and others. We propose the label "prin-ciples of behavioral variation" for all suchfactors that originate behavior. These prin-ciples are analogous to Darwin's laws ofvariation, corresponding to the modern lawsof heredity and ontogeny that provide thephenotypes on which selection (analogousto the principles of reinforcement, see be-low) can act. Thus, the term "variation"is intended to denote not mere variability,but the organized production of novelty, inthe Darwinian sense.

(b) Transition from initial behavior tofinal behavior is the traditional problem oflearning theory and, as we have seen, isessentially unsolved at the level of the indi-vidual organism. However, it is at thispoint that the analogy to the mechanism ofnatural selection becomes most apparent.Broadbent (1961) makes the parallel quiteclear:

Since individual animals differ, and those withuseful characteristics [with respect to a particularniche] survive and pass them on to their children,we can explain the delicate adjustment of eachanimal's shape to its surroundings without requir-ing a conscious purpose on the part of the Polarbear to grow a white coat. Equally each indi-vidual animal [under the Law of Effect] triesvarious actions and those become more commonwhich are followed by consummatory acts [i.e.,reinforcement] [p. 56].

Thus, the transition from initial to final be-havior can be viewed as the outcome of twoprocesses: a process that generates behavior,and a process that selects (i.e., selectivelyeliminates) from the behavior so produced.Since there is no reason to suppose that theprocess which generates behavior following


the first and subsequent reinforcements isdifferent in kind from the process that gener-ated the initial behavior (although theeffects will generally be different, see Ex-tinction, below), we can include both underthe head of principles of behavioral variation.We propose the label "principles of rein-forcement" for the second, selective process.

(c) As we have seen, the Darwinianprinciple of selection is analogous to theprocess that transforms initial behavior intofinal behavior—the "principles of reinforce-ment." The notion of reinforcement (moreexactly, the schedule of reinforcement) itselfis in fact analogous to an earlier concept, onethat preceded evolution by natural selectionand can be derived from it: the Law of Con-ditions of Existence, that is, the fact thatorganisms are adapted to a particular niche.This is apparent in the form of the state-ments : "The Polar bear has a white coatbecause it is adaptive in his environment.""The pigeon pecks the key because he isreinforced for doing so." It is importantto emphasize this distinction between rein-forcement and principles of reinforcement,and the analogous distinction between adap-tation to a niche and the process of selectionby which adaptation comes about. In thecase of adaptation, the process of selectioninvolves differential reproduction, eitherthrough absence of a mate, infertility, ordeath before reproductive maturity. In thecase of reinforcement, the distinction is lessobvious, since the principles of reinforce-ment refer to the laws by means of whichbehaviors that fail to yield reinforcement areeliminated, rather than the simple fact thatreinforced behaviors generally predominateat the expense of unreinforced behaviors.

Thus, both evolution and learning can beregarded as the outcome of two independentprocesses: a process of variation that gener-ates either phenotypes, in the case of evolu-tion, or behavior, in the case of learning;and a process of selection that acts withinthe limits set by the first process. In bothcases, the actual outcome of the total proc-ess is related to, but not identical with, thematerial acted upon: phenotypes reproducemore or less successfully, but a gene pool isthe outcome of selection; similarly, behaviors

are more or less highly correlated with rein-forcement, but learning (i.e., an alteration inmemory) results.

The three aspects of this process—varia-tion, selection, and adaptation—have re-ceived differing emphases at different times,depending on the prevailing state of knowl-edge. Before Darwin, adaptation, in theform of the Law of Conditions of Existence,was emphasized, since the only explanationfor it—the design of the Creator—was notscientifically fruitful. Following Darwin,selection, both natural and artificial, re-ceived increased attention, in the absence offirm knowledge of the mechanism of varia-tion (i.e., inheritance). With the adventof Mendelian genetics, variation has beenmost intensively studied (this is one aspectof the "molecular vs. organismic" debate re-ferred to by Pfaffman).

In terms of this development, the studyof learning is at a relatively primitive level,since the Law of Effect, although a great ad-vance over the level of understanding whichpreceded it, simply represents the identifica-tion of environmental events—reinforcers—with respect to which behavior is adaptive.Lacking is a clear understanding of both se-lection (the principles of reinforcement)and variation (the principles of variation).

Space precludes exhaustive elaborationof all the implications of the classificatoryscheme we are suggesting. However, inorder to provide some context for ouraccount of the facts already discussed, itseems essential to briefly summarize somepossible candidates for principles of varia-tion and reinforcement. It should be ob-vious that current knowledge does not per-mit the categories used to be either exhaus-tive or clear-cut.

Principles of behavioral variation. 1.Transfer processes: One of the main sourcesof behavior in a new situation is obviouslypast experience in similar situations. Trans-fer has been most exhaustively studiedunder the restricted conditions of verballearning (e.g., Tulving & Madigan, 1970),but the principles of memory thus de-rived—proactive and retroactive interfer-ence, primacy and recency, retrieval factors,etc.—are presumably of general applicability.


With few exceptions (e.g., Gonzales,Behrend, & Bitterman, 1967), these prin-ciples have been little used to interpretanimal learning experiments. Other prin-ciples of transfer are stimulus and responsegeneralization (induction) and what mightbe called "compositional transfer," in whichseveral past experiences are combined togenerate a novel behavior, as in insight learn-ing and other forms of subjective organiza-tion of past input.

2. Stimulus substitution: This principle,which is usually identified with Pavlovianconditioning, has already been discussed asa description of the origin of the terminalresponse of Pecking. It may also describethe origin of the metastable terminal re-sponse Head in magazine, since this responseis also elicited by food under these condi-tions, and the animal which showed this re-sponse most persistently (Bird 29) had hadconsiderable experimental experience. Thedifference in the persistence of Head in mag-azine in the case of this bird, as comparedwith others, cannot be explained in this wayand might reflect a difference in other trans-fer processes. The final dominance of Peck-ing, in every case, may reflect a specialsusceptibility of consummatory responses tothe stimulus substitution principle, or the ac-tion of other transfer principles in this par-ticular situation in ways that are presentlyunclear. The indefinite persistence of keypecking following only three peck-contingentreinforcements, found by Neuringer(1970b), tends to support the simpler con-clusion, as does the data of Wolin (1968),who reports a similarity between the to-pography of operant pecking for food orwater reinforcers, and the appropriate un-conditioned response. We return later to ageneral discussion of this principle in rela-tion to these data (pp. 33-34).

3. Preparatory responses: This principleis also frequently associated with classicalconditioning, and in that sense is relatedto, and to some extent overlaps with, thestimulus substitution principle. Thus, someconditioned responses, such as salivation, canbe equally well described by either principle.Others, also respondents (such as heart rate,which increases following electric shock, but

usually decreases in anticipation of it [Zea-man & Smith, 1965]) may be classified aspreparatory responses. Skeletal responsesobserved in classical conditioning situationsare often preparatory in nature (see discus-sion of classical conditioning, below).

4. Syntactic constraints: There are oftensequential constraints among behaviors, sothat a given behavior is determined by someproperty of the sequence of preceding be-haviors. Examples are spontaneous positionalternation observed in rats and other ro-dents, stimulus alternation observed in mon-keys on learning set problems (e.g., Levine,1965), and sequential dependencies observedin most species which cause responses tooccur in runs rather than alternating ran-domly (e.g., position habits and other per-severative errors). Human language pro-vides the most developed example of syntac-tic constraints.

5. Orienting responses: This category in-cludes all those transient behaviors, such asexploration, play, curiosity, etc., that exposethe organism to new stimuli and providethe possibility of transfer to future situations.

6. Situation-specific and species-typicalresponses: Certain situations seem to callforth specific responses, which are oftentypical of the species rather than the indi-vidual and do not seem to depend in anyobvious way on any of the other principlesof variation such as transfer, etc. Examplesare the species-specific defense reactions dis-cussed by Bolles (1970), which occur infear-producing situations, the tendency topeck bright objects shown by many birds(Breland & Breland, 1966), and the diggingshown by small rodents (Fantino & Cole,1968). Other examples are given by Click-man and Sroges (1966).

Principles of reinforcement. Before thediscovery of the mechanism of inheritance,evolution could be explained only in termsof a goal—adaptation—and a means suf-ficient to reach that goal—variation andselection; the generation-by-generation de-tails of the process were obscure: "Ourignorance of the laws of variation is pro-found [Darwin, 1951, p. 170]." We areat present equally ignorant of the mecha-nisms of behavioral variation. Since learn-


ing must usually involve constant interplaybetween variation and reinforcement, we arenot yet in a position to suggest anything spe-cific about the moment-by-moment details ofthe process, either in its variational or se-lective aspects. However, just as Darwinwas able to say something about selectionby pointing out the adaptive role of variousstructures, so it is possible to learn some-thing about the selective role of reinforce-ment by looking at steady-state adaptationsto fixed conditions of reinforcement, that is,reinforcement schedules. On this basis, wesuggest the following tentative generaliza-tions about the effects of reinforcement:

1. Reinforcement acts directly only on theterminal response; activities which occur atother times (interim activities, adjunctivebehavior, etc.) must be accounted for inother ways, to be discussed later. Thisassertion is perhaps closer to a definitionthan an empirical generalization, since it isequivalent to the assertion that the terminalresponse may, in general, be identified asthe activity occurring in closest proximity toreinforcement in the steady state. Identifi-cation is, of course, no problem in condition-ing situations that enforce a contingency be-tween some property of behavior and thedelivery of reinforcement. However, wewill show later that there is no empirical orlogical basis for separating situations thatdo impose a contingency between responseand reinforcement from those that do not(see Classical Conditioning, below).

2. Reinforcement acts only to eliminatebehaviors that are less directly correlatedwith reinforcement than others. This gener-alization, like the first, is also more like adefinition, since all that is observed (underconsistent conditions of reinforcement) isthe eventual predominance of one behaviorover others—which is consistent with eithera suppressive or a strengthening effect. AsSkinner (1966a) points out in a summaryof the Law of Effect:Thorndike was closer to the principle of naturalselection than the [usual] statement of his law.He did not need to say that a response which hadbeen followed by a certain kind of consequencewas more likely to occur again but simply thatit was not less likely. It eventually held the fieldbecause responses which failed to have such

effects tended, like less favored species, to dis-appear [p. 13].

Unfortunately, Skinner, and most other be-haviorists, elected to follow Thorndike inconsidering reinforcement to have a positive,strengthening, or "stamping-in" effect. Oneof the main purposes of the present paper isto suggest that this decision was a mistakeand has given rise to a number of problemsand controversies that can be avoided if theeffects of reinforcement are considered to bepurely selective or suppressive.

There are three kinds of argument which, support a purely selective role for reinforce-\ment. The first, and most important, is thatfthe overall conceptual scheme which results{is simpler and more easily related to bio-logical accounts of behavior than the alterna-tive.

The second point is that situations suchas extinction and shaping by successive ap-proximations that might seem to require anactive role for reinforcement can be inter-

7 preted in a way that does not require any-thing more than a selective effect. Thispoint is discussed later (see Extinction, be-low).

The third point is that the superstitionand related experiments suggest that the re-sponse contingency imposed by most rein-forcement schedules is not essential for theproduction of some terminal response, butonly for the selection of one response overothers, or for directing a response whichwould probably predominate in any case—as in key pecking by pigeons. Our failureto find a consistent difference in rate of ob-server-defined pecking between the response-dependent and response-independent condi-tions of the present superstition experimentsupports this view, as does a recent findingthat the contingency between electric shockand responding is not necessary to the gener-ation of behavior maintained by intermittentshock delivery to squirrel monkeys (Hutch -inson, 1970; Stretch, personal communica-tion, 1970).

The usual interpretation of the fact thatthere is a terminal response in the supersti-tion situation, despite the absence of re-sponse-contingency, is the notion of acci-dental strengthening of a response by con-

'SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR 21

tiguity with the delivery of reinforcement.Here we consider the general implications ofadventitious reinforcement as an explanation,and its incompatibility with the notion ofreinforcement as selection. Other problemsrelated to adventitious reinforcement are dis-cussed later (see Acquisition, Classical Con-ditioning).

First, adventitious reinforcement impliesj failure of constancy, in the sense that the1 animal is presumed to be unable, because1 of the stamping-in mechanism of reinforce-

ment, to distinguish between real and acci-dental correlations between his behavior and

I the occurrence of reinforcement. This is a; strong assumption, in view of the adaptiveutility of the constancy process and itsubiquity in perceptual and motor mecha-nisms. In perception, a similar failure todistinguish changes in sensory input that areproduced by our own behavior from changesthat are independent of behavior might causeus to perceive the world as rotating everytime we turn our head. It is of course true

• that on the basis of one or a few instancesi the animal may not be in a position to be, certain about the reality of a contingent

relationship between his behavior and rein-i forcement—and this kind of sampling limita-

tion might account for a transient supersti-1 tious effect. It is less convincing as an

account of a long-term effect." Second, if reinforcement is considered as

purely selective, it cannot be invoked as anexplanation of behavior when no imposedcontingency exists between reinforcementand behavior (i.e., in the absence of selec-tion). To do otherwise would be like takinga population of white mice, breeding themfor 20 generations without further selectionfor color, and then attributing the resultingwhite population to the results of "accidentalselection." In this case, as in the case ofresponse-independent reinforcement, the out-come reflects a characteristic of the initialpopulation (i.e., the mice gene pool, thenature of the organism), and not a non-

^xistent selection process.// / In short, the notion of adventitious rein-

'forcement is not a tenable one. The extentto which reinforcement can be invoked asan explanation for behavior is directly re-

lated both to the degree of imposed con-tingency between response and reinforce-ment and to the opportunities for that con-tingency to have some selective effect (i.e.,the number of contingent reinforcements).If there is no contingency, or if few con-tingent reinforcements have occurred, theresulting behavior must owe more to prin-ciples of variation than to the selective actionof reinforcement. On the other hand, ifmany contingent reinforcements have beendelivered during a protracted period of shap-ing, the final form of behavior is obviouslymuch less dependent on particular principlesof variation, and the role of reinforcement(selection) may properly be emphasized.

3. There is considerable evidence for thegenerality of a principle implicating rela-tive rate or proximity of reinforcement asthe fundamental independent variable deter-mining the spatial and temporal location ofresponding in steady-state conditioningsituations. Thus, Herrnstein (1970) hasrecently reviewed a number of operant con-ditioning experiments involving differentialreinforcement of simultaneous (concurrentschedules) and successive (multiple sched-ules) choices which support the idea of rela-tive reinforcement rate as the independentvariable most directly related to the rate ofkey pecking in pigeons. Shimp (1969) haspresented an analysis which is formally dif-ferent from Herrnstein's, but which alsoimplicates differences in reinforcement rateas the crucial variable. An extensive seriesof experiments by Catania and Reynolds(1968) suggests a similar (although lessexact) relationship between rate of peckingand relative temporal density of reinforce-ment on interval reinforcement schedules.Jenkins (1970) summarizes a series of stud-ies with a discrete-trials procedure thatled him to suggest relative proximity to re-inforcement as an important determiner ofthe tendency to respond in the presence ofa stimulus. Staddon (1970a) has suggesteda similar principle to account for positive"goal gradients" that underlie the effects ofreinforcement omission on a variety of inter-val schedules.

4. The concept of reinforcement implies acapacity to be reinforced. The fact that a


given stimulus may be reinforcing at onetime, but not at another, requires the ideaof a state corresponding to each class ofreinforcers. The independent variable ofwhich the strength of most of these states isa function is deprivation with respect to theappropriate class of reinforcers (food de-privation for the hunger state, water forthirst, etc.). This will not do for most nega-tive reinforcers (e.g., the removal of electricshock), however, since there is no obviouscounterpart to deprivation in this case. Itis also unlikely that deprivation is the onlyindependent variable sufficient to alter thestrength of states associated with positivereinforcement. For example, evidence isdiscussed later in favor of reciprocal inhibi-tory interaction between states as a possiblefactor in polydipsia and other adjunctive be-havior. Interactions of this sort may alsoalter the strength of states for which thereis no deprivation requirement, as in audioanaesthesia (Licklider, 1961).

Thus, one may hope for a set of principlesof reinforcement that will deal both withthe proper classification of states and withthe interactions among them.

The theoretical vocabulary of learning isfull of terms with an uneasy conceptual sta-tus somewhere between explanation, defini-tion, and category label. This terminology,which is not coherent or internally consist-ent, makes it difficult to approach particulartopics with an open mind. It is too easyto dismiss an experimental result as due toadventitious reinforcement or respondentconditioning without, in fact, having anyclear understanding of what has been said.Simply defining everything operationally isof little help in this situation, since a setof definitions is not a theory. And a theory,in the sense of a system of concepts that isinternally consistent and coherent, is whatis required if we are to be sure, in particularcases, whether we really understand a phe-nomenon—or are merely substituting onemystery for another, with the assistance ofan opaque vocabulary.

What we are proposing is too primitive tobe called a theory in this sense. However,it does offer a system of classification thatmakes it difficult to have the illusion of

understanding a phenomenon if compre-hension is really lacking. In the followingsection, some implications of this scheme areshown in three major areas: acquisition, ex-tinction, and classical conditioning. This isfollowed by a brief discussion of possibledifficulties of this approach. With the aidof this groundwork, it will then be easier toreturn to a general account of the supersti-tion and related experiments in the conclud-ing section.

AcquisitionThe number of trials necessary for learn-

ing is one of those perennial problems thatseems to defy resolution. Appeal to data isnot conclusive because learning curves aresometimes incremental and sometimes step-like. Even in particular cases, theory is notconclusive either, since with sufficient in-genuity, theoretical accounts of both kindsof curve may be constructed on the basisof either one (or a few) trial learningassumptions or incremental assumptions in-volving thresholds. We turn now to thepossibility that the issue is a consequenceof the stamping-in view of reinforcementand becomes less urgent once that view ischallenged.

A comment by Skinner (1953) on thenecessary and sufficient conditions for thedevelopment of superstition provides anillustration:In superstitious operant behavior . . . the processof conditioning has miscarried. Conditioningoffers tremendous advantages in equipping theorganism with behavior which is effective in anovel environment, but there appears to be noway of preventing the acquisition of non-advanta-geous behavior through accident. Curiously, thisdifficulty must have increased as the process ofconditioning was accelerated in the course ofevolution. If, for example, three reinforcementswere always required in order to change theprobability of a response, superstitious behaviorwould be unlikely. It is only because organismshave reached the point at which a single con-tingency makes a substantial change that theyare vulnerable to coincidences [pp. 86-87].

Even within the framework of the stamp-ing-in view, it is clear that the truth of thisstatement depends on a tacit assumptionthat responses will not generally occur morethan once unless followed by reinforcement.


If a given response can be relied on to occurat least 20 times in succession, even withoutreinforcement, then 3-trial, or even 10-triallearning might well be sufficient to insure itsacquisition under the conditions of the super-stition experiment. The assumption thatresponses will occur only once in the absenceof reinforcement is a strong assumptionabout syntactic constraints, in our terminol-ogy. Moreover, it is contradicted by the re-sults of the Williams and Williams study(1969), which show indefinite persistenceof pecking in the absence of any contiguousrelationship between pecks and reinforce-ment. There is no reason to suppose that asimilar persistence is not characteristic ofother behaviors (e.g., position habits), al-though pecking may be more persistent thanmost. Thus, the finding of superstitiousterminal responses, or of indefinite peckingfollowing just three response-contingent re-inforcements (Neuringer, 1970b), needimply nothing about the number of trialsnecessary for learning.

These considerations suggest, as a mini-mum, the need to take variation into accountin discussions of the "speed of condition-ing," since rapid acquisition may either re-flect an unpersistent response that is reallylearned rapidly, or a very persistent onethat may be learned quite slowly. No in-ferences about "speed of conditioning" canbe drawn solely on the basis of speed ofacquisition without information about thefrequency and pattern of a given behaviorto be expected in a given situation (whichmay include predictable delivery of rein-forcement) in the absence of contiguity be-tween that behavior and reinforcement. Inpractice, since information of the requiredsort is rarely, if ever, available, it seemswise to defer the issue of speed of learninguntil behavioral variation has been muchmore thoroughly studied.4

Thus, the moment-by-moment details con-cerning the effect of reinforcement remain

4 Problems of this sort are not solved by re-ferring to a hypothetical "operant level" because(a) this level is often zero in the absence of ahistory of reinforcement in the situation; (6) it israrely constant, as the term level implies; and(c) the problem of the origin of this level isthereby simply evaded.

uncertain until much more is known aboutvariation. In the meantime it seems moreparsimonious and less likely to lead to fruit-less controversies about "speed of condition-ing," continuity versus noncontinuity, etc.to assume that the appearance of one be-havior, rather than another, at a certaintime or place, rather than some other timeor place, always requires explanation interms of principles of variation, with onlythe disappearance of behaviors being attrib-utable to the effects of reinforcement.

This general approach is not novel. Itresembles both Harlow's (1959) account oflearning-set acquisition in terms of the pro-gressive elimination of error factors, andcertain versions of stimulus-sampling theory(Neimark & Estes, 1967). In Harlow'sterms, as in ours, one-trial acquisition is aphenomenon that depends on the existenceof factors that make the correct behaviormuch more probable (and persistent) thanothers (i.e., upon principles of variation).In the learning-set case, these factors areembodied in the prior training procedure,which progressively selects for an initiallyweak behavior (the "win stay, lose shift"strategy) at the expense of the initiallymuch stronger tendencies to approach par-ticular stimuli. A lengthy process may notbe essential, however, for principles of varia-tion involving insight ("compositional trans-fer," see above) may serve the same func-tion, if they are available to the animal.The important point is the shift of emphasisaway from the supposed efficacy of somestamping-in mechanism, the action of whichmust remain obscure in the absence ofknowledge about variation, to the principlesof variation that determine the strength ofbehaviors in advance of contiguity with re-inforcement.5

ExtinctionExtinction is often used as a test for

"what is learned" during a training pro-5 Memory has not been separately discussed in

this account of acquisition because it is embodiedin most of the variational and selective processeswe have described. The argument of the presentsection suggests that a separate account of mem-ory may have to await advances in our knowledgeof these processes.


cedure, as in generalization testing (Gutt-man & Kalish, 1956), and testing for con-trol by temporal factors (Ferster & Skin-ner, 1957; Staddon, 1970b). Under theseconditions, it is assumed that behavior isdetermined almost entirely by transfer fromthe base-line condition. Providing the dif-ference between the extinction and trainingconditions is not too great, either in termsof environmental factors (the stimulus situa-tion is not too different) or temporal factors(the extinction is not prolonged), thisassumption can be justified by the reliabilityand predictability of the behavior usuallyobserved.

When these conditions are not satisfiedor when the training preceding extinctionhas not been protracted, this reliability is notusually found. On the contrary, extinctionunder these conditions is usually associatedwith an increase in the variability of be-havior (Antonitis, 1951; Millenson & Hur-witz, 1961). This increase in variability isexactly what would be expected if, as wehave suggested, reinforcement has a purelyselective effect: in these terms, traininginvolves a progressive reduction in vari-ability under the selective action of rein-forcement (centripetal selection, see be-low), so that absence of reinforcement(extinction) represents a relaxation of se-lection—with an attendant rise in vari-ability. We turn now to a brief accountof the effects of changes in the amount anddirection of selection in evolution, which mayshed some further light on the properties ofbehavioral extinction.

Darwin (1896) comments on the effectof domestication as follows:

From a remote period to the present day, underclimates and circumstances as different as it ispossible to conceive, organic beings of all kinds,when domesticated or cultivated, have varied. . . .These facts, and innumerable others which couldbe added, indicate that a change of almost anykind in the conditions of life suffices to causevariability . . . [Vol. 2, p. 243].

Although Darwin sometimes (erroneously)interpreted this observation as reflecting adirect effect of changed conditions on thereproductive system, it can be interpretedin modern terms as due to a relaxation of

selection. This is clear from the concept ofcentripetal selection (Haldane, 1959; Mayr,1963; Simpson, 1953), which refers to thefact that selection under wwchanging condi-tions, if long continued, acts to weed out ex-tremes, rather than systematically to shiftpopulation characteristics in any particulardirection:

When adaptation is keeping up, selection at anyone time will be mainly in favor of the existingtype. . . . In such cases, the intensity of selec-tion tends to affect not the rate of change but theamount of variation [Simpson, 19S3, p. 147].

Thus, a change in conditions will generallyinvolve a shift away from centripetal selec-tion, with its tendency to reduce variability,and will often lead, therefore, to increasedvariability. The most obvious example ofthe effects of relaxation of selection in evolu-tion is degenerating or vestigial structures,that are no longer being selected for:

It is so commonly true that degenerating struc-tures are highly variable that this may be advancedas an empirical evolutionary generalization [Simp-son, 1953, p. 75].

We have already noted that the onset ofvariability in extinction is often delayed. Asimilar delay in the effect of changed condi-tions is often apparent in evolution, Darwin(1896) notes:

We have good grounds for believing that the in-fluence of changed conditions accumulates, so thatno effect is produced on a species until it has beenexposed during several generations to continuedcultivation or domestication. Universal experi-ence shows us that when new flowers are firstintroduced into our gardens they do not vary;but ultimately all, with the rarest exceptions, varyto a greater or less extent [Vol. 2, p. 249].

Similar delays have also been reported inexperiments on artificial selection (Mayr,1963). These delays seem to reflect whathas been termed "genetic inertia" or "gene-tic homeostasis" (Mayr, 1963), that is, thetendency for a gene pool which is the resultof a long period of consistent selection toresist changes in the direction of selection.A similar mechanism in behavior mightaccount for the dependence of variability inextinction on the duration of the precedingtraining period, which was referred to ear-lier : The amount of variability might be ex-


pected to be greater and its onset soonerfollowing a brief training period than afterone of longer duration. Genetic homeostasisalso seems to be involved in the phenomenonof reversion, to be discussed next.

Not all the variation which occurs eitherin behavioral extinction, or following achange in the conditions of life in evolution,is wholly novel. A relatively common effect,for example, is the reappearance of whatDarwin terms "ancestral types," that is,phenotypes which predominated earlier inphylogeny but which have been selectedagainst more recently. This is the phenome-non of reversion which, because of hisignorance concerning heredity, Darwin(1896) found among the most mysteriousof evolutionary processes:But on the doctrine of reversion . . . the germ[germ plasm] becomes a far more marvellous ob-ject, for, besides the visible changes which it under-goes [i.e., phenotypic expressions], we must believethat it is crowded with invisible characters, properto ... ancestors separated by hundreds or eventhousands of generations from the present time:and these characters, like those written on paperwith invisible ink, lie ready to be evolved wheneverthe organisation is disturbed by certain knownor unknown conditions [Vol. 2, pp. 35-36].

Thus, one effect of a relaxation of selectionis a more or less transient increase in therelative influence of the distant past at theexpense of the immediate past. In be-havioral extinction, this should involve thereappearance of old (in the sense of pre-viously extinguished) behavior patterns;that is, transfer from conditions precedingthe training condition at the expense oftransfer from the training condition.* Inboth cases, evolution and behavior, the effectof the change in conditions may be expectedto depend on variables such as the magni-tude of the change and the time since thepreceding change.

8 Other than clinical accounts of regression, wehave been able to find only one published report ofthis effect—in an account describing shaping por-poises to show novel behaviors (Pryor, Haag, &O'Reilly, 1969). However, we have frequentlyobserved it while shaping pigeons: if a pigeon hasbeen trained in the past to perform a variety ofresponses, the increase in variability during ex-tinction of the most recently reinforced responsegenerally includes the reappearance of earlier re-sponses.

The analogy from Darwin suggests thatany considerable change in conditions shouldincrease variability, yet a change in rein-forcement schedules that includes an in-crease in reinforcement rate is not usuallythought of as producing an increase in vari-ability. This apparent contradiction is re-solved by noting that an increase in rateof reinforcement, in addition to changingconditions, also increases the rate of selec-tion (since the analogy assumes reinforce-ment to have a purely selective effect).Thus, variability may be briefly increased,but since the rapidity of selection is alsoincreased, the net effect may be small. Ananalogous (but impossible) phenomenon inevolution would be to decrease the time be-tween generations at the same time that con-ditions are changed. This would speed upthe attainment of a new equilibrium andminimize the increase in variability generallyassociated with changed environment.

The increase in variability due to extinc-tion is most directly put to use in the processof shaping by successive approximations.Frequently, following the first few reinforce-ments delivered during a "shaping" session,the effect is simply an increase in the rangeand vigor of behavior. This change can beviewed as being due to the interruption ofeating (cf. Handler, 1964), however, ratherthan any direct strengthening effect of rein-forcement (which we are questioning in anycase). In terms of the foregoing analysis,the conditions following the first reinforce-ment should be optimal for an increase invariability: the change is large (from con-tinuous eating to absence of food) and thetraining procedure is of short duration (the3-4-second eating bout), so that time sincethe preceding change is also short. As foodcontinues to be delivered intermittently, se-lection occurs and variability decreases.

We have been suggesting a purely selec-tive (rather than strengthening, stamping-in, or energizing) role for reinforcement.The present discussion suggests that such anessentially passive role is compatible with anumber of phenomena—extinction, the acti-vating effects of isolated reinforcements—that may appear to demand a more activerole for reinforcement. This compatibility


was established by drawing attention tosimilar phenomena in evolution, where thepurely selective effect of the conditions oflife (analogous to the schedule of reinforce-ment) is unquestioned. However, no nec-essary identity between the genetic mecha-nism, which is responsible for the effect ofchanged conditions on variability in struc-ture, and whatever process is responsiblefor analogous effects in behavior, is intended.Any process for the production of variationthat incorporates some latent memory ofpast adaptations is likely to show similareffects.

Classical Conditioning

Our scheme has strong implications forthe distinction between classical (Pavlovian,respondent) and instrumental (operant)conditioning, to the extent that the distinc-tion goes beyond procedural differences.Classical conditioning is often thought of asa paradigmatic instance of the primary proc-ess of learning: "The [learning] process ap-pears to be based entirely on temporal con-tiguity and to have classical conditioningas its behavioral prototype [Sheffield, 1965,p. 321]." The salivation "reference experi-ment" can be interpreted as prototypical inat least two ways that are not always keptseparate. The first (which has some simi-larities to our position) is referred to bySheffield—the notion that learning dependssolely on temporal relationships. Guthrie'saphorism that the animal "learns what hedoes" is a related idea. It is not easy tofind a definitive account of this position,but it may perhaps be summarized by sayingthat reinforcement or reward is simply nec-essary to ensure that some behavior occursin a conditioning situation. Principles in-volving temporal relationships (contiguity)then ensure that whatever occurs will trans-fer from one occasion to the next.

The second way in which classical con-ditioning is discussed as prototypical is interms of the rule that relates the conditionedand unconditioned responses. Pavlov(1927) emphasized stimulus substitution asthe distinctive property of the situation: theresponse originally elicited only by the UCSis later made to the CS, Subsequently, two

kinds of departure from this rule have beenpointed out: (a) Even in the salivationexperiment, there are other readily identifi-able components of the conditioned responsethat do not fit the stimulus substitution rule.These preparatory responses (Zener, 1937)are largely, but not exclusively, skeletal(rather than autonomic). (b) Even in thecase of salivation and other autonomic re-sponses, the CR is rarely identical to theUCR (i.e., a redintegrative response), sothat components of the UCR may bemissing from the CR. More serious aredifferences in direction of change be-tween CR and UCR, which may not evenbe consistent across individuals, as in heartrate and respiratory conditioning (Martin& Levey, 1969; Upton, 1929; Zeaman &Smith, 1965).

Partly because of problems involving pre-paratory responses, classical conditioning hasincreasingly been restricted to autonomicallymediated responses. There were two basesfor this restriction: the apparent difficulty ofconditioning skeletal responses by the opera-tions of classical conditioning and accordingto the stimulus substitution principle (cf.Skinner, 1938, p. 115), and the supposedimpossibility of conditioning autonomic re-sponses via the Law of Effect. This is clearfrom Kimble's (1961) comment:

Obviously the common expression, "the conditionedresponse," is misleading, and probably in impor-tant ways. At the same time it should be recog-nized that the behavior described by Zener [pre-paratory responses] was almost certainly instru-mentally, rather than classically, conditioned [p.54].

The force of this argument is lost once thesusceptibility of autonomic responses suchas salivation to operant conditioning is dem-onstrated.

The foregoing facts are sufficient to showthe error of continuing to regard classicalconditioning as a unified process, muchless as an explanatory element in accountsof operant conditioning. If classical con-ditioning is a single process, then it must bedescribable by principles of operation thatapply to every instance. As we have seen,even stimulus substitution, the most generalsuch principle, fails to apply in every case.


Many, but not all, the anomalous cases areskeletal responses—which suggested thatperhaps the notion of a single process couldbe preserved by restricting the term to auto-nomic responses. The only independentbasis for this is to segregate skeletal andautonomic responses on the grounds thatoperant conditioning of autonomic responsesis impossible. Since this is now known tobe false (Miller, 1969), the only remainingbasis for excluding skeletal responses fromthe class of classically conditionable re-sponses is their failure to conform to theprinciple of stimulus substitution. But, inaddition to being tautologous (classical con-ditioning simply becomes equivalent to learn-ing via stimulus substitution), this fails be-cause many autonomic responses do not con-form to this principle and, based on thework of Brown and Jenkins and of Williamsand Williams, at least one skeletal response—pecking in pigeons—does obey it (otherpossibilities are leg flexion and eyeblink).Thus, the class of classically conditionableresponses can be defined neither in terms ofthe neural mediating system (autonomic vs.skeletal) nor in terms of adherence to a par-ticular principle of learning.

The only remaining feature common toall the situations labeled as classical condi-tioning is the procedure itself. Research inthis area has tended to focus on the proper-ties of the temporal relationship between CSand UCS that are necessary and sufficientfor the CS to acquire the power to elicit theconditioned response, and a consensus ap-pears to be emerging that the crucial factoris the extent to which the CS is a predictorof the UCS (Rescorla, 1967). However,the notion of predictiveness does not appearto differ from relative proximity (of the CSto the UCS, or of a stimulus to reinforce-ment) which, as we have seen (Principleof Reinforcement 3, above), is a factor ofwide applicability in operant conditioning.Thus, for all practical purposes, classicalconditioning may be defined operationallyas a class of reinforcement schedules thatinvolve presentation of reinforcement inde-pendently of the subject's behavior.

We conclude, therefore, that the divisionof the field of learning into two classes—

classical and instrumental conditioning—each governed by separate sets of principles,has no basis in fact. As an alternative, wesuggest an analysis based on the principlesof behavioral variation and reinforcementwe have already discussed. In terms of thisanalysis, all adaptive behavior is subsumedunder an expanded version of the Law ofEffect, and a given situation is to be under-stood in terms of two factors: (a) the rein-forcement schedule, that is, the rule prescrib-ing the delivery of reinforcement, or, moregenerally, stimuli, in relation to the behaviorof the organism, and (b) the nature of theresponse under consideration. In terms of {such an analysis, the properties normally \considered as distinctive of classical con-ditioning may, once attention is directed to ,the question, be seen as due in part to a \reinforcement schedule that happens to pre- jscribe no correlation between the delivery ;of reinforcement and the subject's behavior, \and in part to the special properties of re- ;sponses such as salivation (see Implications /2 and 3, below). 1

Implications. Several puzzles become ''.clearer once classical and instrumental con-ditioning are no longer regarded as separateprocesses:

1. In the earlier discussion of auto-shapedpecking, Williams and Williams noted that"the directed quality of the induced peckingdoes not follow naturally from respondentprinciples." Since we question this frame-work as a general description of anythingoutside the salivation experiment, its inabil-ity to deal with this particular situationposes no problem. The appearance of keypecking in the auto-shaping and superstitionsituations is, of course, not fully understood.It may reflect a special susceptibility of con-summatory responses to the principle ofstimulus substitution, as suggested earlier,or the action of transfer principles in waysthat reflect something in common amongthe past histories of most pigeons. Simi-larly, little can be said about the directednessof pecking until the conditions under whichthis response is, and is not, directed havebeen more fully explored; although the re-sults so far (with pigeons and monkeys)suggest that skeletal responses may always


be directed. Rather than attempt to explain(or explain away) these characteristics, thebest course seems to be simply to observeand classify these and other behaviors undera variety of response-independent and re-sponse-dependent schedules, in the hope thathypotheses can be devised that have morehope of generality than anything that can beinferred from the scanty data presentlyavailable. We return to these issues in the

, concluding section.2. There is little doubt that most auto-

nomic responses are more easily conditionedby response-independent schedules than byresponse-dependent ones. However, thisneed not imply the existence of two differentkinds of conditioning. It can as well beinterpreted as reflecting the existence ofinternal controlling factors that are notamenable to the principles of reinforcement.Because of these fixed factors, these re-sponses are not as free to come under thecontrol of external stimulus factors as areskeletal responses that perform no functionin the internal economy of the organism.Moreover, this principle can be extendedbeyond autonomic responses to deal withany response that is strongly connected toany stimulus, internal or external. Thus,in a recent discussion of operant conditioningof drinking, Black (1970) writes:

This discussion suggests that one dimension alongwhich responses might be classified with respectto operant conditioning is the degree to whichthey are constrained from being changed by operantreinforcement by the properties of the neural sub-systems of which they are a part. The regulatorysystems can vary from very simple reflexes, suchas the knee-jerk, to complex instinctive ones,such as those involved in courtship. The mainpoint is not the complexity of the subsystems butrather the extent to which they limit the condi-tions under which operant reinforcement willwork [p. 267].

Segal (1970) has made a similar suggestionin a thoughtful discussion dealing with anumber of the points raised in the presentpaper.

As an example, the difficulties associatedwith demonstrating operant conditioning ofheart rate can be viewed as analogous tothe problem of acquiring control over anoperant response that is already under the

control of other variables, that is, such sched-ules are really concurrent rather than simpleschedules. The heart rate problem is per-haps analogous to training an animal toalter his rate of bar pressing to receive food,while pressing the same bar is also necessaryto obtain another reinforcer (such as oxygenor heat). Even without considering theproblem of interactions among drives, onewould not be surprised to find rather weakcontrol by the food reinforcer.

3. Williams (1965) recorded salivationin dogs while they were bar pressing forfood reinforcement on both fixed-intervaland fixed-ratio reinforcement schedules. Hefound that the onset of salivation within eachinterreinforcement interval approximatelycoincided with the onset of bar pressing inthe fixed-interval case, but began later thanbar pressing on fixed-ratio. This interestingresult is incompatible with an explanationof the operant response in terms of an under-lying classical conditioning process. How-ever, it may be understood in terms of theview we have been proposing by assumingthat (a) the occurrence of each response(bar pressing or salivation) is separatelyand independently determined by the con-ditions of reinforcement peculiar to it, and(b) both responses tend to occur at timesof greatest relative proximity to reinforce-ment (Principle of Reinforcement 3). Inthe fixed-interval case, these two assump-tions predict a similar time of occurrencefor both behaviors, because time since thebeginning of a trial is, for both responses,the best predictor of reinforcement. How-ever, in the fixed-ratio case, it is apparentthat no matter what determines the timeof onset of bar pressing, once it has stabilizedits onset provides a better predictor of rein-forcement than does trial time (since thefixed number of responses making up thefixed ratio take an approximately fixedtime). Thus, at asymptote our two assump-tions imply that salivation should be reliablydelayed with respect to bar pressing on fixed-ratio, but not on fixed-interval, as Williamsreports.

However, there is no reason to expect thisdelay early in training, since the animal isnot in a position to learn the cue significance


of bar pressing until it has more or lessstabilized. This expectation is also con-firmed by Williams (1965), who notes thatthe delayed onset of salivation in the fixed-ratio case "emerged only after repeated ex-posure to the schedules [pp. 344-345]."

On the basis of a failure to find salivationpreceding bar pressing on a spaced-respond-ing ("controlled latency") procedure, Wil-liams (1965) concludes that "the hypothesisthat the two measures are independent maybe rejected [p. 347]," which contradictsAssumption a, above. However, since bothresponses are assumed (Assumption b) tooccur at times of greatest relative proximityto reinforcement, and reinforcement cannotoccur before a bar press if it is always con-tingent upon bar pressing, there is no reasonto expect salivation reliably to precede barpressing under conditions where the com-mon reinforcement for both responses de-pends on bar pressing alone. Thus, the ap-parent asymmetry between salivation andbar pressing observed by Williams maysimply reflect an asymmetry between theconditions of reinforcement for each re-sponse, and need not imply any fixed internallinkage between them.

4. Recently, considerable attention hasbeen devoted to avoidance of particular foodsconditioned by a nauseous experience (in-duced by insulin or X rays) taking placeseveral hours after ingestion (Garcia, Er-vin, & Koelling, 1966; Kalat & Rozin,1970; Revusky & Bedarf, 1967; Rozin,1969). Since the CS-UCS interval in theseexperiments is considerably longer than iscustomary in classical conditioning experi-ments, these data are even less congenial toa Pavlovian analysis than the auto-shapingresults. They hint at the existence of anumber of unsuspected built-in linkages be-tween response systems and various salientstimuli. Such linkages are not unexpectedfrom a broad evolutionary point of view thatsees principles of variation and reinforce-ment as behavioral characteristics that are

I separately selected for, and bear as much(or as little) relationship to one another asdo morphological characteristics.

Difficulties of the Proposed Classification

Science is conservative and, quite cor-rectly, resists most attempts to alter anestablished theoretical framework. We havealready tried to show that the number ofanomalies facing current learning theoriesis sufficient to justify a search for alterna-tives. Nevertheless, the radical appearanceof the scheme we suggest is a substantialobstacle to its consideration. It is im-portant, therefore, to point out that it islittle more than an extension and reorgan-ization of familiar concepts, that is, rein-forcement, S-R behavior units, learningprinciples such as transfer, and ethologicalobservations on species-related behaviors.The difference is therefore largely one ofemphasis and selection rather than the in-troduction of wholly novel ideas.

Any discussion of evolution and learningnaturally brings to mind the learning-instinctissue. There is no simple parallel betweenthis dichotomy and anything in the schemewe propose. The origin of every behavior issupposed traceable to principles of variation;if, for example, in a particular case a prin-ciple of transfer is involved, one might wantto say that the behavior is learned. However,the question must then simply be askedagain about the previous situation fromwhich transfer has supposedly occurred. Inthis way, almost any question about therelative roles of heredity and environmentwill involve unraveling the whole of ontog-eny. This conclusion will not be unfamiliarto ethologists (cf. Beach, 1955).

It is also important to emphasize that wehave not been directly concerned with theevolution of the capacity to learn; althoughit may be that increasing knowledge of varia-tion will shed light on this issue.

One objection that may be raised tothe proposed scheme is that it is derivedfrom and deals explicitly only with positivereinforcement. However, a recent accountof behavior sustained by negative reinforce-ment (Bolles, 1970) is in perfect agreementwith our position. Bolles points out thatsome activities are much more easily con-ditioned than others in avoidance situations,and these are the unconditioned activities


that normally occur in a variety of poten-tially dangerous circumstances. These spe-cies-specific defense reactions, in Bolles'terminology, occur in advance of reinforce-ment (i.e., the avoidance of electric shock)—in our terms, they are determined initiallyby one of the principles of behavioral varia-tion. The lack of arbitrariness of the re-sponse is perhaps more obvious in avoidancethan in any other situation because of thecomplexity of the schedules involved: theanimal must usually learn something aboutthe pattern of occurrence of an intermittentaversive event, in the absence of responding,before he is in a position to detect altera-tions in that pattern correlated with hisown behavior. Although a similar situationprevails in all reinforcement schedules, thechange to be discriminated seems consider-ably easier both in appetitive conditioning,where the shift is from zero reinforcementin the absence of responding to reinforce-ment following every response, and in es-cape, where it is from continuous presenceof the aversive stimulus in the absence ofresponding to complete absence followingeach response. Bolles suggests other rea-sons, related to the limited opportunities foravoidance (in the schedule sense) in thewild life of small mammals, and thus thelimited opportunities for the capacity toavoid to be selected in phylogeny.

The strongest point in favor of our pro-posal is its promise of parsimony. Conse-quently, the most damaging criticism thatcan be directed against it is the absence offirm specification of the principles of rein-forcement and variation. This appears toallow the creation of such principles at will,enabling us to explain everything—andnothing. There are two defenses againstthis criticism. First, we again emphasizethe tentative nature of the principles wehave suggested. The overlap among theprinciples of variation, particularly, sug-gests that our list is provisional. Second,there is the strong possibility that clearrecognition of the distinction between varia-tion and reinforcement may be essential tofurther advance. In defense of this proposi-tion, we first briefly discuss some examplesfrom synoptic accounts of current learning

theory, which show it to incorporate fewsafeguards against multiple explanations forphenomena. Since our scheme of classifica-tion is at least internally consistent andforces one to relate each new principle ofvariation to others that already exist, it hassome advantages in this respect. Second,we discuss the controversy between cogni-tive and behavioristic theorists regardingthe role of structure in behavior, in relationto a similar controversy in the history ofevolutionary thought. The persistence ofthis controversy and its amenability to anal-ysis in terms of variation and reinforcementsuggest that our classification may be ofsome value despite its incompleteness.

1. Hilgard and Marquis (1940) list asprinciples of reinforcement: stimulus sub-stitution, expectancy, and the principle(law) of effect. It should be apparent fromthe earlier arguments that the Law of Effectis the result of the combined effect of bothvariation and reinforcement, stimulus sub-stitution is a principle of variation, and ex-pectancy refers to a general characteristicwhich can be imputed to most learning.Consequently this set of terms allows forconsiderable uncertainty in application toparticular situations. For example, ouranalysis of the Williams and Williams ex-periment (see p. 33) makes use of bothstimulus substitution and a principle of rein-forcement analogous to what Hilgard andMarquis mean by the Law of Effect. Yetthe same situation could also be analyzedin terms of expectancy; and it appears to beincompatible with the Law of Effect as^traditionally understood. Progress since1940 has not been dramatic, as illustratedby a list of "elementary conditioning proc-esses" inventoried by Jenkins (1970) inconnection with his work on cyclic reinforce-ment schedules: generalization, delay of re-inforcement, conditioned reinforcement, un-conditioned effects of eating, frustrationeffects, effects related to "behavioral con-trast." Despite the number of these proc-esses and the lack of any obvious relation-ship among them, Jenkins finds that theyare unable to account for some rather sim-ple features of his data, which require adescription in terms of the relative proximity


of a stimulus or a response to reinforcementas a major determiner (see Principles ofReinforcement, above).

2. There is a history of fruitless contro-versy between many behaviorists, who placelittle emphasis on the structural propertiesof behavior, and students of cognitive proc-esses, who see structure as the most interest-ing and important behavioral attribute (cf.Staddon, 1967, 1969, for a discussion in re-lation to operant conditioning). The dis-tinction between variation and reinforcementcan shed some light on this issue, which canbe illustrated by briefly considering thecontrasting views of Skinner and Chomsky(Chomsky, 1959; MacCorquodale, 1969)on the causation of learned behavior.

Chomsky's major concern is with prin-ciples of variation, in our terms, as is clearfrom his emphasis on the rule-governednature of language (see Principle of Be-havioral Variation 4, above).

Skinner's position is less obvious, butbecomes clear from his account of the shap-ing of behavior (Skinner, 1953) ; he writes:

Operant conditioning shapes behavior as a sculp-tor shapes a lump of clay. Although at some pointthe sculptor seems to have produced an entirelynovel object, we can always follow the processback to the original undifferentiated lump, andwe can make the successive stages by which wereturn to this condition as small as we wish. Atno point does anything emerge which is very dif-ferent from what preceded it. The final productseems to have a special unity or integrity of de-sign, but we cannot find a point at which thissuddenly appears. In the same sense, an operantis not something which appears full grown in thebehavior of the organism. It is the result of acontinuous shaping process [p. 91].

For Skinner, apparently, moment-to-momentvariation in behavior is small in magnitude,and essentially random (in the sense thatit is unrelated to the final goal) in direc-tion. Behavior is the result of the "accumula-tion . . . of indefinite variations which haveproved serviceable" in Darwin's phrase.The similarity to natural selection is fur-ther emphasized by Darwin's (1951)account of the evolution of complex struc-tures :If it could be demonstrated that any complexorgan existed, which could not possibly have beenformed by numerous, successive, slight modifica-

tions, my theory would absolutely break down[p. 191J.

In the history of evolution after Darwin,the rediscovery of Mendel's laws led to aretreat from gradualism in favor of a salta-tionism that traced evolutionary progress(especially evolutionary novelty) to largechanges (mutations) of a more purposivesort (cf. Mayr, 1960). This position iscloser to the view of Chomsky and othercognitive theorists, who tend to stress theimportance of insight and other rules ofcomposition that can produce sudden jumpsin behavior.

The history of evolution has not supportedthe saltationist view. Fisher (1930) showedthat large changes are much less likely to beadaptive than small ones, and Haldane andothers have shown by a variety of argumentsthat the time available for evolution by theselection of small variations is more thansufficient to account for the observed dif-ferences among taxa: "The saltationism ofthe early Mendelians has been refuted inall its aspects [Mayr, 1960, p. 350]."

At a superficial level, therefore, thesecomparisons might appear to favor Skinner'sgradualism and emphasis on reinforcement(selection), to the detriment of the cognitiveposition. This is probably unjustifiable fortwo main reasons. First, detailed analysisof complex problem solving clearly indicatesthe insufficiency of random variation as anaccount of the process (e.g., Neisser, 1967).The heuristics that are employed may, ofcourse, be attributed to past learning basedentirely on random variation. However,this suggestion meets with quantitative dif-ficulties when applied to the development oflanguage—the best-studied example of rule-governed behavior. Although calculationsin this area are of limited validity in theabsence of established principles of variation(analogous to Mendelian genetics), theattempts that have been made seem to indi-cate that the time available in ontogeny forthe development of language is incompatiblewith any kind of learning by random varia-tion (Chomsky, 1962; McNeill, 1968). Thisnegative result is the opposite of Haldane'saffirmative conclusion on the sufficiency ofsmall mutations as a basis for phylogenetic


changes. It suggests that neither the heuris-tics employed in complex problem solving,nor the rules of syntax, need to be built upentirely de novo during ontogeny.

Second, the relationship between evolu-tion and learning is such as to allow greaterflexibility to learning. This is becausenatural selection can only be a response tosmall differences in "fitness": the most fitgenotype will tend to prevail, although thespecies as a whole may thereby be led intoan evolutionary blind alley. In terms ofcontemporary accounts of goal-directedmechanisms, natural selection represents ahill-climbing process (Minsky, 1961) andhas no provision for prediction. This isclear in a familiar analogy due to SewallWright (1931) which shows the relation-ships among selection, structure, and varia-tion. He pictures the field of possible struc-tural variation as a landscape with hills andvalleys. The range of variation present ina population of organisms is represented bya closed area on this landscape. Selectionpressure is represented by the gradient (up-ward slope) of the landscape, so that eachpeak is an adaptive optimum for a givenconstellation of characters; valleys representunstable equilibria yielding so-called centrif-ugal selection. If the area representing agiven species includes a single adaptive peak,selection will be centripetal, so that thespecies will tend to cluster more and moreclosely around the peak. Thus, small muta-tions are more likely to lead to improvementsin fitness than large ones (with a limitingprobability of .5, as Fisher, 1930, hasshown), and the consequent predominantrole of such small differences in fitness inthe evolutionary process becomes obvious.

Learning is not so limited, however, be-cause the principles of variation can beweighted to take account of regularities inthe past history of the species (these areSkinner's, 1966b, "phylogenic contingen-cies") ; that is, behavior need not occur atrandom in advance of reinforcement, but canreflect a priori probabilities that have beenselected for during phylogeny. Other morecomplex strategies of this sort may also bebuilt up by natural selection, giving learn-ing a predictive capacity largely denied to

evolution itself. Thus, although learned be-havior reflects differences in reinforcementrate, just as evolution reflects selection onthe basis of relative fitness, it need not begenerally true either that small changes inbehavior are more likely to be adaptive thanlarge ones, or that the direction of changeis unrelated to the final goal—as Skinner'saccount implies, and as is usually (althoughnot invariably) the case in evolution. How-ever, since the more elaborate principles ofvariation must themselves be built up step-by-step by natural selection, it is to beexpected that the pattern and range of be-havioral variation must bear some relation-ship to the phylogenetic status of the orga-nism : "higher" organisms, such as man, arelikely to have developed more complex prin-ciples of variation than "lower" organisms,such as the pigeon.

Thus, the major focus of the argumentbetween Skinner and Chomsky is not onthe importance of reinforcement, but aboutthe complexity of the principles of varia-tion that determine the nature of behaviorin advance of reinforcement. Since Skin-ner derives his ideas from work on ratsand pigeons, and Chomsky from the studyof human language, there are considerablegrounds for a disagreement, especially ifits basis is not clearly perceived by eitherparty. Clear conceptual separation of varia-tion from reinforcement makes this kind ofconfusion much less likely.

CONCLUSION

The argument so far has served to drawattention to a number of generalizationsabout steady-state conditioning situations:

1. Most such situations involve sometimes and stimuli associated with relativelyhigh reinforcement probability (e.g., theperiod at the end of the interval on fixed-interval schedules), and others associatedwith relatively low reinforcement probabil-ity (e.g., the period at the beginning of theinterval).

2. The terminal response (a discriminatedoperant in Skinner's terminology) is re-stricted to periods of relatively high rein-forcement probability. This distribution ofthe terminal response with respect to time


and stimuli corresponds to a principle ofreinforcement that relates the strength of aresponse to the relative frequency, density,or proximity of reinforcement associatedwith that response (Catania & Reynolds,1968; Herrnstein, 1970; Jenkins, 1970).

3. The type, as opposed to the temporaland stimulus location, of the terminal re-sponse in situations involving both response-dependent and response-independent rein-forcement is determined by the interactionbetween principles of variation (e.g., trans-fer, stimulus substitution) that describe theoccurrence of the response, in advance ofreinforcement, and principles of reinforce-ment that determine whether it will persistor not (selective function of reinforcement).

4. Periods of low reinforcement prob-ability are generally associated with interimactivities, resembling appetitive behavior.If appropriate stimuli (goal objects) areprovided, stereotyped adjunctive behavior(e.g., polydipsia, pica; Falk, 1969) takesthe place of the more variable and relativelyundirected interim activities.

5. Both terminal response and interimactivities are more correctly labeled as pre-disposing conditions or states rather than be-haviors, since in the absence of response de-pendency, the type of activity falling intothese categories is not fixed. Thus, drink-ing, wheel running, fighting, pecking, and anumber of other activities may be eitherterminal or adjunctive behaviors, dependingon historical and stimulus factors (Segal,1969b; Skinner, 1959; Skinner & Morse,1958). Directing factors for adjunctivebehavior are the availability of appropriategoal objects (see above), and factors relatedto reinforcement that render some kinds ofactivity more probable than others: forexample, polydipsia appears to partially dis-place both adjunctive wheel-running (Segal1969a) and chewing-manipulatory behavior(Freed & Hymowitz, 1969) in rats (insituations with food as terminal reinforcer),even when both supporting stimuli are con-currently available.

The Terminal Response

We have already discussed the probablerole of principles of variation such as stim-

ulus substitution in describing the origin ofterminal responses such as Pecking andHead in magazine in the superstition situa-tion. Before turning to the more complexmatter of the interim activities, a wordshould be said about the paradoxical resultsof the Williams and Williams (1969) study,in which they found persistent pecking ata brief stimulus, ending in reinforcement,despite the fact that a key peck terminatedthe stimulus and thus prevented reinforce-ment. In terms of our analysis, this situa-tion pits reinforcement and variation againstone another; thus, (a) the predictable de-livery of food at the end of each presenta-tion of a brief key stimulus may be a suf-ficient condition for the occurrence of keypecking in that stimulus, by the principleof stimulus substitution. But, (b) becauseof the response-contingency, the occurrenceof a peck turns off the stimulus, omitting re-inforcement on that occasion, and thus re-ducing the overall reinforcement rate. Inturn, this reduction in reinforcement ratewill, via Principle of Reinforcement 3, tendto reduce the tendency to make any terminalresponse, including pecking, in that situa-tion. This process will continue until thetendency to peck has been sufficiently re-duced to allow the key stimulus to continueunpecked until the delivery of reinforcement,which will again provide the occasion forthe operation of the stimulus substitutionprinciple, making pecking likely once again.Thus, an equilibrium will be established ata rate of pecking higher than zero, but lessthan the rate which would obtain if peckinghad no effect on reinforcement rate. Ex-tinction of pecking takes place if key peckingprevents reinforcement, but does not turnoff the key stimulus, because the predict-ability of reinforcement, and thus the nec-essary condition for the operation of stim-ulus substitution, is thereby destroyed.Under these conditions, variation and rein-forcement combine to weaken the tendencyto peck, which therefore declines relativelyrapidly.

The data on "instinctive drift" reportedby Breland and Breland (1961, 1966) are


also compatible with this kind of analysis.7

However, their results strongly suggest thattoo much stress should not be laid on theapparent identity we (and others, e.g.,Brown & Jenkins, 1968; Williams & Wil-liams, 1969; Wolin, 1968) have found be-tween the topography of the terminal re-sponse and the (unconditioned) responsemade to the terminal reinforcer. As in thecase of adjunctive behavior (discussed be-low) and behavior elicited by central stimu-lation in reinforcing brain areas (e.g.,Glickman & Schiff, 1967; von Hoist & vonSaint Paul, 1963), the effect of the experi-mental procedures we have described ap-pears to be the induction of a state (in ourterminology) or "mood" (von Hoist &von Saint Paul) which makes some kindsof activity much more likely than others butpreserves some flexibility in the animal'smode of response to the stimulating environ-ment.

For example, Breland and Breland(1961) describe two situations, both show-ing "instinctive drift" in chickens, but onlyone of which conforms to the stimulus sub-stitution principle. In the first case, thechicken pecked a ball which he had learnedto project (via a remote firing mechanism)at a target—a hit being immediately followedby food. The history of contiguity betweenthe moving ball and reinforcement and thesimilarity of the responses to both ball andfood (i.e., pecking) fit easily into the stim-ulus substitution paradigm. In the secondcase, however, the chicken was reinforcedfor a chain of responses, the last of whichinvolved standing on a platform for 15 sec-onds. After training, the chicken showedvigorous ground scratching while standing

7 The delayed appearance of the food-related be-haviors in the Breland and Breland situations (ascompared to the relatively rapid emergence ofpecking in auto-shaping experiments) reflects thefact that food delivery could not become predict-able (the necessary condition for the operation ofthe stimulus substitution principle) until after theanimals had learned to produce it by making therequired "arbitrary" response. However, once thisinitial response was learned (via principles ofvariation other than stimulus substitution), thestage was set for the operation of stimulus sub-stitution, which could then override the originallearning.

on the platform. This response has no top-ographic resemblance to pecking food, butis, of course, a universal food-getting be-havior in chickens and typically occurs inthe vicinity of food.

These examples, and others discussed byBreland and Breland (e.g., raccoons "wash-ing" poker chips, porpoises swallowingmanipulanda, pigs "rooting" tokens, etc.),as well as exceptions to stimulus substitutionin the auto-shaping literature (e.g., Rachlin,1969; Sidman & Fletcher, 1968), are com-patible with a more general notion, to theeffect that the stimulus (temporal or extero-ceptive) most predictive of reinforcementcomes to control a state or mood (the ter-minal state) appropriate to that reinforcer.The particular activity which occurs duringthe terminal period will then depend onprinciples of variation, which take intoaccount both its motivational properties(e.g., food-related activities become morelikely if the terminal reinforcer is food), andthe nature of the stimulating environment;that is, the nature of the terminal state deter-mines what stimuli will be effective in elicit-ing what behavior. Thus, the Breland andBreland chickens pecked when the stimulusdefining the terminal state was appropriate(in some sense) for pecking, but scratchedwhen it was not; pigeons peck the key inauto-shaping experiments, but (based onour results) are slower to peck in the super-stition situation—presumably because anappropriate target is not provided. Simi-larly, Glickman and Schiff (1967) revieweda large number of studies of behavior in-duced by direct brain stimulation whichsuggested that the effect of stimulation at aparticular site is to induce a predisposingmotivational condition or state which maylead to a variety of behaviors depending onthe presence or absence of appropriate sup-porting stimuli. More recent work (e.g.,Valenstein, Cox, & Kakolewski, 1970) fur-ther emphasizes the similarity between theterminal state and behavior induced by cen-tral stimulation:Hypothalamic stimulation does not activate onlyone specific behavior pattern. The stimulationseems to excite the substrate for a group of re-sponses that in a given species are related to acommon state [Valenstein et al., 1970, p. 30].


Within this more general framework,stimulus substitution becomes a special case,which simply reflects the fact that the re-sponse normally made to the terminal rein-forcer often becomes highly probable whenthe animal is in a terminal state correspond-ing to that reinforcer, and will occur if mini-mal environmental support is provided.

Interim Activities and Adjunctive Behavior

The principles of variation and reinforce-ment so far discussed refer to the origin andmaintenance of the terminal response. Theinterim activities (including adjunctive be-havior) require a separate although com-plementary account, to which we now turn.

There is as yet no general agreement onthe causal factors underlying adjunctive be-havior (i.e., behavior occurring during theinterim period on a variety of intermittentreinforcement schedules). As we have al-ready noted (p. 13), data presently availableappear to rule out simple physiological inter-pretations (Falk, 1969), although they donot clearly point to any alternative. Somehelp is offered by the similarities betweenadjunctive behavior and displacement activ-ities: the same explanation should be ade-quate for both. Possibilities are also some-what restricted by general functional con-siderations. In the present section, we statea tentative general hypothesis and its em-pirical basis in what is presently known ofadjunctive behavior. The relationship of thishypothesis to accounts of displacement be-havior and to general adaptiveness is dis-cussed in the following section.

The hypothesis may be stated in the formof three propositions:

1. The interim and terminal periods corre-spond to states, in the sense described ear-lier, defined by (a) the class of reinforceror reinforcers that are effective at that time,and (6) the applicability of principles ofvariation appropriate to that class of rein-forcer (e.g., food-related behaviors are likelyto occur during the food state, defense reac-tions during the fear state, etc.).

2. The terminal state corresponds to theterminal reinforcer; the state during theinterim period corresponds to all other rein-forcers, although all need not be equally

effective. The linkage between terminal andinterim states is assumed to be direct andreciprocal, so that the strength (definedbelow in terms of rate) of activities duringthe interim period is directly related to thestrength of the terminal response.

3. The strength of the terminal responseis directly related to the "value" of the re-inforcement schedule; that is, to relative rateand amount of reinforcement and to motiva-tional factors (e.g., deprivation).

As a consequence of the reciprocal inter-action between the terminal and interimstates (Proposition 2) and the dependenceof the terminal response on the value of thereinforcement schedule (Proposition 3), thestrength of behaviors associated with theinterim period will be determined both bythe value of the terminal reinforcementschedule, as well as by the value of the rein-forcers proper to them.

The notion of the interim and terminalperiods as states (Proposition 1) has al-ready been discussed. The dependence ofthe strength of the terminal response onvariables related to the value of the rein-forcement schedule (Proposition 3) shouldalso encounter no opposition. It remains toshow, first, that reinforcers other than theterminal reinforcer are effective during theinterim period; second, that a number ofdifferent reinforcers may be effective atthis time; and third, that the direct relationbetween the strength of the terminal re-sponse and the strength of adjunctive be-havior implied by Propositions 2 and 3 hassome basis in fact. Evidence for the effec-tiveness of reinforcers other than the termi-nal reinforcer, during the interim , period,comes largely from studies of polydipsia(excessive drinking), induced by inter-mittent schedules of food reinforcement, asfollows: (a) Consummatory behavior (e.g.,drinking) occurs in the presence of the ap-propriate goal object (water) during theinterim period on a variety of schedules.(b) This goal object can reinforce operantbehavior:If water is not freely available . , . concurrentlywith a food schedule, but is available in smallportions contingent upon the completion of a fixed-ratio schedule, polydipsia is acquired and will sus-tain large fixed-ratios [Falk, 1970, p. 297].


Azrin (1964, reported in Falk, 1970) hasfound similar schedule control, in pigeons,by a bird provided as a target for schedule-induced aggression, (c) Polydipsic drink-ing can be both increased and decreased byappropriate alteration of the palatability ofthe liquid available (Falk, 1964, 1966).(d) Polydipsia is usually reduced by usingliquid terminal reinforcers. In the caseswhere the liquid used is at least as reinforc-ing to a hungry rat as dry food pellets(e.g., liquid Metrecal, condensed milk,liquid monkey diet; Falk, 1964, 1966;Hawkins, Everett, Githens, & Schrot, 1970;Stein, 1964), this decrease may be attributedto a direct effect on the thirst system (state),due to the water content, as in water pre-loading (see below). Wesson oil as terminalreinforcer is also less effective than foodpellets in producing polydipsia (Strieker &Adair, 1966), although it contains no water,but is probably also less reinforcing and mayreduce polydipsia for this reason (see Prop-ositions 2 and 3). ( e ) Acquisition of poly-dipsia can be prevented by presession stomachloads of water (Chapman, 1969, reported inFalk, 1970), although established polydipsiais little affected. This kind of effect is alsofound with food-motivated terminal re-sponses which, once established, will con-tinue to occur at a somewhat reduced rateeven in the presence of ad lib food (Neur-inger, 1970a).

That more than one reinforcer is effectiveduring the interim period is suggested by thefacts that (a) the interim activities (whichoccur in the absence of appropriate goal ob-jects) have no consistent direction and arenot obviously related to any particular rein-forcer; (b) a variety of goal objects—water,wood shavings, another animal—are suf-ficient to elicit appropriate consummatoryreactions (drinking, chewing or eating, ag-gression) ; (c) physiological data, suggest-ing short-term reciprocal interactions be-tween hunger and thirst drives which wereinduced centrally by electrical or chemicalstimulation (Grossman, 1962; von Hoist &von Saint Paul, 1963), indicate a possiblemechanism for the simultaneous effectivenessof reinforcers other than food at nonfoodtimes (interim periods) and food at food

times (terminal periods). While it is aswell to be cautious in generalizing bothacross species and from physiology to be-havior, these hypothalamic mechanisms areevidently quite similar in birds and mammals(cf. Akerman, Andersson, Fabricius, &Svensson, 1960), and recent work supportsthe similarity between schedule-induceddrinking and drinking induced by directhypothalamic stimulation implied by thesecomparisons (Burks & Fisher, 1970). Thiskind of reciprocal interaction suggests thatat a time when activity motivated by hungeris suppressed (e.g., during the period ofinterim activities on a food schedule), activ-ities motivated by thirst might be facilitated.

That the effective reinforcers during theinterim period are other than the terminalreinforcer is suggested both by the nonoccur-rence of the terminal response at that timeand by data reported by Segal (1969b)showing a shift in the status of drinking—from an adjunctive behavior, occurring earlyin the interval, to a terminal response, occur-ring largely at the end. The changeovertook several experimental sessions, but itsuggests that in the steady state, the sameactivity is unlikely to occur during bothterminal and interim periods, as might beexpected if these periods are associated withthe action of different reinforcers.

Propositions 2 and 3 in combination implythat the strength of adjunctive behaviors,like that of the terminal response, should bedirectly related to the "value" of the rein-forcement schedule, as indexed by motiva-tional and reinforcement variables. The evi-dence in favor of this deduction is as fol-lows: (a) With the interreinforcement inter-val held constant, the amount of polydipsicdrinking in a session of fixed length is in-versely related to body weight (Falk, 1969).Similar results have been reported for sched-ule-induced attack and air licking (Falk,1970). (&) Postpellet pause (until the on-set of polydipsic drinking) increases as afunction of the interval duration (Segal etal, 1965). (c) Rate of licking within adrink bout tends to decrease as a function ofinterval length (Segal et al., 1965). (rf)Rate of polydipsic drinking is increased byincreasing the size of food reinforcement


(Hawkins et al., 1970). Hawkins et al.attribute a contrary result by Falk (1967) tosession length differences between the one-and two-pellet conditions of Falk's experi-ment, (e) Polydipsic drinking falls offdrastically at fixed-interval values longerthan about 3 minutes (Falk, 1966; Segalet al., 1965). Falk (1966) also reports adirect relationship between polydipsia (mea-sured as total amount drunk per session)and fixed-interval length over the range 2-180 seconds. However, based on the resultsof experiments indicating the relative con-stancy of ingestion rate within a drinkingbout in rats (Davis & Keehn, 1959; Schaef-fer & Premack, 1961; Stellar & Hill, 1952),and the different times available for poly-dipsic drinking under different fixed-intervalvalues, Falk's finding of a direct relationshipbetween fixed-interval value and totalamount drunk, over part of the range, iscompatible with an overall inverse relation-ship in terms of rate of drinking. This in-ference is confirmed by recent data reportedby Hawkins et al. (1970) which show amonotonically decreasing ingestion rate asa function of fixed-interval length, over therange 1-5 minutes. Given that overall in-gestion rate is probably better than totalamount drunk as a measure of the tendencyto drink, this finding is both consistent withthe deduction from Propositions 2 and 3and more easily reconciled with other mea-sures that indicate an inverse relationshipbetween tendency to drink and frequency ofreinforcement. Falk's finding that totalamount drunk is maximal at intermediateinterval values may reflect, therefore, anoptimal balance between two factors: tend-ency to drink, which decreases as intervalvalue increases, and time available for drink-ing, which increases with interval value.

The above account, in terms of interactionbetween motivational systems (states),seems to be the simplest that can presentlybe given of both adjunctive behavior anddisplacement activities. Its relationships tothese activities, to the ethological interpre-tation of them, and to general functionalconsiderations are discussed in the nextsection.

Adjunctive Behavior and DisplacementActivities

Most behavioral and morphological char-acteristics are adaptive, in the sense thatthey can be directly related to the reproduc-tive fitness of the organism in its naturalenvironment. Morphological exceptions tothis rule are either the result of "correlatedvariation," in Darwin's phrase, or are vesti-gial characters, in the process of being lost.In neither case do they show the ubiquityand reliability that distinguish adjunctiveand displacement activities. It is very likely,therefore, that these behaviors reflect afunction, or functions, of considerable adap-tive value to animals in the wild. Whatmight this function be?

Learning theories generally consider theorganism to be motivated by one thing at atime, for example, hunger, thirst, explora-tory drive, etc. In like spirit, most learningexperiments are designed to ensure the pre-dominance of one kind of motivation at theexpense of all others. In the wild, however,animals must allocate their time among avariety of activities so as to both satisfy cur-rent wants and anticipate future ones. It isreasonable to assume that there has been con-siderable selection pressure favoring an opti-mal balance among the various possibilities.

We have already noted the fact, relatedto stimulus discrimination, that animals tendto make the terminal response only at timeswhen reinforcement is likely. This fact,and the principle of reinforcement based onit, might seem to reflect some kind of Lawof Effort, since responding at times or placeswhen reinforcement never occurs is ob-viously wasteful. However, a considerableweight of evidence suggests that the Law ofEffort is not a major psychological prin-ciple, since it is easy to devise situations inwhich animals make many more responsesthan necessary (cf. Ferster & Skinner,1957). A Law of Effort principle wouldalso not explain active avoidance of situa-tions associated with nonreinforcement.

A more plausible alternative is that thesefacts are related to animals' need to budgettheir time effectively. In these terms, atime, or stimulus, reliably associated with


the absence of a given reinforcer providesinformation just as useful as a time per-fectly correlated with the delivery of thatreinforcer, since it permits the animal toattend to present and future needs otherthan the one associated with the absent rein-forcer. However, other potentialities of theenvironment cannot usually be sampled aslong as the animal remains in the vicinityof the unavailable reinforcer. One mightexpect, therefore, that natural selection willhave fostered the development of a mecha-nism to ensure that animals avoid places attimes when, on the basis of past experience,they have learned that reinforcement is notforthcoming.

Evolution is notoriously opportunistic inthe sense that adaptation is achieved bywhatever structural or functional meanshappen to be available. In the present case,we suggest that the means for ensuring thatanimals will not linger in the vicinity offood (or other reinforcers) at times when itis not available may be provided by the fa-cilitation of drives other than the blockedone (Propositions 2 and 3) ; that is, thatthe relative aversiveness of the stimuli inthe vicinity of food, during the interim pe-riod, may be a direct effect of the simulta-neous suppression of the food state and fa-cilitation of states associated with other re-inforcers. In the wild, such facilitation willusually ensure that the animal leaves thesituation to seek other reinforcers. More-over, once the animal has left the situation,generalization decrement will ensure thatthe effect of factors acting to facilitate theseother drives is reduced, restoring the animalto a state appropriate to his condition of de-privation and allowing him to take advantageof new opportunities to satisfy the previouslyblocked drive.

In experimental situations showing ad-junctive behavior, however, the animal iskept in the vicinity of the -withheld rein~forcer, both by the physical restraint of theenclosure and, perhaps more importantly, bythe properties of the reinforcement schedule.Since enclosure size has not been explicitlyinvestigated, one cannot be sure of the rela-tive importance of these two factors. In ourlaboratory, we have often observed animals

on time-based schedules and noticed thatpigeons tend to turn away from the key dur-ing the no-pecking phase of schedules suchas the fixed-interval, which yield a periodof no pecking followed by pecking. Fig-ure 1, which shows the birds' orientation(Rj) as a function of postfood time, is aquantitative record of this effect. However,the avoidance of the key is much more com-plete on schedules which require peckingfollowed by no pecking (temporal go-no goschedules, Staddon, 1970a, 1970b), pre-sumably because the no-pecking period isterminated by an external event (deliveryof reinforcement) rather than by the birdreturning to the key to peck. These obser-vations provide some evidence both for thetendency of pigeons to avoid the key at timeswhen key pecking is not reinforced and forthe restriction placed on this tendency byfixed-interval schedules.

In situations involving external (ratherthan temporal) stimuli, there is also con-siderable evidence for the aversive characterof stimuli associated with nonreinforcementwhen they occur in a context associated withreinforcement (e.g., multiple and concurrentreinforcement schedules; cf. Beale & Winton,1970; Catania, 1969; Terrace, 1966).

Thus, the temporal locus of adjunctive(and interim) behavior coincides with aperiod when, by other measures, the situa-tion is aversive to the animal so that hewill withdraw from it if he can. Studies ofschedule-induced escape have shown thatanimals will learn to make a response duringthe interim period on fixed-ratio schedulesthat has the effect of removing them fromthe situation, even though the frequency ofthe terminal reinforcement may thereby bereduced (e.g., Azrin, 1961; Thompson,1964). The present argument suggests thatthese data may reflect a general property ofinterim periods, although experimental re-sults with other schedules are presentlylacking.

Falk (1969, 1970) has ably summarizedthe similarities between adjunctive behaviorand displacement activities, which includethe apparent "irrelevance" of both kinds ofactivity, their association with situations inwhich a strong drive is blocked, and their


modifiability by available stimuli and con-ditions of deprivation other than the majordrive (cf. Morris, 1954; Rowell, 1961;Sevenster, 1961). A study by McFarland(1965) concerning displacement pecking in-duced by preventing drinking in thirsty dovesfurther emphasizes the "state" property ofdisplacement behavior:

The evidence for the view that this pecking be-longs to the feeding system . . . [is] as follows:1. Total time spent pecking is increased by thepresence of grain. 2. Time spent pecking is partlyreplaced by time spent at a specific food gettingactivity, when the birds have previously beentrained to obtain food in this way. 3. Food de-privation . . . increases the time spent peckingwhen grain is present, and this effect is counter-acted by [prefeeding] [p. 298].

The interpretation of adjunctive behaviorthat we have presented is very similar to thedisinhibition hypothesis concerning displace-ment activities first suggested by Andrew(1956). Hinde (1966) summarizes thisview as follows:

When mutual incompatibility prevents the appear-ance of those types of behaviour which wouldotherwise have the highest priority, patterns whichwould otherwise have been suppressed are per-mitted to appear [p. 279].

Displacement behavior is usually exhibitedin approach-avoidance conflict situations(e.g., territory defense, birds returning tothe nest after an alarm, etc.) when theanimal is consequently prevented from leav-ing the situation. We have already -seenthat during the interim period on intermit-tent reinforcement schedules, animals arealso restrained in the situation both by theenclosure and, probably, by the propertiesof the schedule. Both situations thereforemeet the conditions necessary and sufficient,by our hypothesis, for the elevation (ratherthan merely disinhibition) of motivationalstates other than the blocked one, leading inthe schedule case to adjunctive behavior, andin the approach-avoidance case to displace-ment, redirection, or vacuum activities—theparticular activities being a function of thestrength and nature of the blocked response,the proximity to the goal, the stimuli avail-able, the past history of the animal, and theduration of the blocking. This modifica-

tion causes no difficulty in application todisplacement behavior and meets theoreticalobjections to certain forms of the disinhibi-tion hypothesis raised by McFarland (1966).

Finally, our account of the adaptive signif-icance of adjunctive behavior, as a reflectionof the integrative capacities of the organismwhich enable it to strike an efficient balanceamong a number of activities, finds a coun-terpart in McFarland's (1966) account ofthe significance of displacement behavior:

Thus it is suggested that the functional signif-icance of displacement activities is that they arethe by-product of a mechanism which enablesanimals to break away from a specific course ofaction, when progress in that course of actioncomes to a standstill [p. 231].

We conclude, therefore, that interim, ad-junctive, and displacement behaviors may begrouped together on the basis of similarfunctional properties, similar probable causalfactors, and similar adaptive role.

In summary, the argument relating to theinterim period is as follows: (a) Extant dataon adjunctive behavior are consistent witha tentative general interpretation in termsof interactions among motivational systems(states). ( b ) On the basis of generaladaptive considerations, we have suggestedthe probable existence of a mechanism whichenables animals to budget their time effi-ciently (e.g., by giving up temporarily in-effective activities). (c) McFarland hassuggested that displacement and other "irre-levant" activities may reflect the action ofsuch a mechanism, (rf) Falk has pointedout the extensive similarities between ad-junctive and displacement activities, (e)McFarland's suggestion may therefore beextended to interim and adjunctive behav-iors, both on the basis of their resemblanceto displacement activities and their restric-tion to the aversive interim period. Thus,the general interpretation of adjunctive be-havior offered earlier gains additional sup-port from the resemblances between adjunc-tive and displacement activities, from itssimilarity to the disinhibition hypothesis fordisplacement activities, and from its ade-quacy as a mechanism for enabling theanimals to budget their time efficiently.


EPILOGUE

This ends our outline of conditioning.We have dealt with both terminal periods—which, we suggest, reflect a Law of Effectprocess that can best be understood byanalogy with evolution by means of naturalselection—and interim periods—which mayreflect a mechanism enabling animals toallocate their activities efficiently. Learnedbehavior, under the relatively simple condi-tions of reinforcement schedules at least, isviewed as reflecting the sequencing, withrespect to time and stimuli, of terminal andinterim periods; and the scheme is thereforepotentially comprehensive, although nec-essarily incomplete as to details.

Our proposal is founded on the belief thatthe most distinctive thing about living crea-tures is the balance they maintain among anumber of tendencies to action, each oneadaptive, yet each destructive if pursued tothe exclusion of others. This emphasis onthe integration of behavior has required thatthe scheme attempt to be comprehensive andthat it relate in a natural way to biologicaland physiological considerations. Suchmerits as it possesses lie not in formal ele-gance or precision, but in an ability toorganize otherwise unrelated facts and tosuggest gaps where others may possibly befound.

REFERENCES

AKERMAN, B., ANDERSSON, E., FABRICIUS, E., &SVENSSON, L. Observations on central regula-tion of body temperature and of food and waterintake in the pigeon {Columba livia). AdaPhysiologica, Scandinavica, 1960, 50, 328-336.

ANDREW, R. J. Some remarks on behaviour inconflict situations, with special reference toEniberisa Spp. British Journal of Animal Be-haviour, 1956, 4, 41-4S.

ANTONITIS, J. J. Response variability in thewhite rat during conditioning, extinction, andreconditioning. Journal of Experimental Psy-chology, 1951, 42, 273-281.

AZRIN, N. H. Time-out from positive reinforce-ment. Science, 1961, 133, 382-383.

AZRIN, N. H. Aggression. Paper presented atthe meeting of the American PsychologicalAssociation, Los Angeles, September 1964.

AZRIN, N. H., HUTCHINSON, R. R., & HAKE, D. F.Extinction-induced aggression. Journal of theExperimental Analysis of Behavior, 1966, 9,191-204.

BEACH, F. A. The descent of instinct. Psycho-logical Review, 1955, 62, 401-410.

BEALE, I. L., & WINTON, A. S. W. Inhibitorystimulus control in concurrent schedules. Jour-nal of the Experimental Analysis of Behavior,1970, 14, 133-137.

BLACK, A. H. Constraints on the operant con-ditioning of drinking. In, Schedule-induced andschedule-dependent phenomena. Vol. 2. Toronto:Addiction Research Foundation, 1970.

BOLLES, R. C. Species-specific defense reactionsand avoidance learning. Psychological Review,1970, 77, 32-48.

BRELAND, K., & BRELAND, M. The misbehavior oforganisms. American Psychologist, 1961, 16,661-664.

BRELAND, K., & BRELAND, M. Animal behavior.New York: Macmillan, 1966.

BROADBENT, D. E. Behaviour. London: Methuen,1961.

BROWN, P. L., & JENKINS, H. M. Auto-shaping ofthe pigeon's key-peck. Journal of the Experi-mental Analysis of Behavior, 1968, 11, 1-8.

BURKS, C. D. Schedule-induced polydipsia: Areresponse-dependent schedules a limiting condi-tion? Journal of the Experimental Analysis ofBehavior, 1970, 13, 351-358.

BURKS, C. D., & FISHER, A. E. Anticholinergicblockade of schedule-induced polydipsia. Phy-siology and Behavior, 1970, 5, 635-640.

CANE, V. Some ways of describing behaviour.In W. H. Thorpe & O. L. Zangwill (Eds,),Current problems in animal behaviour. Lon-don : Cambridge University Press, 1961.

CATANIA, A. C. Concurrent performances: Inhi-bition of one response by reinforcement ofanother. Journal of the Experimental Analysisof Behavior, 1969, 12, 731-744.

CATANIA, A. C., & REYNOLDS, G. S. A quantita-tive analysis of the responding maintained byinterval schedules of reinforcement. Journal ofthe Experimental Analysis of Behavior, 1968,11 (Ft. 2), 327-383.

CHAPMAN, H. W. Oropharyngeal determinantsof non-regulatory drinking in the rat. Unpub-lished doctoral dissertation, University ofPennsylvania, 1969.

CHOMSKY, N. A review of B. F. Skinner's Ver-bal behavior. Language, 1959, 35, 26-58.

CHOMSKY, N. Explanatory models in linguistics.In E. Nagel, P. Suppes, & A. Tarski (Eds.),Logic, methodology and philosophy of science:Proceedings of the 1960 International Congress.Stanford: Stanford University Press, 1962.

CRAIG, W. Appetites and aversions as constituentsof instincts. Biological Bulletin of the MarineBiological Laboratory, Woods Hole, Mass.,1918, XXXIV, 91-107.

DARWIN, C. The origin of species. Oxford: TheUniversity Press, 1951. (Reprinted from thesixth edition, 1872.)

DARWIN, C. The variation of animals and plantsunder domestication. New York: Appleton,1896. 2 vols.

'SUPERSTITION" EXPERIMENT AND ADAPTIVE BEHAVIOR 41

DAVIS, J. D., & KEEHN, J. D. Magnitude of re-inforcement and consummatory behavior.Science, 19S9, 130, 269-270.

ESTES, W. K. Of models and men. AmericanPsychologist, 1957, 12, 609-617.

FALK, J. L. Studies on schedule-induced poly-dipsia. In M. J. Wayner (Ed.), Thirst: Firstinternational symposium on thirst in the regula-tion of body water. New York: PergamonPress, 1964.

FALK, Jt' L. Schedule-induced polydipsia as afunction of fixed interval length. Journal of theExperimental Analysis of Behavior, 1966, 9,37-39.

FALK, J. L. Control of schedule-induced polydip-sia: Type, size, and spacing of meals. Journalof the Experimental Analysis of Behavior, 1967,10, 199-206.

FALK, J. L. Conditions producing psychogenicpolydipsia in animals. Annals of the New YorkAcademy of Sciences, 1969, 157, 569-593.

FALK, J. L. The nature and determinants of ad-junctive behavior. In, Schedule-induced andschedule-dependent phenomena. Vol. 2. Toronto:Addiction Research Foundation, 1970.

FANTINO, E., & COLE, M. Sand-digging in mice:Functional autonomy? Psychonomic Science,1968, 10, 29-30.

FERSTER, C. B., & SKINNER, B. F. Schedules ofreinforcement. New York: Appleton-Century-Crofts, 1957.

FISHER, R. A. The genetical theory of naturalselection. Oxford: Clarendon Press, 1930.

FLORY, R. Attack behavior as a function ofminimum inter-food interval. Journal of theExperimental Analysis of Behavior, 1969, 12,825-828.

FREED, E. X., & HYMOWITZ, N. A fortuitousobservation regarding "psychogenic" polydipsia.Psychological Reports, 1969, 24, 224-226.

GARCIA, J., ERVIN, F. R., & KOELLING, R. Learn-ing with prolonged delay of reinforcement. Psy-chonomic Science, 1966, S, 121-122.

GHISELIN, M. T. The triumph of the Danvinianmethod. Berkeley: University of CaliforniaPress, 1969.

GILBERT, R. M. Psychology and biology. Cana-dian Psychologist, 1970, 11, 221-238.

GLICKMAN, S. E., & SCHIFF, B. B. A biologicaltheory of reinforcement. Psychological Review,1967, 74, 81-109.

GLICKMAN, S. E., & SROGES, R. W. Curiosity inzoo animals. Behaviour, 1966, 26, 151-188.

GONZALEZ, R. C., BEHREND, E. R., & BITTERMAN,M. E. Reversal learning and forgetting in birdand fish. Science, 1967, 158, 519-521.

GROSSMAN, S. P. Direct adrenergic and choliner-gic stimulation of hypothalamic mechanisms.American Journal of Physiology, 1962, 202, 872-882.

GUTTMAN, N., & KALISH, H. I. Discriminabilityand stimulus generalization. Journal of Experi-mental Psychology, 1956, 51, 79-88.

HALDANE, J. B. S. Natural selection. In P. R.Bell (Ed.), Darwin's biological work. Cam-bridge : University Press, 1959.

HARLOW, H. F. Learning set and error factortheory. In S. Koch (Ed.), Psychology: A studyof a science. Vol. 2. New York: McGraw-Hill,1959.

HAWKINS, T. D., EVERETT, P. B., GITHENS, S. H.,& SCHROT, J. F. Adjunctive drinking: A func-tional analysis of water and alcohol ingestion.In, Schedule-induced and schedule-dependentphenomena. Vol. 1. Toronto: Addiction Re-search Foundation, 1970.

HERRNSTEIN, R. J. "Will." Proceedings of theAmerican Philosophical Society, 1964, 108, 455-458.

HERRNSTEIN, R. J. Superstition: A corollary ofthe principles of operant conditioning. In W. K.Honig (Ed.), Operant behavior: Areas of re-search and application. New York: Appleton-Century-Crofts, 1966.

HERRNSTEIN, R. J. On the law of effect. Journalof the Experimental Analysis of Behavior, 1970,13, 243-266.

HERRNSTEIN, R. J., & HINELINE, P. N. Negativereinforcement as shock-frequency reduction.Journal of Experimental Analysis of Behavior,1966, 9, 421-430.

HILGARD, E. R., & MARQUIS, D. G. Conditioningand learning. New York: Appleton-Century,1940.

HINDE, R. A. Animal behaviour: A synthesis ofethology and comparative psychology. NewYork: McGraw-Hill, 1966.

HUTCHINSON, R. The production and maintenanceof behavior by shock and shock-associated stim-uli. In R. Ulrich (Chm.), The maintenance ofresponding through the presentation of electricshocks. Symposium presented at the meetingof the American Psychological Association, Mi-ami, September 1970.

JENKINS, H. M. Sequential organization in sched-ules of reinforcement. In W. N. Schoenfeld &J. Farmer (Eds.), Theory of reinforcementschedules. New York: Appleton-Century-Crofts,1970.

KALAT, J. W., & ROZIN, P. "Salience": A factorwhich can override temporal contiguity in taste-aversion learning. Journal of Comparative andPhysiological Psychology, 1970, 71, 192-197.

KEEHN, J. D. Beyond the law of effect. In,Schedule-induced and schedule-dependent phe-nomena. Vol. 1. Toronto: Addiction ResearchFoundation, 1970.

KIMBLE, G. A. Hilgard and Marquis' condition-ing and learning. New York: Appleton-Cen-tury-Crofts, 1961.

LEVINE, M. Hypothesis behavior. In A. M.Schrier, H. F. Harlow, & F. Stollnitz (Eds.),Behavior of nonhuman primates. Vol. 1. NewYork: Academic Press, 1965.

LICKLIDER, J. C. R. On psychophysiologicalmodels. In W. A. Rosenblith (Ed.), Sensory


communication. Cambridge: M.I.T. Press,1961.

MAcCoRQUODALE, K. B. F. Skinner's Verbal be-havior: A retrospective appreciation. Journal ofthe Experimental Analysis of Behavior, 1969,12, 831-841.

MANDLER, G. The interruption of behavior.Nebraska Symposium on Motivation, 1964, 12,163-219.

MARTIN, L, & LEVEY, A. B. The genesis of theclassical conditioned response. Oxford: Perga-raon Press, 1969.

MAYR, E. The emergence of evolutionary novel-ties. In S. Tax (Ed.), The evolution of life.Vol. 1. Chicago: University Press, 1960.

MAYR, E. Animal species and evolution. Cam-bridge: Harvard University Press, 1963.

McFARLAND, D. J. Hunger, thirst and displace-ment pecking in the barbary dove. Animal Be-haviour, 1965, 13, 293-300.

MCFARLAND, D. J. On the causal and functionalsignificance of displacement activities. Zeit-schrift fur Tierpsychologie, 1966, 23, 217-235.

McNEiLL, D. On theories of language acquisition.In T. R. Dixon & D. L. Horton (Eds.), Ver-bal behavior and general behavior theory. Engle-wood Cliffs, N.J.: Prentice-Hall, 1968.

MILLENSON, J. R., & HURWITZ, H. M. B. Sometemporal and sequential properties of behaviordaring conditioning and extinction. Journal ofthe Experimental Analysis of Behavior, 1961, 4,97-106.

MILLER, N. E, Learning of visceral and glandularresponses. Science, 1969, 163, 434-445.

MINSKY, M. Steps toward artificial intelligence.Proceedings of the Institute of Radio Engineers,1961, 49, 10-30.

MORRIS, D. The reproductive behaviour of theZebra finch (Poephila guttata), with specialreference to pseudofemale behaviour and dis-placement activities. Behaviour, 1954, 6, 271-322.

MORRIS, D. The reproductive behaviour of theTen-spined Stickleback (Pygosteus pungitiusL.). Behaviour, 1958, Supplement VI, 1-154.

NEIMARK, E. D., & ESTES, W. K. Stimulussampling theory. San Francisco: Holden-Day,1967.

NEISSER, U. Cognitive psychology. New York:Appleton-Ccntury-Crofts, 1967.

NEURINGER, A. J. Many responses per food re-ward with free food present. Science, 1970,169, 503-504. (a)

NEURINGER, A. J. Superstitious key pecking afterthree peck-produced reinforcements. Journal ofthe Experimental Analysis of Behavior, 1970,13, 127-134. (b)

PAVLOV, I. P. Conditioned reflexes. (Trans, byG. V. Anrep.) London: Oxford UniversityPress, 1927.

PFAFFMAN, C. The behavioral science model.American Psychologist, 1970, 25, 437-441.

PRINGLE, J. W. S. On the parallel between learn-ing and evolution. Behaviour, 1951, 3, 174-215.

PRYOR, K. W., HAAG, R., & O'REILLY, J. Thecreative porpoise: Training for novel behavior.Journal of the Experimental Analysis of Be-havior, 1969, 12, 653-661.

RACHLIN, H. Autoshaping of key pecking inpigeons with negative reinforcement. Journalof the Experimental Analysis of Behavior, 1969,12, 521-531.

RESCORLA, R. A. Pavlovian conditioning and itsproper control procedures. Psychological Re-view, 1967, 74, 71-80.

REVUSKY, S. H., & BEDARF, E. W. Associationof illness with prior ingestion of novel foods.Science, 1967, 155, 219-220.

REYNIERSE, J. H., & SPANIER, D. Excessivedrinking in rats' adaptation to the schedule offeeding. Psychonomic Science, 1968, 10, 95-96.

ROSENBLITH, J. Z. Polydipsia induced in the ratby a second-order schedule. Journal of the Ex-perimental Analysis of Behavior, 1970, 14, 139-144.

ROWELL, C. H. F. Displacement grooming in theChaffinch. Animal Behaviour, 1961, 9, 38-63.

ROZIN, P. Central or peripheral mediation oflearning with long CS-US intervals in the feed-ing system. Journal of Comparative and Phy-siological Psychology, 1969, 67, 421-429.

SCHAEFFER, R. W., & PREMACK, D. Licking ratesin infant albino rats. Science, 1961, 134, 1980-1981.

SEGAL, E. The interaction of psychogenic poly-dipsia with wheel running in rats. PsychonomicScience, 1969, 14, 141-144. (a)

SEGAL, E. Transformation of polydipsic drinkinginto operant drinking. A paradigm? Psycho-nomic Science, 1969, 16, 133-135. (b)

SEGAL, E. Speculations on the provenance ofoperants. In, Schedule-induced and schedule-dependent phenomena. Vol. 2. Toronto: Addic-tion Research Foundation, 1970.

SEGAL, E., & HOLLOWAY, S. M. Timing be-havior in rats with water drinking as a mediator.Science, 1963, 140, 888-889.

SEGAL, E., ODEN, D. L., & DEADWYLER, S. A.Determinants of polydipsia: IV. Free-reinforce-ment schedules. Psychonomic Science, 1965,3, 11-12.

SEVENSTER, P. A causal analysis of a displace-ment activity (Fanning in Gasterosteus Aculea-tusL.*). Behaviour, 1961, Supplement IX, 1-170.

SHANAB, M. E., & PETERSON, J. L. Polydipsia inthe pigeon. Psychonomic Science, 1969, I S , 51-52.

SHEFFIELD, F. S. Relation between classical con-ditioning and instrumental learning. In W. F.Prokasy (Ed.), Classical conditioning: A sym-posium. New York: Appleton-Century-Crofts,1965.

SHIMP, C. P. Optimal behavior in free-operantexperiments. Psychological Review, 1969, 76,97-112.

SIDMAN, M., & FLETCHER, F. G. A demonstrationof auto-shaping with monkeys. Journal of the


Experimental Analysis of Behavior, 1968, 11,307-309.

SIMMELHAG, V. L. The form and distribution ofresponding in pigeons on response-independentfixed and variable interval schedules of reinforce-ment. Unpublished master's thesis, Universityof Toronto, 1968.

SIMPSON, G. G. The major features of evolution.New York: Columbia University Press, 19S3.

SKINNER, B. F. The behavior of organisms. NewYork: Appleton-Century, 1938.

SKINNER, B. F. "Superstition" in the pigeon.Journal of Experimental Psychology, 1948, 38,168-172.

SKINNER, B. F. Science and human behavior.New York: Macmillan, 1953.

SKINNER, B. F. Reinforcement today. AmericanPsychologist, 1958, 13, 94-99.

SKINNER, B. F. An experimental analysis of cer-tain emotions. Journal of the ExperimentalAnalysis of Behavior, 1959, 2, 264.

SKINNER, B. F. Cumulative record. New York:Appleton*Century-Crofts, 1961.

SKINNER, B. F. Operant behavior. In W. K.Honig (Ed.), Operant behavior: Areas of re-search and application. New York: Appleton-Century-Crofts, 1966. (a)

SKINNER, B. F. The phytogeny and ontogeny ofbehavior. Science, 1966, 153, 1205-1213. (b)

SKINNER, B. F. Contingencies of reinforcement.New York: Appleton-Century-Crofts, 1969.

SKINNER, B. F., & MORSE, W. H. Fixed-intervalreinforcement of running in a wheel. Journalof the Experimental Analysis of Behavior, 1958,1, 371-379.

STADDON, J. E. R. Some properties of spacedresponding in pigeons. Journal of the Experi-mental Analysis of Behavior, 1965, 8, 19-27.

STADDON, J. E. R. Asymptotic behavior: Theconcept of the operant. Psychological Review,1967, 74, 377-391.

STADDON, J. E. R. Inhibition and the operant.A review of G. v. Bekesy, Sensory inhibition,and F. Ratliff, Mach bands: Quantitative stud-ies on neural networks in the retina. Journalof the Experimental Analysis of Behavior, 1969,12, 481-487.

STADDON, J. E. R. Reinforcement after-effects.In, Schedule-induced and schedule-dependentphenomena. Vol. 2. Toronto: Addiction Re-search Foundation, 1970. (a)

STADDON, J. E. R. Temporal effects of rein-forcement: A negative "frustration" effect.Learning and Motivation, 1970, 1, 227-247. (b)

STEIN, L. Excessive drinking in the rat: Super-stition or thirst? Journal of Comparative andPhysiological Psychology, 1964, 58, 237-242.

STELLAR, E., & HILL, J. H. The rat's rate ofdrinking as a function of water deprivation.Journal of Comparative and Physiological Psy-chology, 1952, 45, 96-102.

STRICKER, E. M., & ADAIR, E. R. Body fluidbalance, taste, and postprandial factors in sched-

ule-induced polydipsia. Journal of Compara-tive and Physiological Psychology, 1966, 62,449-454.

TERRACE, H. S. Stimulus control. In W. K.Honig (Ed.), Operant behavior: Areas of re-search and application. New York: Appleton-Century-Crofts, 1966.

THOMPSON, D. M. Escape from SD associatedwith fixed-ratio reinforcement. Journal of theExperimental Analysis of Behavior, 1964, 7,1-8.

TINBERGEN, N. "Derived" activities: Their causa-tion, biological significance, origin, and eman-cipation during evolution. Quarterly Review ofBiology, 1952, 27, 1-32.

TULVING, E., & MADIGAN, S. A. Memory andverbal learning. Annual Review of Psychology,1970, 21, 437-484.

UPTON, M. The auditory sensitivity of guineapigs. American Journal of Psychology, 1929,41, 412-421.

VALENSTEIN, E. S., Cox, V. C, & KAKOLEWSKI,J. W. Reexamination of the role of the hypo-thalamus in motivation. Psychological Review,1970, 77, 16-31.

VERPLANCK, W. S. Since learned behavior is in-nate, and vice versa, what now? PsychologicalReview, 1955, 52, 139-144.

VON HOLST, E., & VON SAINT PAUL, U. On thefunctional organisation of drives. Animal Be-haviour, 1963, 11, 1-20.

WILLIAMS, D. R. Classical conditioning and in-centive motivation. In W. F. Prokasy (Ed.),Classical conditioning: A symposium. NewYork: Appleton-Century-Crofts, 1965.

WILLIAMS, D. R., & WILLIAMS, H. Auto-main-tenance in the pigeon: Sustained pecking despitecontingent non-reinforcement. Journal of theExperimental Analysis of Behavior, 1969, 12,511-520.

WOLIN, B. R. Difference in manner of pecking akey between pigeons reinforced with food andwith water. In A. C. Catania (Ed.), Contem-porary research in operant behavior. New York:Scott, Foresman, 1968.

WRIGHT, S. Evolution in Mendelian populations.Genetics, 1931, 16, 97-159.

WUTTKE, W. The effects of d-amphetamine onschedule-controlled water licking in the squirrel-monkey. Psychopharmacologia (Berlin), 1970,17, 70-82.

ZEAMAN, D., & SMITH, R. W. Review of somerecent findings in human cardiac conditioning.In W. F. Prokasy (Ed.), Classical conditioning:A symposium. New York: Appleton-Century-Crofts, 1965.

ZENES, K. The significance of behavior accom-panying conditioned salivary secretion fortheories of the conditioned response. AmericanJournal of Psychology, 1937, 50, 384-403.

(Received March 19, 1970)

THE SUPERSTITION EXPERIMENT: A...

Documents

Transcript of THE SUPERSTITION EXPERIMENT: A...