Chapter 11

Learned Reinforcers and Learned Aversive Stimuli

Concept

Behavioral School Psychology

THE TOKEN ECONOMY

AND REMEDIAL EDUCATION Dr. Montrose Wolf and his colleagues at the University of Kansas headed a project to help culturally deprived

children hurdle the barriers to a sound education. They helped 16 fifth-grade and sixth-grade students, who were all

at least 2 years below their grade level on a reading achievement test. The children were from a low-income

neighborhood in Kansas City. Most were from families of more than five children. The families got welfare support,

and often no father was present. The parent or parents voluntarily enrolled the children in Mont’s remedial class.1

Mont and his colleagues set up the classroom in the basement of a church. During the school year, the students

attended the special classroom each weekday after school and on Saturday mornings for 21⁄2 years. During summer,

they attended school every morning, except Sunday.

They used a token economy where the learned generalized reinforcers consisted of checkmarks the teacher

placed in the students’ folders after they had finished an assignment correctly. When the children first joined the

program, the teacher gave them this learned generalized reinforcer after the satisfactory completion of each problem.

As the students achieved a higher frequency of work and a more accurate output, the work needed to obtain the

learned generalized reinforcers gradually increased in amount and difficulty. Sometimes students negotiated with the

teacher the number of checkmarks a particular amount of work would earn.

The students could exchange these learned generalized reinforcers for various backup reinforcers:

o weekly field trips involving circuses, swimming, zoos, picnics, sporting events, and movies

o daily snacks, sandwiches, fruit, milk, and cookies, for exchange of the check marked pages

o money and items available in the school store, such as candy, toiletries and novelties; more expensive

reinforcers, with the exchange of large numbers of tokens, such as clothing, watches, and secondhand

bicycles

o a shopping trip to a local department store for any child who accumulated two dollars’ worth of

checkmarks

The children earned these tokens for three general types of activities:

o completed work from the regular classroom

o completed homework assignments and remedial work in the remedial classroom

o good 6-week report card grades (an A paid off maximally and an F paid nothing at all)

The teachers also used other reinforcer-based procedures with the students:

o They made participation in favorite subjects or academic activities contingent on the completion of less

favored academic work. (Thus the opportunity to do popular academic work was a reinforcer for doing less

popular academic work. Tricky, eh?)

o They often reinforced academic productivity by letting the productive students instruct other students in

their deficient areas.

o They allowed the productive students to grade the assignments of the other students. Even trickier, eh?

o They also gave a large bonus each month to every student who had perfect attendance for that period. The

bonus was cumulative: For 2 months’ perfect attendance, the bonus was twice as large; for 3 months’

perfect attendance, the bonus was three times as large; and so forth.

o They held a party every 6 weeks for the students whose grade average improved. The parties involved such

activities as dining in a restaurant, camping, and going on plane rides.

1 Based on Wolf, M., Gives, D. K., & Hall, R V. (1967). Experiments with token reinforcements in a remedial classroom. Unpublished

manuscript.

o On Saturdays, the students formed group games similar to TV college quiz contests with two students on

each team. They competed against one another for the bonus checkmarks the teacher gave to the team with

the most correct responses. The students could choose their partners; and, of course, everyone wanted a

teammate who could answer the most questions—the academically skilled. The regular students sought the

good scholars as heroes, much as they sought the good athletes on the playground.

o As if this were not enough, the behavior analysts developed an additional method for the presentation of

social reinforcers for good academic behavior. Each group of students, who worked with one instructor,

competed with the other groups in accumulating public school tests with grades of A. Whenever a student

brought in an A paper, the teacher announced it in class and tacked the paper above the student’s desk.

Each Saturday, the team with the most papers received reinforcers; the members of the team got the candy

bars of their choice. The notion of Saturday’s hero now has a new meaning. The mere idea of such an

environment should warm the hearts of the scholars among you.

In addition, the behavior analysts used a few punishment contingencies to decrease inappropriate behavior:

o An alarm clock rang at three random times during each 21⁄2-hour session. Students got a negative mark after

their names on the blackboard if they were out of their seats when the alarm rang.

o They also got negative marks for any other disruptive behavior, such as hitting another student.

o Also, the teachers in the public school classrooms could give points and remove store privileges from

students who attended their classes. They did this by sending reports to the remedial classroom teacher.2

At the end of the day, the student with the fewest negative marks earned a large number of extra positive

checkmarks in his or her folder. To put it another way, the other students lost the opportunity to earn those extra

reinforcers. That wasn’t all: Any student who got more than four negative marks lost a privilege, such as the use of

the store at the end of the day.3

The behavior analysts also used reinforcers to encourage the parents to support the academic behavior of their

children. Wolf and his crew included in the store items of interest to the students’ families. The students could

purchase those items with the checkmarks they had earned for good work.

The program also involved the use of learned generalized reinforcers for the instructors. So, in that way, Mont

Wolf and his colleagues also supported effective instruction. They gave a bonus of 10 dollars to the assistant

instructors whenever a student brought in a 6-week report card with a grade average higher than that of the previous

6 weeks.

This is a great arrangement, probably something only a person skilled in the use of learned generalized

reinforcers could dream up. But was it effective? Yes. Students often asked to continue their academic work after

the remedial session. Also, the students attended about 85% of the remedial classes, though the program regularly

met on Saturdays and most holidays. (The students voted to work on school holidays. However, the instructors drew

the line at working on Thanksgiving and Christmas day.)

The students worked hard, but did they learn? Yes, the results were impressive. During each of the preceding

2 years, the students had advanced 0.6 grade levels per year on a scholastic aptitude test. During the year of the

token economy, the typical gain was 1.5 grade levels per year on the SAT. A similar group of students (the control

group) who had not been in the learned generalized reinforcer program showed a gain of only 0.8 grade levels per

year in that same period of time. During that year, the report card grade average improved from a D to a C, while the

comparison group showed practically no improvement.4 (See Figure 11.3.)

<Insert figure 11.3 from page 194 about here>

Figure 11.3 Tokens Improve Remedial Education

2 Probably, the teachers in the public school classrooms would tell the students immediately after the relevant response that they were going to

give or remove points, though the actual giving or removing would be done much later by the remedial classroom teacher. And probably a

teacher’s immediate statement would be a learned reinforcer or a learned aversive condition, depending on whether it involved the promise of giving or removing the points. But the delay between giving and removing and the actual giving and removing are undoubtedly too great for the

statements to have acquired their reinforcing and aversive values through the simple pairing procedure defined earlier in this chapter. Instead,

those values must have been acquired through some sort of verbal, rule-governed analog to a pairing procedure, a pairing procedure that would not work with animals and nonverbal human beings.

3 Again, the extra reinforcing and aversive properties these contingencies added to the positive and negative marks must also have been the

result of some sort of complex, verbal, rule-governed analog to a pairing procedure.

4 Of the 16 students, unfortunately, one of the older 6th graders dropped out during the spring term. She married and dropped out of school.

So these results don’t apply to her.

Was it cost-effective? Was it worth it? Each student earned $250 during the school year—a small amount of

learned generalized reinforcers for the large reinforcer going to the society that makes valuable citizens out of

people who might otherwise be lost.

QUESTIONS 1. Describe the use of learned generalized reinforcers to help remedial grade school students:

a. What were the learned generalized reinforcers? b. What were some backup reinforcers? c. What were three different reinforcement procedures used? d. What was a punishment contingency? e. How did the behavior analysts encourage social reinforcement in the classroom? f. How did they encourage the parent’s support of the student’s academic performance? g. What were the academic results? 2. Token economy—define it.

VALUES AND TASTES What follows is a set of quotes by famous authors. The quotes are about values and taste. I’ve inserted in bold,

rephrasing in terms of reinforcers and aversive stimuli to illustrate this point (what I replaced are the italicized,

parenthetical words): By values and tastes we mean reinforcers and aversive stimuli, both unlearned and

learned. In this way we can see the relevance of behavior analysis to a major human concern—our values. Of

course, these famous authors might roll over in their graves (or her bed, in Sontag’s case), if they knew I had

behavioralized their works. But we hope you enjoy it.

Values The least pain in our little finger is more aversive to us (gives us more concern and uneasiness) than the

destruction of millions of our fellow-beings—William Hazlitt (1778–1830).

Home is not a reinforcer (You can’t appreciate home) till you’ve left it, money till it’s spent, your wife till

she’s joined a woman’s club, nor Old Glory till you see it hanging on a broomstick on the shanty of a consul

in a foreign town—O. Henry (1862–1910).

The three biggest reinforcers (most important things) a man has are, briefly, his private parts, his money, and

his religious opinions—Samuel Butler (1835–1902).

Taste The discovery of the reinforcing value (good taste) of bad taste can be very liberating. The man who insists

on high and serious reinforcers (pleasures) is depriving himself of reinforcers (pleasure); he continually

restricts what he can enjoy; in the constant exercise of his good taste he will eventually price himself out of the

market, so to speak.—Susan Sontag (b. 1933).

Taste is nothing but an enlarged capacity for being reinforced by (receiving pleasure from) works of

imagination—William Hazlitt (1778–

1830).

I cannot cure myself of that most woeful of youth’s follies—thinking that those who care about us will find

reinforcing (care for) the things that we find reinforcing (mean much to us)—D. H. Lawrence (1885–1930).

So when we say we value honor, truth, justice, and beauty, we mean honor, truth, justice, and beauty are

powerful reinforcers for us, and their violation is a powerfully aversive event. And when people repeat the old

hackneyed cop-out, there’s no accounting for taste, they are not taking into account the science of behavior analysis

as illustrated in this chapter. The concepts of the value-altering pairing procedure and learned reinforcers does allow

us to account for taste. So we might say, There’s no accounting for taste, unless you’re a student of behavior

analysis.

And here’s the point of this section: Behavior analysis provides a powerful set of concepts and tools with

which we can think about, analyze, and even improve our world. It allows us to address age-old concerns of

humankind in new, exciting, productive ways. It provides us with a behavior-analytic worldview. And viewing the

world of values and taste in terms of learned and unlearned reinforcers and aversive stimuli is just one humble

instance of this behavior-analytic worldview.

IMPRINTED REINFORCERS

The Chick and the Beer Bottle I learned about the process of imprinting in my first term of teaching at Denison University. Several of us were

teaching sections of introductory psychology. All the other faculty members wanted to have a live classroom

demonstration of the principles of behavior. I was reluctant, because the first principle of science education is

“Never do experimental demonstrations in class.” No matter how many times you have done it in private, as soon as

you get in front of a group of students the experiment fails. It’s much safer to do the demonstration on the

blackboard.

My colleagues decided we should show imprinting. I opposed this because none of us had ever seen such a

demonstration. Imprinting may be real, but we had no idea of the problems involved in such a demonstration.

Nonetheless, my colleagues, eagerly, and I, reluctantly, began preparing the demonstration.

We needed newly hatched chicks, just a few hours old. We should have deprived the chicks of most visual

stimulation from the time they hatched. Such creatures are not common around psych labs, but it happened we had

them. By coincidence, some scientists in our department were doing research that involved hatching and rearing

chicks in the dark. They were generous enough to take care of our needs.

I remember well the demonstration of imprinting with the chick in my class. I removed the animal from the

lightproof transportation box and placed it in the center of a large 4-by-10-foot table with a 1-foot-high cardboard

wall around the perimeter. The 30 students in the class immediately crowded around the table to watch. In spite of

my insistence that they maintain absolute silence, the students were unable to do so. Their noise and their eager

faces peering at the chick over the cardboard wall would ruin the experiment. But I went on with the futile

demonstration. I took an empty beer bottle, tied it to one end of a string, and attached the string to a pole. Then I

dangled the beer bottle in front of the chick, and after a few seconds, it began to approach the bottle. As the bottle

moved around in the enclosed space, the chick followed close behind; no matter where the bottle moved, the chick

was right there. The demonstration amazed the students, but not nearly so much as it amazed me. This is a

demonstration of the imprinted reinforcer—a reinforcer that acquires its unlearned reinforcing properties as a

result of being the first stimulus the organism contacts during a brief period shortly after birth. The first object the chick saw was the beer bottle. From that time on, the presence of the beer bottle was a

strong reinforcer. The presence of the bottle reinforced the chick’s response of following it. Normally, the first thing

a chick sees is the mother hen; and the presence of the hen becomes reinforcing. So the chick spends most of its time

in the presence of that reinforcer. That is the most common course of events; but when we intervened, the chick

followed the beer bottle. Researchers have shown imprinting with a variety of birds and a variety of imprinted

reinforcers. For example, in some situations, a person has become the imprinted reinforcer for the bird; then the bird

may constantly follow the person.

Bateson and Reese showed that the imprinted stimulus is truly a reinforcer; they used an imprinted reinforcer

to reinforce a key-peck response. The sight of the imprinted mother is a reinforcer, and the bird will do almost

anything that keeps the reinforcing sight in view. It will even peck a special response key, if that’ll keep the mother

in view. But no one other than a behavior analyst would require the bird to peck a key; nature required only that a

bird follow its mother to keep her in sight.5

By the way, we’ve appended this section on imprinting to this learned reinforcer chapter, because we think it’s

a useful concept to have. Not because it’s an example of a learned reinforcer. It’s not; we might call it an acquired

reinforcer though; but, in any case, we don’t have a better place to put it.

And by the by way, why isn’t imprinted stimulus a learned reinforcer? Why isn’t the beer bottle a learned

reinforcer? Because the procedure for creating an imprinted reinforcer doesn’t fit the definition of the procedure for

creating a learned reinforcer. Remember, a learned reinforcer (activity,) becomes a reinforcer because it has been

paired with another reinforcer. But an imprinted reinforcer isn’t paired with anything. A stimulus simply becomes an

imprinted reinforcer by being the first stimulus the organism contacts during a brief period shortly after birth. And

that’s not the way a learned reinforcer acquires its reinforcing properties. For the beer bottle to have become a

learned reinforcer, it would have to have been paired with another reinforcer, like beer, for you, or chicken feed for

the chick. But it wasn’t. It was just there, hanging from a string, its increasing proximity immediately reinforcing the

chicks approaching it and following it around, right out of the box.

5 Bateson, T. G., & Reese, E. P. (1969). The reinforcing properties of conspicuous stimuli in the imprinting situation. Animal Behaviour, 17,

692–699. 14 Zimmerman, J. & Hanford, P.V. (1966). Sustaining behavior with conditioned reinforcement as the only response-produced consequence.

Psychological Reports, 19, 391-401.

Why do we say the imprinted reinforcer is an acquired reinforcer, but not a learned reinforcer? The imprinted

reinforcer is not an unlearned reinforcer, because it’s not a reinforcer when the chick is born. And yet, it’s not a

learned reinforcer, because it doesn’t need to be paired with another reinforcer to get its reinforcing value, as is the

case with learned reinforcers. So we use the term imprinted reinforcer to describe those reinforcers that become

reinforcing just as a result of early exposure. And we use the term acquired reinforcer to indicate that it is not

unlearned but it is also not what we’d call a learned reinforcer.

QUESTION 1. Imprinted reinforcer—describe a demonstration of it.