Chapter 9

Adjusting to Schedules of Partial Reinforcement

Schedules of Reinforcement Specify the conditions the individual

has to satisfy in order to obtain the reinforcing event

These conditions can include… How many times the individual must

perform the behavior When the behavior has to be performed Performing a certain pattern of behaviors

Classifying Schedules of Reinforcement

Continuous Reinforcement = provides the consequence for each occurrence of the target behavior

Schedule of Partial (Intermittent) Reinforcement = specifies which instances of the target behavior are followed by the consequence

It is the type of schedule established when not every instance of the target behavior is followed by the consequence

Most of these schedules require that the individual either perform the target behavior a certain number of times, perform that behavior after a certain time interval has elapsed or some combo of both

e.g., The Lottery takes advantage of the fact that some individuals will continue to play even though they win only occasionally

3 Most Common Classes of Schedules 1) Ratio Schedules = based on the

number of individual actions required to obtain a reinforcer

2) Interval Schedules = behaving after a certain time interval has elapsed in order to obtain a reinforcer

3) Interresponse Time Schedules = spacing individual actions a certain time apart in order to be rewarded

Ratio Schedules Specify the number of times an individual has to do

something in order to obtain a certain consequence That is the only requirement The consequence occurs each time the requirement

is satisfied Direct Variables = formally imposed requirements of

a schedule Indirect Variables = unstated requirements These variables affect how the individual adjusts to it

e.g., in all ratio schedules, the faster the individual gets through the ratio, the faster he or she earns the consequence

Ratio-Schedules Cont. Ratio schedules tie the rate of reinforcement

directly to the rate of behavior Molar Feedback Functions = define the

relationship b/w the rate of performance of the target behavior and the rate at which the consequence is earned

These are called molar functions b/c they relate the overall rate of reinforcement to the overall rate of behavior

Molar feedback functions for ratio schedules encourage high rates of behavior b/c pausing & dawdling delay earning the next reward

Molecular Feedback = involves the moment-to-moment relationship b/w behaviors and consequences

Ratio Schedules Cont. Ratio schedules are more likely to

reinforce short interresponse times (IRTs) rather than long ones b/c the IRT that is followed immediately by reinforcement is likely to be a short one, and short IRTs are synonymous with rapid behavior

Most common types of ration schedules: 1) Fixed Ratio 2) Variable Ratio

Fixed-Ratio Schedules (FR n)

The number of times the target behavior must occur is the same across successive reinforcers

Symbolized as FR n where n specifies the required number of times the target behavior must occur in order to obtain the consequence e.g., An FR 5 schedule means that the individual

must always perform the required action 5 times in order to obtain a certain consequence

Continuous Reinforcement – is a special case of a fixed-ratio schedule in the which the ratio requirement is 1 (FR 1)

Typical Performance on Fixed-Ratio Schedules

“Break and Run” Pattern: the “break” refers to a pause for a period of time after the presentation of each consequence

The length of this pause called a Post-Reinforcement Pause (PRP) is typically an increasing function of the fixed-ratio requirement: the longer the required ratio, the longer the pause

Following the PRP (the “break”), the individual typically makes an abrupt transition into the “run,” a high steady rate of behavior that ends with the delivery of reinforcement

Why there is a post-reinforcement pause on fixed-ratio schedules

The time b/w the receipt of successive reinforcers called the Interreinforcer Interval depends on how long the individual takes to complete each action and the number of actions required by the schedule

This interval affects the length of the post-reinforcement pause (PRP): the longer the interreinforcer interval, the longer the PRP

Examples of Fixed-Ratio Schedules in Everyday Life Working on Commission, being paid for

the number of tasks or objects completed (piece work), or receiving a bonus for each item or task above the minimum number

In all these 3 cases, the amount of pay earned depends on the number of items sold (or made) based on a pay schedule which specifies how much is earned per sale or item produced

Variable-Ratio Schedules (VR n)

Require the individual to perform the target behavior a certain number of times to obtain the consequence, but that number varies around a certain average (n)

VR schedules generate high rates of behavior e.g., a VR 5 schedule means that the

individual will obtain the consequence after performing the target behavior an average of 5 times such that sometimes a single instance is sufficient, sometimes 3, sometimes 10, and other times perhaps 15 times

Examples of Variable-Ratio Schedules in Everyday Life

All forms of Gambling – winning depends on betting, and the more times you bet, the more likely you are to win

In Basketball, not every shot is successful, the more you shoot, the more likely you are to get a score

These examples are different from other schedules b/c the individual continues to work on the schedule after receiving the reward; in all the other schedules, once the reward is earned, the individuals typically go on to perform other tasks

FI vs. FR Schedule

Interval Schedules Require the individual to perform the target behavior after

a certain time has elapsed; it does not matter what the individual does before that time

Note: it is not merely the passage of time that produces the consequence – the individual has to do something after that interval has elapsed in order for the consequence to occur

As long as the individual is performing the target behavior some minimum number of times over the course of the experimental session, he/she will earn the maximum number of reinforcers possible, but more rapid behavior will not earn any more.

Therefore, unlike ratio schedules, after a certain point, the rate of reinforcement is independent of the rate of behavior

Interval Schedules Cont. These schedules encourage lower rates

of behavior than do ratio schedules b/c pausing and low rates of behavior do not appreciably delay the consequence

The most common ways to program the interval schedule requirement are: 1) To require the same interval for each

reinforcer (Fixed Interval) 2) To unsystematically vary the interval

across successive possibilities (Variable Interval)

Fixed-Interval Schedule (FI t) The first occurrence of the target behavior

after a specified interval of time has elapsed will produce the consequence; that interval is the same across successive reinforcers

Symbolized as FI t, where t specifies the time interval in whatever units are appropriate (seconds, minutes, and so on)

e.g., an FI 1 minute schedule programs the consequence following the first instance of the target behavior that occurs after 1 minute has elapsed since the last consequence

Typical Performance on Fixed-Interval Schedules

Performance on FI schedules is characterized by a post-reinforcement pause followed by either a progressively increasing rate of behavior or an abrupt transition to the terminal rate of behavior (break and run)

These schedules usually generate higher rates of behavior closer to the time of reinforcement

e.g., delaying studying until just before a test e.g., waiting for something to cook (You are

rewarded for checking your food when it is done; if you check it before it is done, you have wasted some effort)

Variable-Interval Schedules (VI t)

Requires the individual to perform the target behavior after a certain time has elapsed, but that time is not the same for each reinforcer

These schedules generate slow, steady rates of behavior with little or no post-reinforcement pauses

e.g., a VI 1 minute schedule, the individual can obtain the consequence for the first occurrence of the target behavior after a certain time has elapsed, but the required time will vary across successive reinforcers with an average interreinforcer time of 1 minute

Everyday Examples of VI Schedules If you are trying to call a friend on

the telephone and the line is busy (b/c that person does not have call waiting), you tend to wait for some time and then try again. Repeatedly pressing the redial button does not speed up your getting through, but if you wait too long, that person may hang up and take another call

VI vs VR Schedule

Schedules of Reinforcement

The Effects of Prior Experience

How an individual adjusts to a schedule of partial reinforcement is usually affected by his or her prior experiences

Individuals working under one schedule of reinforcement and then shifted to a second schedule may perform differently on the second schedule than individuals who did not have that prior experience with the first schedule

Repeated experience with change makes individuals more sensitive to change so that they more readily adapt to it

These carryover effects provide another perspective on how individuals adjust to changing circumstances

Behavioral Persistence The tendency to continue to perform a

behavior when that behavior no longer produces a reinforcing event

The conventional measure is called Resistance to Extinction how long it takes an individual to cease performing

a behavior when that behavior no longer produces the reinforcing event (the procedure of extinction)

the longer it takes the individual to cease performing a behavior when that behavior no longer produces the reinforcing event, the greater the resistance to extinction

3 Factors that Influence Behavioral Persistence

1) How much success the individual has experienced obtaining the reinforcing event by performing the target behavior: the greater the amount of success, the more persistent the behavior

2) How discriminable the change in procedure is to the individual - i.e., how quickly the individual can detect that reinforcement is no longer available

Discriminability is determined by how similar the 2 situations are: with respect to persistence, the more similar (less discriminable) extinction is to training, the greater the persistence (the more resistance to extinction)

3) Having had prior experience in which persistence has been rewarded: experience with partial reinforcement produces resistance to extinction

Behavioral Momentum The tendency for a behavior to persist

when the reinforcement contingency is changed

Behaviors that are more resistant to change have more momentum

From this perspective, amount of training increases persistence (resistance to extinction) b/c successful behaviors acquire more momentum and thus become more resistant to change when extinction is introduced

Discriminability and Persistence

Persistence is a function of how discriminable the conditions of reinforcement are from the conditions of extinction: extinction occurs faster (there is less persistence) when these two are more easily discriminated

Discriminability is related to Salience and is determined in part by the physical attributes of the stimulus and in part by what the individual brings to the situation (sensory capacities, knowledge)

Two events are discriminable when the individual can tell them apart

Persistence and Partial Reinforcement

Behaviors maintained by partial reinforcement persist longer in extinction than behaviors maintained by continuous reinforcement This is called the Partial Reinforcement Extinction

Effect (PREE) The PREE is due in part to the fact that it is

easier for individuals to discriminate the difference between continuous reinforcement and extinction than b/w partial reinforcement and extinction, and in part to the fact that experience with partial reinforcement teaches individuals to tolerate frustration

Persistence & Partial Reinforcement Cont. Experience with partial reinforcement

increases persistence: individuals who experience partial reinforcement, even if their most recent experience is with continuous reinforcement, show more resistance to extinction (persistence) than individuals who only experience continuous reinforcement of that behavior

This occurs because partial reinforcement increases frustration tolerance