Operant ConditioningOperant Conditioning

Thomas G. Bowers, Ph.D.Thomas G. Bowers, Ph.D.

Penn State HarrisburgPenn State Harrisburg

Operant ConditioningOperant Conditioning

Thought to operate upon the Thought to operate upon the environmentenvironment

So-called “voluntary behavior” So-called “voluntary behavior” Thorndike aimed to explain goal Thorndike aimed to explain goal

directed behaviordirected behavior– Developed Law of EffectDeveloped Law of Effect

Law of EffectLaw of Effect

Behavior is sensitive to its Behavior is sensitive to its consequencesconsequences

Positive reinforcement - Positive reinforcement - Pleasurable consequences stamp Pleasurable consequences stamp in the behaviorin the behavior

Punishment - Unpleasant events Punishment - Unpleasant events stamp out the behaviorstamp out the behavior

Behavior-consequence Behavior-consequence RelationshipRelationship

Positive reinforcement acts to Positive reinforcement acts to increase the probability of increase the probability of behaviorbehavior

Punishment acts to decrease the Punishment acts to decrease the probability of behaviorprobability of behavior

Behavior-consequence Behavior-consequence RelationshipRelationship

Negative reinforcement (or Negative reinforcement (or escape) acts to increase behavior escape) acts to increase behavior which eliminates or removes the which eliminates or removes the negative stimulusnegative stimulus

Omission removal of a positive Omission removal of a positive stimulus decreases behavior stimulus decreases behavior

Response-consequence Response-consequence Relationships Relationships

PositiveReinforcement

Punishment

Omission Escape orAvoidance

Response

Stimulus

Conditioning and Conditioning and ExtinctionExtinction

Responses are developed by a Responses are developed by a shaping process of successive shaping process of successive approximationsapproximations

Extinction refers to the cessation Extinction refers to the cessation of reinforcementof reinforcement

Behavioral UnitsBehavioral Units

Time

Response/Min

•Acquisition Extinction

Contingency LearningContingency Learning

P(Sr/No Res)

P(Sr/R)

1.0

.01.0

Operant Contingency Operant Contingency SpaceSpace

Reinforcement has a contingent Reinforcement has a contingent effect, increasing behavior, while effect, increasing behavior, while punishment or even non-punishment or even non-reinforcement will decrease behaviorreinforcement will decrease behavior

When reinforcement and responses When reinforcement and responses are independent, or noncontingent, are independent, or noncontingent, then learned helplessness resultsthen learned helplessness results

Operant Contingency Operant Contingency SpaceSpace

Learned helplessness resembles Learned helplessness resembles depressiondepression

Seligman developed the paradigmSeligman developed the paradigm Leads to a global failure to initiate Leads to a global failure to initiate

behaviorbehavior Associated with depletion of Associated with depletion of

monoamine neurotransmitters monoamine neurotransmitters

Operant ConditioningOperant Conditioning

What is learned?What is learned? R - S relationship?R - S relationship? How can something temporally How can something temporally

remote (i.e. following) cause an remote (i.e. following) cause an event?event?

Some theorists emphasize S - R Some theorists emphasize S - R relationshipsrelationships

Avoidance BehaviorAvoidance Behavior

Much of our day to day behavior Much of our day to day behavior may serve to avoid negative or may serve to avoid negative or aversive stimuli or consequencesaversive stimuli or consequences

Signaled avoidance trialsSignaled avoidance trials– Early training does not avoidance, but Early training does not avoidance, but

escapes the stimulusescapes the stimulus– Latency tends to decreaseLatency tends to decrease

Avoidance BehaviorAvoidance Behavior

Shock postponement procedureShock postponement procedure– Also called free-operant avoidanceAlso called free-operant avoidance– Sidman avoidanceSidman avoidance

Most animals manage to learn this Most animals manage to learn this well, with few actual shocks well, with few actual shocks experiencedexperienced

Theories of AvoidanceTheories of Avoidance

Does the animal “know”?Does the animal “know”?

Theories of AvoidanceTheories of Avoidance

Two factor theory - MowrerTwo factor theory - Mowrer– Initial learning by reinforcement of Initial learning by reinforcement of

escape behavior escape behavior – Classical conditioning also occurring, Classical conditioning also occurring,

and CS acquires fear eliciting and CS acquires fear eliciting propertiesproperties

– Response here is reinforced by fear Response here is reinforced by fear (CS) removal(CS) removal

– Avoidance behavior resultsAvoidance behavior results

Theories of AvoidanceTheories of Avoidance

Tests to inhibitory procedures Tests to inhibitory procedures appear to confirm predictionsappear to confirm predictions

Some problems, thoughSome problems, though– animals will respond reliably even if animals will respond reliably even if

only a reduction of shock frequency is only a reduction of shock frequency is the contingencythe contingency

– there is little evidence of conditioned there is little evidence of conditioned fear in well-trained animalsfear in well-trained animals

Theories of AvoidanceTheories of Avoidance

Some problems, thoughSome problems, though– avoidance of extinctionavoidance of extinction– avoidance can be extinguished, but avoidance can be extinguished, but

by response blockingby response blocking

Cognitive TheoriesCognitive Theories

Expectancy theoryExpectancy theory– Organism prefers no shock to shockOrganism prefers no shock to shock– Organism expects if it responds, no Organism expects if it responds, no

shock will occurshock will occur– Organism expects if it does not Organism expects if it does not

respond, shock will occurrespond, shock will occur– Expectancies are strengthened when Expectancies are strengthened when

confirmed, weakened when confirmed, weakened when disconfirmeddisconfirmed

Cognitive TheoriesCognitive Theories

Expectancy theoryExpectancy theory– Probability of avoidance increases as Probability of avoidance increases as

the degree of confirmation increasesthe degree of confirmation increases

Biological TheoriesBiological Theories

Bolles emphasized the adaptive Bolles emphasized the adaptive significance to persistent significance to persistent avoidance learningavoidance learning

Described a repertoire of defensive Described a repertoire of defensive reactionsreactions

Species-specific defensive Species-specific defensive reactions (SSDR)reactions (SSDR)

Biological TheoriesBiological Theories

Hierarchical organizationHierarchical organization Some patterns of responses are Some patterns of responses are

much easier to acquire than othersmuch easier to acquire than others

Conditioned Conditioned ReinforcementReinforcement

Neutral stimuli can also become a Neutral stimuli can also become a conditioned reinforcerconditioned reinforcer

Predictiveness, informativeness is Predictiveness, informativeness is important to becoming a important to becoming a secondary reinforcersecondary reinforcer

Animals will respond for the Animals will respond for the opportunity to gain informative opportunity to gain informative stimulus conditionsstimulus conditions

Applications of Secondary Applications of Secondary ReinforcementReinforcement

Token economiesToken economies– Common in our environmentCommon in our environment– Generalized secondary reinforcersGeneralized secondary reinforcers– FunctionsFunctions

Provide feedbackProvide feedback Provide information about what to do nextProvide information about what to do next Serve to bridge long gaps in Serve to bridge long gaps in

reinforcementreinforcement

– Economies now build in inflationEconomies now build in inflation