Post on 15-Jan-2016
Aversive Control: Avoidance and Punishment
Avoidance/Escape
• Escape: getting away from an aversive stimulus in progress
• Avoidance: preventing the delivery of an aversive stimulus
• Negative contingency between response and aversive stimulus
• Increase in operant responding
Brogden et al. (1938)
• Guinea pigs• CS = tone, US = shock,
UR = pain, CR = running
• Classical conditioning group– CS followed by US
• Avoidance group– CS -- CR --> no US– CS -- no CR --> US
Discriminative Avoidance
• Stimulus signals onset of aversive US
CS
US
R
CS
US
R
Avoidance Escape
Shuttle Box
• Standard experimental paradigm
Escape
• In presence of aversive stimulus
• Make response
• Aversive terminated
• Negative reinforcement
Avoidance “Paradox”
• Make response before aversive delivered
• Behaviour clearly increases, so reinforcer
• But what is taken away (or delivered)?• Mowrer & Lamoreaux (1942)
– “…not getting something can hardly, in and of itself, qualify as rewarding.”
Two-Process Theory• Two mechanisms: classical and
instrumental– 1. Classical conditioning process activated
by CS when avoidance not made; CR of fear produced
– 2. Negative reinforcement: successful avoidance removes fear caused by CS
• Classical and instrumental conditioning processes are independent
• Avoidance = escape from fear, not prevention of shock
Acquired Drive Experiment
• Phase 1: condition fear to CS through classical conditioning procedure
• Phase 2: let subject make operant response to terminate CS– No shock
• Drive to avoid learned through classical conditioning
Brown and Jacobs (1949)
• Rats in shuttle box• Experimental and control groups• Phase 1: light/tone CS --> shock• Phase 2: CS --> no shock; turn CS off by
crossing barrier• Measure: time to change sides• Supports two-process theory• Termination of fear CS drives operant
response
Rescorla & LoLordo (1965)
• Dog in shuttlebox– No signal– Response gives “safe time”
• Pair tone with shock– Tone increases rate of response
• CS+ can amplify avoidance
• CS- can reduce avoidance
Problems for Theory
• Fear a necessary component
• Fear reduction with experience
Kamin, Brimer & Black (1963)
• Rats• Lever press in operant
chamber for food• Auditory CS+ for shock;
avoidance in shuttle box until: 1, 3, 9, 27 avoidances in a row
• CS+ in operant chamber; check for suppression of lever press
Alternation of Behaviour (Yo-yo)
• Every successful avoidance puts CS on extinction
• With extinction, fear drops, so motivation to avoid decreases
• Resulting in more shocks, strengthening CR again and increasing avoidance response
• But… we don’t really see this
Persistence of Avoidance
• Sometimes a problem
• Phobias• Need to extinguish
avoidance• Flooding, response
prevention
Sidman Free-Operant
• Can avoidance be learned without warning CS?
• Shocks at random intervals• Response gives safe time• Extensive training, but rats learn avoidance
(errors, high variability across subjects)
Hernstein & Hineline (1966)
• Rapid and slow shock rate schedules
• Response switches from rapid to slow
• Shift back to rapid random so no time signal
• Response produces shock reduction
Reduction of Shock Frequency
• Molar account
• Response reduces in amount of shocks over long run
• Negative reinforcement– Overall shocks taken away, behaviour
increases
Safety Signals
• Molecular account
• Positive reinforcement
• Context cues associated with “safety”– Either SD or CS-
• Making response gives safety
• Giving explicit stimuli makes avoidance learning easier
SSDRs
• Species-specific defense reactions• Innate responses; evolved• SSDRs predominate in initial stages of
avoidance• Hierarchy
– If first SSDR works, keep it– If not, try next, etc.
• Aversive outcome (punishment) is the selector of appropriate avoidance response
SSDRs
• Fight, flight, freeze• Also thigmotaxis, defensive burying, light
avoidance, etc.• Environmental content influences selected
SSDR– E.g., freezing not useful if predator right in front of
you…
• Some responses easier to learn than others– E.g., rats: wheel run --> avoid shock (easy)– E.g., rats: rear --> avoid shock (hard)
Predatory Imminence
• Different innate defensive behaviours at different danger levels
Differences from SSDR
• 1. Behaviours in anticipation, not response
• 2. Predatory imminence, not environmental cues leads to response
• 3. Not selected via punishment
Punishment
• Positive punishment– Delivery of stimulus --> reduction in
behaviour
• Negative punishment– Removal of stimulus --> reduction in
behaviour
• Time out• Overcorrection
Introduction of Punisher
• Effective use of punishment
• Tolerance
• Start with high(er) intensity
• Can then reduce and behaviour will remain suppressed
Response-Contingent vs. Response-Independent
• Does your response cause the aversive outcome?• More behavioural suppression if aversive stimulus
produced by operant response
Phase 1: train on VI-60 sec
Phase 2: tone light
FR-3 response-independent punishment
punishment Yoked
tone
light
Sup
pres
sion
rat
ioTrials
Delay
• Interval between response and delivery of aversive
• Longer the delay, less suppression of behaviour
Punishment Schedule• Continuous or intermittent schedules• Azrin (1963)
– Different FR punishment schedules; responding maintained with VI reinforcement
no punishment FR 1000
FR 500
FR 100
FR 5
Time
Cum
ulat
ive
resp
onse
s
Positive Reinforcement Schedules and Punishment
• Without some positive reinforcement, behaviour generally stops quickly– As in previous study, responding maintained with
appetitive outcome on VI schedule
• Interval– Overall decrease– VI: suppressed but stable– FI: scalloping
• Ratio– Increases post-reinforcement pauses
Alternative Sources of Reinforcement
• Options– No alternatives but
punished behaviour– Alternative behaviours (e.g.,
differential reinforcement schedules; DRA, DRI, etc.)
• Availability of reinforceable alternatives increases suppression of punished response
no punishment Punishment, no alternative response available
Punishment, alternative response available
Time
Cum
ulat
ive
resp
onse
s
SD for Punishment
• Suppression limited to presence of SD
• E.g., garden owl• E.g., cardboard
“cops” and “kids”
Punishment as SD for Availability of Pos. Reinf.
• Sometimes punishment seeking behaviour
• Punisher becomes S+ for positive reinforcement
• E.g., masochism, children seeking attention
CER Theory of Punishment
• Estes (1944)• Conditioned suppression
– E.g., freeze prevents lever press
• CER incompatible with making response• Punishment suppresses behaviour through
same mechanism• In real world, no explicit CS
– Stimuli immediately before punished response serve this function
• Estes (1969): incompatible motivational state
Avoidance Theory of Punishment
• Tied to two-process theory• Engage in incompatible behaivour
– Prevents making punished behaviour
• Strengthening of competing avoidance response– Not weakening of punished response
• Same theoretical problems of avoidance
Negative Law of Effect
• Thorndike (1911)– Positive reinforcement and punishment are
symmetrical opposites
• Similar to Premack Principle– Low probability behaviours reduce high
probability behaviours– Forced to engage in low-valued behaviour
after doing high probability behaviour