Aversive Control: Avoidance and Punishment

Avoidance/Escape

• Escape: getting away from an aversive stimulus in progress

• Avoidance: preventing the delivery of an aversive stimulus

• Negative contingency between response and aversive stimulus

• Increase in operant responding

Brogden et al. (1938)

• Guinea pigs• CS = tone, US = shock,

UR = pain, CR = running

• Classical conditioning group– CS followed by US

• Avoidance group– CS -- CR --> no US– CS -- no CR --> US

Discriminative Avoidance

• Stimulus signals onset of aversive US

CS

US

R

CS

US

R

Avoidance Escape

Shuttle Box

• Standard experimental paradigm

Escape

• In presence of aversive stimulus

• Make response

• Aversive terminated

• Negative reinforcement

Avoidance “Paradox”

• Make response before aversive delivered

• Behaviour clearly increases, so reinforcer

• But what is taken away (or delivered)?• Mowrer & Lamoreaux (1942)

– “…not getting something can hardly, in and of itself, qualify as rewarding.”

Two-Process Theory• Two mechanisms: classical and

instrumental– 1. Classical conditioning process activated

by CS when avoidance not made; CR of fear produced

– 2. Negative reinforcement: successful avoidance removes fear caused by CS

• Classical and instrumental conditioning processes are independent

• Avoidance = escape from fear, not prevention of shock

Acquired Drive Experiment

• Phase 1: condition fear to CS through classical conditioning procedure

• Phase 2: let subject make operant response to terminate CS– No shock

• Drive to avoid learned through classical conditioning

Brown and Jacobs (1949)

• Rats in shuttle box• Experimental and control groups• Phase 1: light/tone CS --> shock• Phase 2: CS --> no shock; turn CS off by

crossing barrier• Measure: time to change sides• Supports two-process theory• Termination of fear CS drives operant

response

Rescorla & LoLordo (1965)

• Dog in shuttlebox– No signal– Response gives “safe time”

• Pair tone with shock– Tone increases rate of response

• CS+ can amplify avoidance

• CS- can reduce avoidance

Problems for Theory

• Fear a necessary component

• Fear reduction with experience

Kamin, Brimer & Black (1963)

• Rats• Lever press in operant

chamber for food• Auditory CS+ for shock;

avoidance in shuttle box until: 1, 3, 9, 27 avoidances in a row

• CS+ in operant chamber; check for suppression of lever press

Alternation of Behaviour (Yo-yo)

• Every successful avoidance puts CS on extinction

• With extinction, fear drops, so motivation to avoid decreases

• Resulting in more shocks, strengthening CR again and increasing avoidance response

• But… we don’t really see this

Persistence of Avoidance

• Sometimes a problem

• Phobias• Need to extinguish

avoidance• Flooding, response

prevention

Sidman Free-Operant

• Can avoidance be learned without warning CS?

• Shocks at random intervals• Response gives safe time• Extensive training, but rats learn avoidance

(errors, high variability across subjects)

Hernstein & Hineline (1966)

• Rapid and slow shock rate schedules

• Response switches from rapid to slow

• Shift back to rapid random so no time signal

• Response produces shock reduction

Reduction of Shock Frequency

• Molar account

• Response reduces in amount of shocks over long run

• Negative reinforcement– Overall shocks taken away, behaviour

increases

Safety Signals

• Molecular account

• Positive reinforcement

• Context cues associated with “safety”– Either SD or CS-

• Making response gives safety

• Giving explicit stimuli makes avoidance learning easier

SSDRs

• Species-specific defense reactions• Innate responses; evolved• SSDRs predominate in initial stages of

avoidance• Hierarchy

– If first SSDR works, keep it– If not, try next, etc.

• Aversive outcome (punishment) is the selector of appropriate avoidance response

SSDRs

• Fight, flight, freeze• Also thigmotaxis, defensive burying, light

avoidance, etc.• Environmental content influences selected

SSDR– E.g., freezing not useful if predator right in front of

you…

• Some responses easier to learn than others– E.g., rats: wheel run --> avoid shock (easy)– E.g., rats: rear --> avoid shock (hard)

Predatory Imminence

• Different innate defensive behaviours at different danger levels

Differences from SSDR

• 1. Behaviours in anticipation, not response

• 2. Predatory imminence, not environmental cues leads to response

• 3. Not selected via punishment

Punishment

• Positive punishment– Delivery of stimulus --> reduction in

behaviour

• Negative punishment– Removal of stimulus --> reduction in

behaviour

• Time out• Overcorrection

Introduction of Punisher

• Effective use of punishment

• Tolerance

• Start with high(er) intensity

• Can then reduce and behaviour will remain suppressed

Response-Contingent vs. Response-Independent

• Does your response cause the aversive outcome?• More behavioural suppression if aversive stimulus

produced by operant response

Phase 1: train on VI-60 sec

Phase 2: tone light

FR-3 response-independent punishment

punishment Yoked

tone

light

Sup

pres

sion

rat

ioTrials

Delay

• Interval between response and delivery of aversive

• Longer the delay, less suppression of behaviour

Punishment Schedule• Continuous or intermittent schedules• Azrin (1963)

– Different FR punishment schedules; responding maintained with VI reinforcement

no punishment FR 1000

FR 500

FR 100

FR 5

Time

Cum

ulat

ive

resp

onse

s

Positive Reinforcement Schedules and Punishment

• Without some positive reinforcement, behaviour generally stops quickly– As in previous study, responding maintained with

appetitive outcome on VI schedule

• Interval– Overall decrease– VI: suppressed but stable– FI: scalloping

• Ratio– Increases post-reinforcement pauses

Alternative Sources of Reinforcement

• Options– No alternatives but

punished behaviour– Alternative behaviours (e.g.,

differential reinforcement schedules; DRA, DRI, etc.)

• Availability of reinforceable alternatives increases suppression of punished response

no punishment Punishment, no alternative response available

Punishment, alternative response available

Time

Cum

ulat

ive

resp

onse

s

SD for Punishment

• Suppression limited to presence of SD

• E.g., garden owl• E.g., cardboard

“cops” and “kids”

Punishment as SD for Availability of Pos. Reinf.

• Sometimes punishment seeking behaviour

• Punisher becomes S+ for positive reinforcement

• E.g., masochism, children seeking attention

CER Theory of Punishment

• Estes (1944)• Conditioned suppression

– E.g., freeze prevents lever press

• CER incompatible with making response• Punishment suppresses behaviour through

same mechanism• In real world, no explicit CS

– Stimuli immediately before punished response serve this function

• Estes (1969): incompatible motivational state

Avoidance Theory of Punishment

• Tied to two-process theory• Engage in incompatible behaivour

– Prevents making punished behaviour

• Strengthening of competing avoidance response– Not weakening of punished response

• Same theoretical problems of avoidance

Negative Law of Effect

• Thorndike (1911)– Positive reinforcement and punishment are

symmetrical opposites

• Similar to Premack Principle– Low probability behaviours reduce high

probability behaviours– Forced to engage in low-valued behaviour

after doing high probability behaviour