Berridge DecisionUtility

7/29/2019 Berridge DecisionUtility

1/51

This is good news and bad news for utilitarians: thelimbic system reward pathways seem to correspondto a utility pump, but specialized brain circuitryprocesses experience in ways that are notnecessarily consistent with relentless maximization ofhedonic experience.

Daniel McFadden

Frisch LectureEconometric Society World Congress,London August 20, 2005


2/51

Cue-triggered wanting

& utility pumps

?

Daniel Kahneman utility types Predictedutility = cognitive value expectation Decisionutility = choice

Experiencedutility = actual hedonic outcome Liking-Wanting neuroscience

Translated into decision moment

Ordinary rational wanting Max DU = PU = EU

Wanting as one expects to like best(and does like best)


3/51

Types of Miswanting

?

Conventional miswanting Choosing what one wrongly

expects to like

(DU = PU) > EU Affective forecasting Distorted predicted utility

Truly irrational miswanting

Choosing what onedoes NOT expect to like

DU > (PU = EU)


4/51

Todays thesis:Activation of brain mesolimbic dopamine

systems may elevate decision utility alone

Moments of vulnerability as

cue-triggered wanting


5/51

Experienced utility generation:Brain hedonic hotspot network

VTA

Amygdala

Hippocampus

Prefrontal

Cortex

Ventral

Pallidum

Mediodorsal

Thalamus

Brainstem

outputs

After Everitt & Robbins 2005; Nakamura & Kalivas 1999

BL

Ce

core shell

Nucleus

Accumbens

How to study causation?


6/51

Liking & Disliking expressions to tasteReflect palatability of physiological hunger, learning, & brain hedonics

Neural hierarchy: forebrain hedonics control brainstem reactions


7/51

Brain hedonic archipelago: Pleasure sitesOpioid hotspot microinjections cause increases inliking reactions to sweetness

VTA

Amygdala

Hippocampus

Prefrontal

Cortex

Ventral

Pallidum

Mediodorsal

Thalamus

Brainstem

outputs


BL

Ce

core shell

Nucleus

Accumbens


8/51

Experienced utility & dopamine?

Rewards activate(food, cocaine, sex, money, videogames...)

Neural suppression blocks rewards

Liking (experienced utility)? Wanting (decision utility)?Learning (predicted utility)?


9/51

Experienced Utility

Not dopamine(Not a pleasure neurotransmitter)


10/51

Extra dopamine signal Less dopamine transporterfor reuptake (10%)

More dopamine in the synapse (170%)

DA

DA

DA

DA

DA

DA

DA

!"#$%&'()*+,-.#/"0'123

Dopamine as sufficient cause for:Hyper-dopamine mutant mouse

Reward wanting vs learning & liking

Collaboration withDr. Xiaoxi Zhuang

University of Chicago


11/51

Mutants want sweet rewards more

Do they like sweets more?

Taste reactivity: sweet liking in DAT mutant mouse


12/51

Extra-dopamine mutants like sucrose less

3 concentrationsMutants show fewer

liking reactionstosweet taste

Pecia, Cagniard, Berridge, Aldridge & Zhuang, 2003

Positiveaffect

(reward)

want like

Control Mutant


13/51

Dopamine as wanting & not liking in people?

Dopamine binding correlates best

to subjective ratings of drugwanting (not drug liking) Evans et al. 2006 L-dopa & PET [11C]-raclopride

displacement in DDS patients

Positiveaffect

(reward)

want like


14/51

Humans want without like?Ex: False pleasure electrodes

Rat LH stimulation Food wanting without liking

Famous 1960s human studies

Neurologists implanted patients Robert Heath, M.D.

Patient B-19 - 20-yr man MFB electrode Stimulated electrode 1500 times in a session,protested whenever stimulator taken away

(p. 6, Heath, 1972)

feelings of pleasure, alertness, and warmth

(goodwill); he had feelings of sexual arousal anddescribed a compulsion to masturbate.But really pleasure?

No exclamations of pleasure And electrode did not substitute for sex

Electrode only made him want sex more

want

Positiveaffect

(reward)

like


15/51

Another false pleasure electrode?

Woman (40 yrs-old) in 1980s study Thalamic electrode to control pain

Portenoy et al., 1986

"At its most frequent, the patient self-stimulated

throughout the day, neglecting personal hygiene and

family commitments

Though sexual arousal was prominent, no orgasm

occurred"

She described erotic sensations often intermixed with

an undercurrent of anxiety. She also noted extremethirst, drinking copiously during the session, and

alternating generalized hot and cold sensations"

Insufficient electrode experienced utilityto explain

compulsive decision utility?want

Positiveaffect

(reward)

like


16/51

Implications of liking vs wanting?Incentive-Sensitization theory of Addiction

Terry Robinson

Neural Sensitization of mesolimbicdopamine systems:Hyper-excitableCaused by amphetamine, cocaine, heroin,

alcohol, nicotine

Lasts years (Evans et al., 2006)

SubjectivePleasure

Time

Relative

Effect

IncentiveValueInitial

Use

Addiction

want

Positiveaffect

(reward)

like


17/51

Pulling apartPredicted vs Decision Utility

Dopamine as cue-triggered wanting?Or dopamine as cue-reward learning & predicting?


18/51

Agreed: dopamine neurons code prediction errorsQuestion: Cause or Consequence?

Schultz, Dayan, &Montague, 1997

Science, v. 275p. 1594


19/51

Computational dopamine learning models

Rescorla-Wagner model: V= (-V)

Temporal difference model:

Learns by reward prediction errors:

L+++==++

=

+! 22

10

)( ttti

iti

t rrrrsV """

)()()( 1 tttt sVsVrs !+= +"#


20/51

Conditioned Incentive CuesBolles, Bindra, Toates: Pavlovian reward cues

gain incentive value of their rewards

Incentive Salience Wanting

Transforms the neural representation of a

conditioned stimulus Changes perception of CS+

Grabs attention & becomes attractive

Elicits approach & becomes a goal

Integrates with physiological state... Elicits specific wanting for reward

Cues trigger wanting


21/51

Liking & Wanting versus Learningfrom neurons point of view

J. Wayne Aldridges electrophysiology laboratory

Ventral Pallidum

Mesolimbic final common path

VTA

Amygdala

Hippocampus

Prefrontal

Cortex

Ventral

Pallidum

Mediodorsal

Thalamus

Brainstem

outputs


BL

Ce

core shell

Nucleus

Accumbens


22/51

Ventral pallidal neurons codeCS cues and UCS rewards

UCS

(sucrose)

10 secTime

CS+1

(tone)

Tindell et al,


23/51

Serial cues dissociate learnedPrediction vs Wanting

CS+1 predicts all that follows (V) CS+2 elicits greatest cue-triggered wanting (incentive salience)

UCS has greatest liking reinforcement value ((t))(stamping-in or teaching signal value)

CS+1

(tone)

CS+2(click) UCS

(sugar)

10 sec 1 sec

Time

Tindell, Berridge, Zhang,

Pecia, Aldridge, 2005


24/51

Profile analysis:ventral pallidumfiring



CS+1

(tone)

CS+2

(click) UCS

(sugar)

10 sec 1 sec

TDValueUCS

TDError

CS+1

Inc

entive

C

S+2

Direction =

tan -1 ([ (CS1-UCS)/2] (2CS2 CS1 UCS)/2)

Magnitude r =

[(CS1-CS2)2 + (CS2-UCS)2 + (UCS-CS1)2]/2

Profile logic by Jun Zhang, AmyTindell, & J. Wayne Aldridge

Aldridge Lab

Predi

cted

Utility

DecisionUtility

ExperiencedUtility


25/51




CS+1

(tone)

CS+2

(click) UCS

(sugar)

10 sec 1 sec

TDError

TDValueUCS

TDError

CS+1

Inc

entive

C

S+2


Aldridge Lab

Direction =


Magnitude r =

[(CS1-CS2)2 + (CS2-UCS)2 + (UCS-CS1)2]/2

Predictionresponse

CS+1

Predi

cted

Utility

DecisionUtility

ExperiencedUtility


26/51




CS+1

(tone)

CS+2

(click) UCS

(sugar)

10 sec 1 sec

TDError

TDValueUCS

TDError

CS+1

Inc

entive

C

S+2

Direction =


Magnitude r =

[(CS1-CS2)2 + (CS2-UCS)2 + (UCS-CS1)2]/2


Aldridge Lab

Incentiveresponse

CS+2

Predi

cted

Utility

DecisionUtility

ExperiencedUtility


27/51




CS+1

(tone)

CS+2

(click) UCS

(sugar)

10 sec 1 sec

TDError

TDError

CS+1

Inc

entive

C

S+2

TDValueUCS

Direction =


Magnitude r =

[(CS1-CS2)2 + (CS2-UCS)2 + (UCS-CS1)2]/2


Aldridge Lab

Predictionerror

response

UCS

Pred

icted

Utility

DecisionUtility

ExperiencedUtility


28/51

PriorSensitization

Orbitomedial Frontal Cortex

Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala

Ventral Tegmental AreaSNc/SNr


Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala

Ventral Tegmental AreaSNc/SNr Tindell, Berridge, Zhang,

Pecia Aldridge, 2005


VP

1 0 -1

1

2

3

Medial

Anterior

Bregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized

Electrode Location Reconstruction

ratsPredic

ted

Utility

DecisionUtility

Experienced

Utility


29/51

PriorSensitization


Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala




VP

1 0 -1

1

2

3

Medial

Anterior

Bregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized


rats

ratsPre

dicted

Utility

DecisionUtility

Experienced

Utility


30/51

PriorSensitization


Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala




VP

1 0 -1

1

2

3

Medial

Anterior

Bregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized


Acute

Amphetamine

injection

rats

ratsPre

dicted

Utility

DecisionUtility

DecisionUtility

ExperiencedUtility

Experienced

Utility

Predic

ted

Utility


31/51

PriorSensitization


Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala




VP

1 0 -1

1

2

3

Medial

Anterior

Bregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized


Acute

Amphetamine

injection

rats

rats

injection *

Predic

ted

Utility

DecisionUtility

DecisionUtility

ExperiencedUtility

Experienced

Utility

Predic

ted

Utility


32/51

VP

1 0 -1

1

2

3

Medial

AnteriorBregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala




DA Amplifies CS+ Incentive Code

CS US

incentivesalience

pursuit

CS+1 CS+2 SugarUCS

Time

Incentive cue

Pred

icted

Utility

Decision

Utility


33/51

VP

1 0 -1

1

2

3

Medial

AnteriorBregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala




DA magnifiesmotivational

transform oflearned CS+2signal


CS US

incentivesalience

pursuit

CS+1 CS+2 SugarUCS

Time

Incentive cue

Pred

icted

Utility

Decision

Utility


34/51

VP

1 0 -1

1

2

3

Medial

AnteriorBregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala





CS+1 CS+2 SugarUCS

Time

Incentive cue

CS US

incentivesalience

pursuit



Pred

icted

Utility

Decision

Utility


35/51

VP

1 0 -1

1

2

3

Medial

AnteriorBregma

AcbShSI

Posterior

Lateral

IPAC

Sensitized

Nonsensitized



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala



Hippocampus

Thalamus

Hypothalamus

Nucleus Accumbens

Ventral Pallidum

Amygdala





CS US

incentivesalience

pursuit

CS+1 CS+2 SugarUCS

Time

Incentive cue



Pred

icted

Utility

Decision

Utility


36/51

Wanting is triggered by cues: Images & imagerySimultaneous synergy: Reward cue & brain dopamine activation

e

Le

Amphetamine

microinjection

NAcc

Dickinson style Pavlovian-Instrumental Transfer

Wyvell & Berridge (2000; 2001)

0.0 2.0 10.0 20.0

1

2

3

4 CS+

CS-

BaselineCSBaselineCS-

*

**

**

Amphetamine Microinjection Dos

DA = cue incentive salience

Sensitized

(even without drug)**

CS+

CS-

Baseline

No increases in sucrose liking


37/51

Incentive-Sensitization modelTerry Robinson

SubjectivePleasure

Time

Relative

Effect


Use

Addiction

CS US

incentive

salience

pursuit

Tindell, Berridge, Zhang, Pecia, Aldridge, 2005


38/51

Inside utility experience Why dont we feel individual utility pumps?

Basic utility components may in a sense be

unconscious Not directly experienced

Analogous to unconscious vision in blindsight

Needing additional brain translation for awareness

E.g. Conscious readout by frontal cortex areas


39/51

Prime, Pour and ConsumeWinkielman Berridge & Wilbarger 2005 unconscious emotion experiment

Subliminal Emotion (16 ms)Happy, Neutral, Angry

J. S. Morris, A. hman, and R. J. Dolan PNAS 1999

WDS Killgore & DA Yurgelun-Todda Neuroimage 2004

Subliminal maskedfaces activate

limbic brainWhalen et al J Neurosc1998


40/51


Visible Neutral(400 msec)

Gender Identification Task



41/51


Visible Neutral(400 msec)

Gender Identification Task

Rate Mood + Arousal

Pour + Consume



42/51

16 msec(you dont see this)

Winkielman, Berridge, & Wilbarger, 2005, PSPB


43/51

400 msec(you see for half a second)

Gender?



44/51

Then pour & drink

+ -

Please rate yourmood How do you feel right now?

Hedonic Pleasant-Unpleasant 1-10 scale

20 item PNAS scale (irritated, content.)


45/51

Subjective experience


0

1

2

3

4

5

0

1

2

3

4

5Mood Arousal

Positive

Hedonic

Neutral

High

Low

+ -


46/51

Consumption behavior(thirsty participants)


0

20

40

60

80

100

Pour Drink

Volume

ml poured

or drunk


47/51


0

$0.10

$0.30

$0.50

angry happy

Willing to Pay?

Drink Evaluations: Sip & rate results

$


48/51

Unconscious utilities appear possible Both causal event and emotional reaction

are unperceived at moment they occur

Affective target triggers later valenced manifestation ofdecision utility: drink more & value more

We may lack direct access to basic utility pumps



49/51

Images & mesolimbic wanting in everyday life?

Do people ever want independently from

like (or even predict) in impulsive urges toobtain food, sex, gambling, money,shopping?

Brain mesolimbic systems may reveal

fracture lines between reward utility pumps Decision utility

Predicted utility

Experienced utility


50/51

Colleague Thanks

Long term collaboratorsTerry Robinson

J. Wayne Aldridge

Lab data today

J. Wayne Aldridge, Amy Tindell,

Xiaoxi Zhuang, Cindy Wyvell,

Susana Pecia, Jun ZhangRelated work by

Sheila Reynolds, Kyle Smith,

Steve Mahler, Alexis Faure

SubjectivePleasure

Time

Relative

Effect


Use

Addiction


51/51

Berridge DecisionUtility

Documents

Transcript of Berridge DecisionUtility