Critical Review

The Strength of Aversive and Appetitive

Associations and Maladaptive Behaviors

Yossef Itzhak*

Daniel Perez-Lanza

Shervin Liddie

Department of Psychiatry and Behavioral Sciences, University of MiamiMiller School of Medicine, Miami, FL, USA

Abstract

Certain maladaptive behaviors are thought to be acquired

through classical Pavlovian conditioning. Exaggerated fear

response, which can develop through Pavlovian conditioning,

is associated with acquired anxiety disorders such as post-

traumatic stress disorders (PTSDs). Inflated reward-seeking

behavior, which develops through Pavlovian conditioning,

underlies some types of addictive behavior (e.g., addiction to

drugs, food, and gambling). These maladaptive behaviors are

dependent on associative learning and the development of

long-term memory (LTM). In animal models, an aversive rein-

forcer (fear conditioning) encodes an aversive contextual and

cued LTM. On the other hand, an appetitive reinforcer results

in conditioned place preference (CPP) that encodes an appeti-

tive contextual LTM. The literature on weak and strong associa-

tive learning pertaining to the development of aversive and

appetitive LTM is relatively scarce; thus, this review is particu-

larly focused on the strength of associative learning. The

strength of associative learning is dependent on the valence of

the reinforcer and the salience of the conditioned stimulus that

ultimately sways the strength of the memory trace. Our studies

suggest that labile (weak) aversive and appetitive LTM may

share similar signaling pathways, whereas stable (strong) aver-

sive and appetitive LTM is mediated through different path-

ways. In addition, we provide some evidence suggesting that

extinction of aversive fear memory and appetitive drug mem-

ory is likely to be mediated through different signaling mole-

cules. We put forward the importance of studies aimed to

investigate the molecular mechanisms underlying the develop-

ment of weak and strong memories (aversive and appetitive),

which would ultimately help in the development of targeted

pharmacotherapies for the management of maladaptive behav-

iors that arise from classical Pavlovian conditioning. VC 2014

IUBMB Life, 66(8):559–571, 2014

Keywords: Pavlovian conditioning; appetitive memory; aversive

memory; cocaine; fear conditioning; memory strength; nitric oxide.

IntroductionClassical Pavlovian conditioning is viewed as a “behavioralreflex” that is dependent on a previous experience, which enc-odes memory. For instance, when an animal encountersthreatening and aversive stimuli, the immediate behavioral

response is freezing; when the subject is exposed to appetitiveand rewarding stimuli, the response is approach and explora-tion. These responses are important because they shape futurebehavior and survival skills of the organism. When the stimuliare presented within neutral contexts and environmental cues,the subject learns to associate a specific stimulus with a spe-cific cue. Subsequently, presentation of the cue (or context)alone elicits a similar response as the primary reinforcer eli-cited (freezing or approach behavior). The reinforcer is anunconditioned stimulus (US) and the cue or context that wasassociated with the US becomes a conditioned stimulus (CS),which now elicits the conditioned response (CR). The CR isdependent on the formation of long-term memory (LTM) of theassociation between the CS and the US. Depending on thememory strength and circumstances, reexposure to the CSevokes the memory of the US and elicits the CR.

Operant conditioning is also based on reinforcement (posi-tive or negative) learning. However, it is different from

VC 2014 International Union of Biochemistry and Molecular BiologyVolume 66, Number 8, August 2014, Pages 559–571

*Address correspondence to: Yossef Itzhak, Department of Psychiatry andBehavioral Sciences, University of Miami Miller School of Medicine, Gaut-ier Bldg Room 503, 1011 NW 15th Street, Miami, FL 33136, USA. Tel:1305-243-4635. Fax: 1305-243-2989. E-mail: yitzhak@med.miami.eduReceived 18 July 2014; Accepted 23 August 2014DOI 10.1002/iub.1310Published online 5 September 2014 in Wiley Online Library(wileyonlinelibrary.com)

IUBMB Life 559

classical Pavlovian conditioning because an operant behavioris modified according to the experience. For instance, expo-sure to appetitive US (e.g., food or drug) will result in activelever pressing for “more reinforcer.” On the other hand, expo-sure to aversive stimulus (e.g., footshock) will result in avoid-ance or escape behavior (not just freezing).

Classical conditioning is relevant to the formation of LTM(1). Maladaptive behaviors such as anxiety disorders (2) anddrug addiction (3,4) are associated with learning and memoryprocesses, involving classical conditioning. This review isfocused on differences in the strength of classical Pavlovianconditioning as a result of changes in the valence of aversive(footshock) and appetitive (drug) stimuli.

Fear-Associated Memory

Acquisition of Fear MemoryFear is an inherent emotion which helps animals and humansavoid or escape dangerous conditions. However, for those whoexperience painful trauma, fear can become chronic and debil-

itating. Anxiety disorders are a major public health concernassociated with high prevalence, economic burden, and socialconsequences. In the United States, the lifetime prevalence ofanxiety disorders is 28.8%, making them the most prevalentmental health disorder (5). Because many anxiety disordersare associated with the formation of aversive LTM, the fearconditioning paradigm has been used in rodents to investigatefear-associated LTM. When a painful stimulus (footshock, US)is paired with a neutral context and cue (e.g., sound), the lat-ter becomes a CS which elicits freezing response. Several stud-ies suggest that acquisition of fear memory is relevant to thedevelopment of post-traumatic stress disorders (PTSDs; refs.6–8). Elucidation of mechanisms underlying the formation andextinction of fear-associated LTM would facilitate therapy foranxiety-related disorders (2,9,10).

In a laboratory setting, depending on species and rodents’strain, single or multiple CS–US pairings were executed toencode LTM. However, only a few studies investigated the out-come of different schedules of fear conditioning training. Forinstance, strong training (10 paired CS–US) resulted in tran-sient resistance to cued memory reconsolidation, whereasweak training (one CS–US pairing) resulted in facilitative cuedmemory reconsolidation (11). Rats with hippocampal lesionsacquired contextual LTM when multiple shocks, but not a sin-gle shock, were administered (12). The exact mechanismsunderlying memory formation as a result of weak and strongtraining are not clear.

Single and Multiple CS–US Pairings: Nitric Oxide-Dependent and -Independent Fear MemoryUsing the fear conditioning paradigm to investigate contextualand cued LTM, we have shown that a single CS–US pairingwas sufficient to encode optimal contextual and cued LTM inwild-type (WT) mice; however, it was insufficient to encodecontextual LTM in neuronal nitric oxide synthase (nNOS)knockout (KO) mice (13). However, four CS–US pairingsresulted in successful contextual LTM of nNOS KO mice(Fig. 1; ref. 13). These findings suggest that mild training (oneCS–US paring) engages NO signaling in the formation of LTM,whereas stronger training (four CS–US pairings) encodes LTMin the absence of NO signaling. Single CS–US pairing and mul-tiple CS–US pairings resulted in similar magnitude of contex-tual freezing in WT mice probably as a result of a ceiling effectfollowing the single footshock.

Our research focused on the role of nNOS in the formationof contextual and cued LTM of fear because NO produced bynNOS has a role in synaptic plasticity. In mouse hippocampalslices, synaptic plasticity such as late phase LTP is dependenton NO/cGMP/PKG/CREB signaling (14,15). Evidence frombehavioral studies in invertebrates suggests that NO has amajor role in consolidation of LTM (16,17).

Additional evidence from our research supports the role ofnNOS in the formation of labile contextual LTM following amild fear conditioning training. First, a single CS–US pairingthat was insufficient for contextual LTM of nNOS KO mice was

Acquisition of fear conditioning: The magnitude of

contextual and cued freezing response of WT and

nNOS KO mice was tested 24-h post-training, which

indicates the expression of long-term memory (LTM).

No difference between WT and nNOS KO mice was

observed pretraining. WT mice that received saline

before training by one CS–US pairing exhibited high

freezing on reexposure to the context or the cued

tone. nNOS KO mice that received saline before

training by one CS–US pairing exhibited significantly

low freezing on reexposure to the context when

compared with WT mice (*P<0.05), but showed the

same freezing response as WT mice on reexposure

to the cued tone. The deficit in contextual freezing of

nNOS KO mice was rescued by pretraining adminis-

tration of the NO donor molsidomine (MOL) and the

HDAC inhibitor sodium butyrate (NaB) or by four

CS–US pairings. The treatments that enhanced con-

textual freezing response of nNOS KO mice had no

significant influence on WT mice that had developed

maximal freezing response following one CS–US

pairing (adapted from refs. 13, 18, and 19).

FIG 1

IUBMB LIFE

560 Memory Strength and Maladaptive Behaviors

sufficient to induce auditory-cued LTM in nNOS KO mice (Fig.1; ref. 13). The hippocampus has a major role in contextualLTM, and the amygdala has a major role in cued fear memory;however, the distinction between NO dependency in contextualand cued memory is unclear. Nevertheless, extinction of con-textual fear memory is achieved faster than extinction of cuedmemory (18), suggesting that auditory-cued fear memory,which is apparently NO-independent, may be “stronger” thancontextual memory. Second, administration of the nNOS inhib-itor S-methyl-L-thiocitrulline (SMTC) prior to a single CS–USpairing suppressed contextual but not auditory cued fearmemory of WT mice (19). Third, administration of the NOdonor molsidomine (MOL) prior to a single CS–US pairingdose-dependently improved contextual fear conditioning ofnNOS KO (Fig. 1; ref. 19). Fourth, the histone deacetylase(HDAC) inhibitor sodium butyrate (NaB) that increases histoneacetylation in hippocampus and amygdala rescued contextualLTM of nNOS KO mice following a single CS–US pairing (Fig. 1;ref. 18). The findings that four CS–US pairings and NaB res-cued contextual LTM of nNOS KO mice suggest an NO-independent mechanism in the formation of contextual LTM.

Additional differences between contextual memoryencoded by single CS–US pairing and multiple CS–US pairingsof WT mice were observed in reconsolidation studies. Reconso-lidation is a process by which retrieval of LTM becomes labileand susceptible to changes. It can either be strengthened orweakened depending on the interference during the reconsoli-dation process. We found that administration of the nNOSinhibitor SMTC (which suppressed the formation of contextualLTM) immediately following retrieval of contextual LTM thatwas encoded by a single CS–US presentation was suppressedon a subsequent test when compared with saline-treated mice.However, a similar treatment given to mice that were trainedby four CS–US pairings did not influence subsequent freezingresponse (unpublished observations). These findings furthersupport the hypothesis that LTM encoded by single, but notmultiple CS–US presentations, is NO-dependent. Hence, boththe acquisition/consolidation and reconsolidation of contextualfear memory following a single CS–US pairing involve NO sig-naling pathway. However, LTM following multiple CS–US par-ings involves other or additional signaling pathways whichrender that memory insensitive to manipulation of the NOsignaling pathway.

Role of Corticosterone in Fear MemoryIn rodents, the stress hormone corticosterone has been impli-cated in the formation of contextual (20,21) and cued fearmemory (22–24). We investigated how nNOS gene deletionaffects the physiological response to fear conditioning. Wefound a relationship between plasma corticosterone levels andsuccessful contextual fear learning for the following reasons.First, elevated corticosterone was observed 15 min after a sin-gle CS–US session in WT mice, which exhibited contextualLTM, but not in KO mice, which did acquire contextual fearlearning (13). Second, 15 min after four CS–US pairings, KO

mice showed elevated corticosterone levels concomitant withsignificant improvements in contextual LTM (13). Third, sex-dependent differences in freezing response of nNOS KO micewere correlated with differences in levels of plasma corticos-terone (13). Therefore, in our paradigm, (a) the absence of thenNOS gene results in deficits in stress-induced corticosteronerelease, (b) multiple footshocks are required to induce corti-costerone release in nNOS KO mice and subsequent formationof contextual LTM, and (c) increased corticosterone is requiredfor contextual LTM more than cued LTM.

pCREB and Histone AcetylationGiven that a single CS–US pairing requires NO signaling andhistone acetylation (Fig. 1), we investigated (a) the levels ofphosphorylated cAMP response element-binding protein(pCREB) and histone H3 and H4 acetylated marks in hippo-campus and amygdala of WT and nNOS KO mice and (b) theeffects of MOL and NaB on expression of pCREB and the his-tone marks. By using Western blot analysis, we found signifi-cant high levels of pCREB in hippocampus and amygdala ofnNOS KO mice when compared with WT (Table 1; ref. 25).Because NO/cGMP/PKG pathway is upstream of pCREB, theincrease in pCREB expression in nNOS KO mice was unex-pected. Yet, as expected, administration of the nNOS inhibitorSMTC to WT mice resulted in a reduction in amygdalar pCREBand inhibited contextual LTM (25). The NO donor MOL alsohad an opposite effect on WT and KO mice. Systemic adminis-tration of MOL to WT and nNOS KO mice resulted in increasedand decreased expression of pCREB, respectively (25).Although NO-induced increase in pCREB in WT mice wasexpected, the effect of the NO donor on nNOS KO mice wasunexpected. However, because MOL rescued contextual mem-ory of nNOS KO mice, we hypothesize that the reduction inpCREB might have “restored” its level to optimal concentra-tions which are required for the formation of LTM (25). Thehigh basal levels of pCREB observed in na€ıve nNOS KO whencompared with WT mice is unclear; however, it may be due tocompensatory mechanisms. Initial low levels of pCREB in theabsence of NO signaling may be subsequently compensatedand even enhanced through various signaling moleculesbypassing the NO signaling pathway.

Recent studies suggest that regulation of chromatin struc-ture is one of the essential molecular mechanisms that con-tribute to the formation of synaptic plasticity and LTM (26).One of the regulatory processes of chromatin structure is theacetylation and deacetylation of histone proteins. Histone ace-tyltransferases (HATs) acetylate conserved lysine amino acidson histone proteins by transferring an acetyl group from ace-tyl CoA to form e-N-acetyl lysine. Acetylation brings in a nega-tive charge, which neutralizes the positive charge on the his-tones and decreases the interaction of the N-termini ofhistones with the negatively charged phosphate groups ofDNA. As a result, the condensed chromatin is transformedinto a more relaxed structure, which is associated withgreater levels of gene transcription (27). HDACs are classes of

Itzhak et al. 561

enzymes that remove acetyl groups, increasing the positivecharge of histone tails and the binding between histones andDNA. The increased DNA binding condenses DNA structureand prevents transcription.

Histone acetylation is one of the epigenetic processes thathave been associated with learning and memory; particularly,histone acetylation is critical for the consolidation ofhippocampus-dependent LTM (28–30). We investigated the levelsof H3K14 and H4K8 acetylation in hippocampus and amygdalaof WT and nNOS KO mice and the effects of the NO donor MOLand the HDAC inhibitor NaB on these marks. We investigatedH3K14 and H4K8 acetylation marks because several studies sup-port their involvement in learning and memory (31–33).

We found that acetylation levels of H3K14 and H4K8 inhippocampus and amygdala of nNOS KO mice were lower thanin WT mice (Table 1). Interestingly, the NO donor and theHDAC inhibitor had the same effects on H3 and H4 histoneacetylation. They both increased histone acetylation in hippo-campus and amygdala of nNOS KO mice (Table 1; ref. 18).However, in WT mice, MOL and NaB increased H3K14 andH4K8 acetylation in hippocampus but not amygdala (Table 1;ref. 34). These findings suggest that (a) in the absence of thenNOS gene, levels of basal H3K14 and H4K8 acetylated marksare low, (b) rescuing contextual fear memory of nNOS KO miceby MOL may be due to NO-dependent increase in histone acet-ylation, similar to the effect of the HDAC inhibitor NaB, and (c)

Role of CREB phosphorylation and histone acetylation in acquisition of contextual fear memory (results are adapted from

refs. 25 and 32)

Treatment Genotype Amygdala Hippocampus Contextual LTM: acquisition

Vehicle CREB phosphorylation: pCREB

WT Baseline Baseline Optimal

KO " " Absent

SMTC (nNOSi) WT # 5 Reduced

KO N/A N/A N/A

MOL (NO donor) WT " 5 Optimal

KO # 5 Optimal

Vehicle H3 Histone acetylation: H3K14

WT Baseline Baseline Optimal

KO # # Absent

MOL (NO donor) WT 5 5 Optimal

KO " " Optimal

NaB (HDACi) WT 5 " Optimal

KO " " Optimal

Vehicle H4 Histone acetylation: H4K8

WT Baseline Baseline Optimal

KO # # Absent

MOL (NO donor) WT 5 5 Optimal

KO " " Optimal

NaB (HDACi) WT 5 " Optimal

KO " " Optimal

Abbreviations: CREB, cAMP response element-binding protein; WT, wild type; KO, knockout; SMTC, S-methyl-L-thiocitrulline; NO, nitric oxide;

nNOSi, neuronal nitric oxide synthase inhibitor; MOL, molsidomine; HADCi, histone deacetylase inhibitor; NaB, sodium butyrate; LTM, long-term

memory; N/A, not applicable; “"” and “#,” increase and decrease of the specific marks when compared with vehicle WT and vehicle KO; 5, no

effect.

TABLE 1

IUBMB LIFE

562 Memory Strength and Maladaptive Behaviors

the role of NaB-induced histone acetylation in hippocampus,but not amygdala, of WT mice is likely related to strengtheningof contextual LTM (34).

Extinction of Fear MemoryExtinction learning by “exposure therapy” is thought to beessential for the management of maladaptive behaviors thatwere acquired through Pavlovian conditioning (35–40). Inextinction experiments, the subject is reexposed to the contextand/or cue (CS) in the absence of the reinforcer (US) until theCS loses its excitatory properties. Typically, although memoryreactivation/reconsolidation requires short/single CS reexpo-sure, extinction requires long or multiple CS reexposures(41,42). The results of fear conditioning studies suggest thatthe extinction process does not eliminate or cause“unlearning” of the initial CR; rather, the organism learns thatthe CS does not elicit the previous stimulus (35,36). Thus,extinction requires associative learning, consolidation, and for-mation of a new memory (43,44).

When we compared the extinction rate of contextual freez-ing acquired by one CS–US pairing and four CS–US pairings,we found that in WT mice, extinction of freezing acquired byone CS–US pairing was faster than that acquired by four CS–US pairings, suggesting that the strength of training influencessubsequent extinction learning. We then compared betweenthe extinction of contextual and cued fear memory which wasacquired by one CS–US pairing. The results showed that cuedfreezing was more resistance to extinction than contextualfreezing (18). The finding that contextual, but not cued, fearmemory is dependent on NO signaling (Fig. 1) further supportsthe premise that “weak” memory and not “strong” memory isNO-dependent.

As depicted in Fig. 1, we found that the HDAC inhibitorNaB had no effect on acquisition of contextual and cued fearmemory of WT mice. Thus, we investigated whether adminis-tration of the HDAC inhibitor pretraining may have had along-term effect on extinction of fear memory. In this experi-ment, extinction training began 10 days following administra-tion of NaB prior to the acquisition of fear conditioning (18).We found that NaB had no effect on the rapid extinction ofcontextual freezing; however, it facilitated the extinction ofcued freezing. We then tested for a renewal effect. In this par-adigm, the subject that extinguished cued freezing in context Bis reexposed to the original training context A; renewal ofcued freezing is observed in context A because the extinctionwas context B-dependent. We found that both control andNaB-pretreated mice had renewal effect in context A. How-ever, once again, the NaB group extinguished renewed-freezing significantly faster than controls even though the sin-gle NaB injection was given 2 weeks prior to testing forrenewal effect (18). These findings have three major implica-tions: (a) cued freezing is more resistant to extinction thancontextual freezing, (b) the HDAC inhibitor facilitated learningof extinction-resistant phenotype (cued memory), and (c) inhi-bition of HDAC and increase in histone acetylation may have

profound long-term effects to facilitate extinction learning atleast 2 weeks after administration of NaB.

Fear Memory SummaryThe results suggest that depending on the intensity of fearconditioning, different signaling pathways may encode LTM.Relatively weak training (one CS–US pairing) requires NOsignaling for successful acquisition and consolidation of con-textual LTM, which is also dependent on increased plasmacorticosterone levels. Cued LTM seems to be independent ofNO signaling and increase in plasma corticosterone (13).However, cued fear memory is more resistant to extinctionthan contextual fear memory. This could be the result of dif-ferent circuitries involved in the formation of cued (amyg-dala) and contextual (hippocampus) memory. Yet, increasein histone acetylation even prior to fear conditioning accel-erates the extinction of cued LTM; thus, HDAC inhibitorscould be potential “prophylactics” against resistance toextinction learning.

Deficits in contextual LTM in the absence of the nNOSgene can be rescued by an exogenous NO donor or HDACinhibitor, both of which increase histone acetylation. In addi-tion, multiple CS–US pairings rescue contextual LTM ofnNOS KO mice through a NO-independent pathway. Themultiple footshocks increased plasma corticosterone ofnNOS KO mice, which could have facilitated the consolida-tion of contextual LTM. However, further studies arerequired to determine whether multiple CS–US pairingscause changes in histone acetylation, which are comparablewith changes induced by administration of NaB to single-shocked nNOS KO mice.

Drug-Associated Memory

Acquisition of Drug MemoryAddiction is a multifaceted disease, and increasing evidencesupports the role of learning and memory in the develop-ment of drug addiction. Studies suggest that neural sub-strates and pathways associated with learning and memorybecome controlled by addictive drugs, resulting in persistentdrug-seeking behavior and failure to extinguish such behav-ior (45–48). Brain regions such as amygdala, hippocampus,striatum, nucleus accumbens, and prefrontal cortex, whichhave a role in learning and memory, are targeted by drugsof abuse (49–52). Subsequently, drug-paired stimuli con-verge into conditioned stimuli that can induce powerfulcraving and precipitate relapse in abstinent drug users(53–57). In this review, we focused on the learning andmemory aspect of addiction.

The acquisition of drug memory requires (a) unconditionedstimulus that changes the affective state of the organism and(b) associative learning (58). In animal models, the role of Pav-lovian conditioning in drug-seeking behavior has been impli-cated in conditioned place preference (CPP; nonoperant

Itzhak et al. 563

behavior) and drug self-administration (operant behavior)studies (59). Traditionally, in CPP studies, the subject is pairedwith drug reward in a specific context for a few days. Subse-quently, when the subject has a free choice between the drug-and no-drug-paired contexts, it shows approach behavior forthe drug-paired context. The expression of place preference isviewed as reactivity to drug-associated CS (60). The CPP para-digm has also been used to investigate extinction of “drug-seeking behavior” and reinstatement of conditioned response(61–65). We have shown that reinstatement of cocaine CPP(following its extinction) is a drug-specific phenomenon thatcan be triggered only by drugs that share a similar mechanism

of action with that of cocaine (65). These findings suggest thatthe subject (a) learns the association between a context andan explicit reward, (b) discriminates between differentrewards, and (c) encodes LTM of drug reward, which is noterased after extinction learning because it resurfaces on reex-posure to the drug.

The transition from drug use to drug addiction involvesescalation in drug intake that becomes excessive and difficultto control (66–68). Several studies have demonstrated the dif-ferences between short (1–2 h) and long (6–8 h) drug self-administration in rats. Long but not short access results inescalation in psychostimulants self-administration (69–72).Notably, however, in these studies, the escalation of drugintake is rather modest, about 1.25- to 1.5-fold increase in thenumber of drug infusion over a 2-week period. Some studiesquestioned the significance of escalation in psychostimulantself-administration because the final outcome of escalated andfixed drug self-administration was not different (73–75).

A major barrier in previous CPP studies is inherent in thedesign of a fixed daily dose of drug administration that doesnot simulate the escalation in drug use in addicts. Recently,we extended our studies on the development of drug memoryby varying the dose of cocaine during the conditioning phase.The most significant findings are that (a) escalating regimen ofcocaine during CPP training resulted in high magnitude ofCPP, which was long-lasting relative to conditioning by fixedor descending doses of cocaine (Fig. 2; ref. 76), and that (b)conditioning by descending doses of cocaine resulted in notonly low magnitude of CPP but also extinguished previouslyestablished “strong” CPP (76). Recently, Conrad et al. (77)reproduced our studies and confirmed our findings.

The reason for the enhanced and persistent place prefer-ence following training by escalating dose of cocaine whencompared with the fixed daily dose is unclear. Recently, how-ever, we found that upregulation of the NR2B subunit of theNMDAR in hippocampus of mice conditioned by the escalatingcocaine regimen is significantly higher than that in mice condi-tioned by a fixed daily dose of cocaine (S. Liddie and Y. Itzhak,submitted). Given the role of NR2B subunit in learning andmemory (78), the strength of cocaine-associated memoryencoded by escalating doses of cocaine may be due to themarked upregulation of this receptor subunit.

At the behavioral level, contemporary theories on Pavlov-ian conditioning point to the role of prediction error in encod-ing LTM. The studies of Rescorla and Wagner (79) on naturalreinforcement suggest that reinforcement learning is depend-ent on the discrepancy between expected and obtainedreward. Hence, a “surprise element” is required for optimalassociative learning. The Rescorla–Wagner’s model (79) isillustrated by the following equation:

DVCS5cðVmax 2VallÞ

where V is the associative strength, D is the amount of change,c is the learning rate parameter (must be a number between 0

The magnitude (A) and persistence (B) of cocaine-

induced place preference (CPP) is dependent on the

different schedules of cocaine administration. A: The

magnitude of CPP following training by escalating

doses of cocaine was significantly higher than that

induced by fixed and descending doses of cocaine

(#P<0.05). Note that no differences between the two

escalating schedules of cocaine were observed

despite the difference in total amount of drug given

during the four training days (*P<0.05 is the differ-

ences between all drug groups when compared with

saline control group). B: The persistence of CPP in

the escalating group (Esc-C) was significantly longer

than in the fixed group (Fix-C; *P<0.05) for at least 7

days post-training. The x-axis represents the number

of days that place preference was recorded post-

training (data from ref. 76).

FIG 2

IUBMB LIFE

564 Memory Strength and Maladaptive Behaviors

and 1.0 and is a result of multiplying the CS intensity by theUS intensity), Vmax is the maximum amount of associativestrength that the US can support (100%), Vall is the totalamount of associative strength for all stimuli present (initially0 because no conditioning has occurred), and VCS is the asso-ciative strength of the CS.

Table 2 computes theoretical values for associativestrength rising from fixed, escalating, and decreasing doses ofa reinforcer US. We computed only changes in the US as fol-lowing: for fixed schedule, a ratio of 1 (same dose everyday);for escalating schedule, a ratio of 2 (a twofold increase in dailydose); and for descending schedule, a ratio of 0.5 (a twofolddecrease in daily dose; see Fig. 2A as an example). Weassumed an arbitrary fixed value for the CS to equal 0.2(which is within the range of 0–1.0) because we are not surewhat is the actual change in the CS. The results summarizedin the Table 2 are presented as learning curves in Fig. 3. Theresults show associative strength of the escala-ting>fixed>descending, which is in agreement with experi-mental values shown in Fig. 2A. Based on this equation, theprogressive daily change in the reinforcer US is the criticalparameter, whereas the actual quantity of the reinforcer isnot. As shown in Fig. 2A, the magnitude of CPP induced by 3,6, 12, and 24 mg/kg cocaine is similar to that induced by 2, 4,8, and 16 mg/kg cocaine, although the total amount of the for-mer is 1.5-fold higher than the latter. In Fig. 3, we plotted the-oretical values for up to 10 conditioning sessions, whereas inour experiments, we had only four conditioning days. Thethree different schedules of reinforcer US show asymptoticlearning curves; however, it is evident that maximal associa-tive strength is achieved a lot faster by the escalating schedule(Fig. 3). If we assume that changes in the US contribute tochanges in the salience of the CS, then inflation (escalatingregimen) and deflation (descending dose) of the CS will furthercontribute to the differences between the learning curves ofthe different schedules (Fig. 3).

Studies by Schultz and coworkers (80–83) posit that ifreward outcome is greater than expected, a positive predictionerror is encoded; if reward is no different than expected, no pre-diction error occurs, and when reward is lower than expected, anegative prediction error occurs. Our findings from escalating

and descending schedules of cocaine may be reminiscent of theprediction error theory. The findings that daily increments incocaine dosage led to the highest and the long-lasting magnitudeof place preference (Fig. 2) suggest that this schedule elicited (a)daily increase in reward magnitude and (b) strengthening ofcocaine-associated memory. Conditioning by daily descendingdoses of cocaine induced (a) the lowest and the shortest-lastingmagnitude of place preference and (b) extinction learning (76).

It is thought that changes in phasic ventral tegmental area(VTA) dopaminergic transmission encode prediction error-dependent learning (80–83). In addition, phasic stimulation ofdopaminergic VTA neurons resulted in behavioral conditioningmore so than tonic stimulation of dopaminergic VTA neurons

Theoretical associative strength acquired through conditioning by fixed, escalating, and descending schedules of drug

unconditioned stimulus (US)

Fixed US: US/US 5 1 Escalating US: 2US/US 5 2 Descending US: 0.5US/US 5 0.5

Day c 5 US 3 CS 5 1 3 0.2 5 0.2 DVCS (%) c 5 US 3 CS 5 1 and 2 3 0.2 DVCS (%) c 5 US 3 CS 5 1 and 0.5 3 0.2 DVCS (%)

1 0.2 3 (100 2 0) 5 20 20 0.2 3 (100 2 0) 5 20 20 0.2 3 (100 2 0) 5 20 20

2 0.2 3 (100 2 20) 5 16 36 0.4 3 (100 2 20) 5 32 52 0.1 3 (100 2 20) 5 8 28

3 0.2 3 (100 2 35) 5 12.8 48.8 0.4 3 (100 2 52) 5 19.2 71.2 0.1 3 0.2 3 (100 2 28) 5 7.2 35.2

4 0.2 3 (100 2 48) 5 10.24 59.04 0.4 3 (100 2 71.2) 5 11.5 82.7 0.1 3 0.2 3 (100 2 35.2) 5 6.4 41.6

Hypothetical learning curves of appetitive condition-

ing by fixed, escalating, and descending schedules of

a reinforcer unconditioned stimulus (US). Associative

strength was computed by the Rescorla–Wagner’s

model (79), and the actual values are depicted in

Table 2. Note, however, that we had only four condi-

tioning days (depicted by dashed vertical line and

Table 2), whereas the theoretical values are extended

to 10 days. The three different schedules of the rein-

forcer US show asymptotic learning curves; how-

ever, it is evident that maximal associative strength

is achieved a lot faster by the escalating schedule.

These curves are reminiscent of the experimental

values of cocaine CPP depicted in Fig. 2A.

TABLE 2

FIG 3

Itzhak et al. 565

(84). Cocaine influences both tonic and phasic dopaminergictransmission, and it is unclear what the changes in phasicdopamine release are during place conditioning experiments.However, it has been suggested that in the presence of robustchanges in tonic dopamine (e.g., presence of the drug), the lat-ter may serve also as a prediction error signal (81). In addi-tion, it has been proposed that chronic exposure to cocaineattenuates tonic dopamine and increases phasic dopaminerelease (85). Therefore, we posit that our pattern of escalationin drug exposure, using the CPP paradigm, may fulfill some ofthe conditions of a positive prediction error manifestation. Assuch, this paradigm may serve as a powerful tool to investigatethe strength of appetitive-drug memory, which has not beeninvestigated thus far.

Labile and Stable Drug Memory: Nitric Oxide-Dependent and -Independent PathwaysIn earlier studies, we have shown that the acquisition andreconsolidation of cocaine-associated contextual memory isdependent on NO signaling. These results have been based onconditioning of mice by a fixed daily dose of cocaine (20 mg/kg/day for 4 days). We found that unlike WT mice, nNOS KOmice acquired short-lived CPP, which was extinguished within7 days post-training in the absence of extinction training (e.g.,repeated reexposure to the context; ref. 86). Moreover, unlikeWT mice, administration of cocaine priming injection to nNOSKO mice did not reinstate place preference (86). These findingssuggest that in the absence of NO signaling, cocaine memoryis relatively weak and prone to extinction. In recent studies,we investigated the effect of the nNOS inhibitor 7-nitroindazoleon acquisition of CPP that was induced by escalating regimenof cocaine (3, 6, 12, and 24 mg/kg) and a fixed daily dose(11.25 mg/kg/day; average of the escalating schedule). Wefound that the nNOS inhibitor had no effect on the acquisitionof place preference by the escalating regimen, whereas it sup-pressed the formation of fixed-dose cocaine CPP (S. Liddie andY. Itzhak, submitted).

In previous studies, we also showed that reconsolidation ofcocaine-related memory that was acquired following condition-ing by a fixed daily dose of cocaine (20 mg/kg) was susceptibleto disruption by inhibition of nNOS. Mice that received thenNOS inhibitor 7-nitroindazole upon retrieval of cocaine mem-ory (first CPP test) ceased to show place preference in the nexttest, whereas mice that received saline maintained place prefer-ence (87). In a recent study, we found that administration of thenNOS inhibitors SMTC or 7-nitroindazole upon retrieval ofcocaine memory that was acquired following conditioning byescalating regimen had no effect on drug memory; the micemaintained their preference for the drug-paired compartment.This finding suggests that unlike reconsolidation of cocaine mem-ory that was acquired by a fixed dose of cocaine, reconsolidationof memory encoded by the escalating dose is not susceptible todisruption through the NO signaling pathway. If we assume thatconditioning by fixed and escalating doses of cocaine results in“weak” and “strong” memory, respectively (Fig. 2B), the results

from inhibition of NO signaling are reminiscent of the results offear conditioning: “weak” but not “strong” fear memory is sus-ceptible to inhibition of the NO signaling pathway.

Extinction of Drug MemoryBecause the severity of addiction may depend on the strengthof drug memory, and cocaine memory encoded by escalatingregimen is persistent (76), we sought to investigate whetheracceleration of extinction learning can facilitate the inhibitionof cocaine memory.

Phosphodiesterase inhibitors (PDEis) inhibit the degradationof the second messengers cGMP and cAMP. Evidence suggeststhat some PDEis improve cognition, and thus an increase inbrain cAMP and cGMP facilitates learning and memory (88). Wehad investigated the efficacy of three different PDEis in extinc-tion of cocaine-induced CPP in B6129S mice conditioned byescalating doses of cocaine (3, 6, 12, and 24 mg/kg). Followingacquisition of place preference, mice were reexposed to thetraining context for several days in the absence of cocaine(extinction training). Immediately following each extinction ses-sion, mice received (a) saline/vehicle, (b) rolipram, a PDE4inhibitor, which increases cAMP levels in brain, (c) BAY-73-6691, a PDE9 inhibitor, which selectively increases cGMP levelsin brain, or (d) papaverine, a PDE10A inhibitor, which increasesboth cAMP and cGMP levels. Mice that received saline/vehicleduring extinction training did not show reduction in CPP for>10 days (Fig. 4; ref. 89). The results of administration of BAY-73–6691 showed (a) dose-dependent increase in cGMP in hippo-campus and amygdala, (b) significant facilitation of extinction,and (c) diminished reinstatement of cocaine CPP (Fig. 4; ref.89). Rolipram and papaverine had no significant effect onextinction of cocaine CPP (Fig. 4; papaverine data not shown;ref. 89). These results suggest that increase in hippocampal andamygdalar cGMP levels through blockade of PDE9 has a signifi-cant role in memory consolidation of extinction learning. It alsoappears that targeting a specific PDE is more critical than tar-geting any PDE which metabolizes cGMP.

As described in the section on fear memory, we found thatthe HDAC inhibitor NaB accelerated extinction of cued fearmemory, which is relatively more resistant to extinction thancontextual memory. Therefore, we investigated whether NaBaccelerates extinction of cocaine-associated memory. C57BL/6mice were conditioned with either a fixed daily dose of cocaine(15 mg/kg) or escalating regimen (3, 6, 12, and 24 mg/kg). A sin-gle systemic administration of NaB (1.2 g/kg, the same dose thatwas tested for fear memory extinction) either before or after theconditioning phase enhanced expression and delayed the extinc-tion of place preference (34). The results suggest that regardlessof the scheduling of either cocaine or the HDAC inhibitor, NaB-induced histone hyperacetylation in the hippocampus strength-ened cocaine-associated contextual memory (34).

Because NaB had opposing effects on the extinction ofcued fear memory and contextual drug memory, it is likelythat different mechanisms underlie the extinction of these mem-ories. The opposing effects of NaB may not be attributed to the

IUBMB LIFE

566 Memory Strength and Maladaptive Behaviors

differences between the substrates involved in the formation ofcued (amygdala-dependent) and contextual (hippocampus-dependent) memory because NaB had no effect on contextualfear memory (18) but it impeded extinction of contextualcocaine memory (34). Brain regions involved in extinction offear memory include the infralimbic structure of the prefrontalcortex, ventral hippocampus, and amygdala (90). Brain regionsassociated with extinction of drug memory include not only theinfralimbic prefrontal cortex but also the nucleus accumbensshell and medial dorsal hypothalamus (91). Thus, the infralim-bic structure is one of the overlapping substrates required forextinction of both fear memory and drug memory. However, itis unclear whether NaB-induced histone hyperacetylation in theinfralimbic structure should have a similar effect on extinctionof fear and drug memory. It is likely that NaB-induced hyper-acetylation has differential effects on reward- and aversion-related memory. In other words, an interaction betweenenhanced transcriptional activity and the affective state of theorganism (fear versus pleasure or craving during withdrawal)may result in differences in synaptic plasticity underlyingextinction of LTM. In addition, it is apparent that some brainregions involved in extinction of fear and drug memory do notcompletely overlap; thus, further studies are needed to investi-gate these differences at the molecular level.

Concluding RemarksThe role of Pavlovian conditioning in maladaptive behaviorssuch as debilitating fear and overpowering desire to continue

to take drugs has been the focus of many investigations (seerefs. 2–4 and 92 for recent reviews). The goal of the currentreview was to tackle the distinction between memory strengthin the formation of maladaptive behaviors associated withaversive and appetitive conditioning. Few studies have focusedon the significance of memory strength in terms of mecha-nisms underlying acquisition, consolidation, and reconsolida-tion of weak and strong memory (aversive or appetitive). Itappears that strong fear memory, which was associated withdownregulation of the NR2B subunit of the NMDAR in thebasal and lateral amygdala, was more resistant to reconsolida-tion than weaker fear memory (11). By and large, no notabledifferences between strong and weak appetitive conditioningwere observed (93).

Our studies on fear memory and cocaine memory suggest(a) some similarities in signaling molecules involved in theacquisition of aversive and appetitive memories and (b) distinc-tion in putative pathways underlying the acquisition of weakversus strong memory. The results of fear conditioning studies,both acquisition and reconsolidation, suggest a notable role ofNO signaling in weak but not strong conditioning. In the caseof WT mice, it does not mean that strong conditioningbypasses the NO signaling, it only suggests that strong condi-tioning recruits other signaling molecules that may over-shadow the participation of the NO signaling.

In the same vein, our studies on cocaine memory suggestthat the strength of the memory is dependent on the condition-ing paradigm. Fixed daily dose of cocaine results in relativelylabile memory that is sensitive to manipulations through the

Effect of phosphodiesterase (PDE) inhibitors on extinction of cocaine-induced CPP. The y-axes represent the mean 6 SEM of dif-

ference in time spent on the drug-paired versus the saline-paired compartment. The x-axis shows the timeline for the extinction

experiment. Day 1 represents CPP expression that was acquired by escalating schedule of cocaine (3, 6, 12, and 24 mg/kg). The

test drugs and vehicles were administered immediately following each additional test, staring from the second day (first extinc-

tion training session) and for seven more days with a 2-day break (days 5 and 8). A: Escalating schedule of cocaine resulted in

persistent place preference, and the PDE 4 inhibitor rolipram had no significant effect on extinction. B: The PDE 9 inhibitor

BAY-73-6691 significantly reduced the magnitude of CPP over time (P<0.001 overall two-way ANOVA). Significant differences

between the two groups were observed on days 10 and 11 of extinction training (*P<0.05). BAY-73-6691 also attenuated place

preference reinstatement. Cocaine priming injection on day 12 resulted in significant increase in CPP in vehicle group

(# P<0.001), but not in the BAY-73-6691 group (adapted from ref. 89).

FIG 4

Itzhak et al. 567

NO signaling pathway. Escalating regimen of cocaine resultedin stable memory that cannot be manipulated through the NOsignaling pathway. These results are evident from both acqui-

sition and reconsolidation of drug memory. Again, it is likely toassume that encoding strong cocaine memory is dependent onadditional signaling molecules that overshadow participationof NO.

Figure 5 summarizes a putative model for the formation ofstable and labile LTM of aversive or appetitive stimulus. TheNMDAR is depicted as an initial molecular target because ithas a major role in synaptic plasticity associated with (a) theformation of fear memory and (b) the effects of drugs of abuse.However, apart from the NMDAR, various neurotransmittersystems are involved in fear response and appetitive responseto drugs of abuse. For instance, fear response involves activa-tion of the hypothalamic–pituitary–adrenal axis, stress hor-mones, and glutamatergic transmission and dopamine in theprefrontal cortex (94). Appetitive response to drugs of abuseinvolves an increase in the levels of mesocorticolimbic dopa-mine; however, various monoamine transporters and recep-tors also take part in the reinforcing effects of drugs of abuse.Apart from the known dopamine–glutamate interaction (95),we posit that one common molecular target for fear responseand appetitive response is the NMDAR.

Additionally, increases in calcium influx, which is associ-ated with NMDAR activation, may result from activation ofother receptors and signaling molecules. nNOS, however, islinked to the NMDAR through PSD-95, and increased calciuminflux activates nNOS (96). The increased NO level stimulatessoluble guanylate cyclase (sGC) leading to cGMP-mediated acti-vation of PKG, which subsequently stimulates ERK1/2. Evi-dence suggests that NO-induced S-nitrosylation of HDAC2increases histone acetylation and subsequent transcriptionalactivity (97), which is likely involved in consolidation of LTM;in this case, we posit it is labile LTM. NO signaling may furthermodulate pCREB levels (14,15,25), as depicted in Table 1.However, we posit that transcription that develops throughNO-pCREB pathway may result in labile LTM (Fig. 5). On theother hand, we posit that other downstream NMDAR as wellas downstream PKA signaling molecules are recruited for theformation of stable LTM. This alternate pathway (left side Fig.5) may involve PKA/ERK/pCREB/CBP (CREB binding protein);CBP is a main HAT involved in the consolidation of LTM (98).Transcriptional activity through this pathway may be differentthan that generated through the NO signaling pathway; as aresult, the memory strength encoded by each pathway may bedifferent.

Although similarities in the acquisition and reconsolidationof aversive and appetitive memory were observed, in terms ofNO-dependent signaling, it should be noted that signaling mol-ecules involved in the formation of strong aversive and strongappetitive memory could be different. For instance, as indi-cated above, strong fear conditioning was associated withdownregulation of amygdala NR2B subunit (11), whereasstrong cocaine memory (escalating doses of cocaine) was asso-ciated with upregulation of hippocampal NR2B subunit of theNMDAR (S. Liddie and Y. Itzhak, submitted). In addition, it isimportant to point out that unlike in the laboratory setting,

A putative model for the formation of stable and labile

LTM of aversive (fear) and appetitive (drug) stimulus.

The N-methyl-D-aspartate receptor (NMDAR) has a

major role in synaptic plasticity associated with (a) for-

mation of fear memory and (b) drugs of abuse. nNOS

is linked to the NMDAR through PSD-95, and

increased calcium influx activates nNOS (96). The

increased NO level stimulates the soluble guanylate

cyclase (sGC) leading to cGMP-mediated activation of

PKG, which subsequently stimulates ERK1/2. NO can

induce histone acetylation and subsequent transcrip-

tional activity through S-nitrosylation of HDAC2 (97).

This pathway is likely involved in consolidation labile

of LTM. NO signaling may further modulate pCREB

levels (14,15,25), as depicted in Table 1; transcription

that develops through NO-pCREB pathway may also

result in labile LTM. On the other hand, other down-

stream NMDAR signaling molecules as well as D1/D5

dopamine receptors may activate PKA signaling mole-

cules, which are recruited for the formation of a stable

LTM. This alternate pathway (left side) may involve

PKA/ERK/pCREB/CBP (CREB binding protein); CBP is a

main histone acetyltransferase involved in consolida-

tion of LTM (98). Transcriptional activity through this

pathway may be different than that generated through

the NO signaling pathway; as a result, the memory

strength encoded by each pathway could be different.

[Color figure can be viewed in the online issue, which

is available at wileyonlinelibrary.com.]

FIG 5

IUBMB LIFE

568 Memory Strength and Maladaptive Behaviors

acquisition of fear memory and drug memory in humans ismore complex. For instance, fear memory can be fixated to aspecific cue and/or context, whereas drug memory can beencoded through associations with multiple cues (drug para-phernalia, images, lights, sounds, etc.) and multiple contexts(crack house, party house, friend’s house, etc.). However, thequestions that remain are (a) what are the mechanisms under-lying the formation of a distinctive strong memory and (b) howcan such memories be manipulated to manage particularbehavioral phenotype.

The results of extinction studies suggest some differencesbetween the extinction of fear memory and cocaine memory.Inhibition of HDAC, which resulted in increased histone acetyla-tion primarily in the hippocampus and not amygdala of WTmice (Table 1; ref. 34), facilitated extinction of cued fear mem-ory but delayed the extinction of contextual cocaine memory.This difference is probably not related to the differencesbetween cued and contextual memory, because contextual fearmemory was rapidly extinguished and NaB had no effect.Hence, these findings suggest that increased histone acetylationmay have different effects on appetitive and aversive memory.An interaction between enhanced histone acetylation-inducedtranscriptional activity and the affective state of the organismmay result in differences in synaptic plasticity underlying thestability and extinction of LTM. Further studies are required tounravel (a) the signaling molecules underlying the formation ofweak and strong associative memory and (b) the mechanismsinvolved in extinction of aversive and appetitive memory.

Elucidation of mechanisms that encode different strengthof aversive and appetitive LTM will enable targeting specificsignaling molecules to inhibit the formation of such memoriesthat yield disruptive behavior. Elucidation of signaling mole-cules and gene expression associated with (a) reconsolidationand (b) extinction of aversive and appetitive memory is criticalfor the development of pharmacotherapies that can success-fully destabilize these memories. Manipulation of aversivememory can ameliorate anxiety-related disorders; manipula-tion of drug memory may ameliorate “automatized behavior”and compulsive drug-seeking behavior.

AcknowledgmentsThe work described was supported by Bridge Funding Grantsfrom the University of Miami Miller School of Medicine, Miami,FL, and RO1DA026878 from the National Institute on DrugAbuse, National Institutes of Health (YI).

References

[1] Bouton, M. E., and Moody, E. W. (2004) Memory processes in classical condi-

tioning. Neurosci. Biobehav. Rev. 28, 663–674.

[2] Mineka, S., and Oehlberg, K. (2008) The relevance of recent developments in

classical conditioning to understanding the etiology and maintenance of anx-

iety disorders. Acta Psychol. 127, 567–580.

[3] Milton, A. L., and Everitt, B. J. (2012) The persistence of maladaptive mem-

ory: addiction, drug memories and anti-relapse treatments. Neurosci. Biobe-

hav. Rev. 36, 1119–1139.

[4] Everitt, B. J. (2014) Neural and psychological mechanisms underlying com-

pulsive drug seeking habits and drug memories—indications for novel treat-

ments of addiction. Eur. J. Neurosci., 40, 2163–2182.

[5] Kessler, R. C., and Wang, P. S. (2008) The descriptive epidemiology of com-

monly occurring mental disorders in the United States. Annu. Rev. Public

Health 29, 115–129.

[6] DeRbiec, J., Bush, D. E., and LeDoux, J. E. (2011) Noradrenergic enhancement

of reconsolidation in the amygdala impairs extinction of conditioned fear in

rats—a possible mechanism for the persistence of traumatic memories in

PTSD. Depress. Anxiety 28, 186–193.

[7] Johansen, J. P., Cain, C. K., Ostroff, L. E., and LeDoux, J. E. (2011) Molecular

mechanisms of fear learning and memory. Cell 147, 509–524.

[8] Zovkic, I. B., and Sweatt, J. D. (2013) Epigenetic mechanisms in learned fear:

implications for PTSD. Neuropsychopharmacology 38, 77–93.

[9] Maren, S., Phan, K. L., and Liberzon, I. (2013) The contextual brain: implica-

tions for fear conditioning, extinction and psychopathology. Nat. Rev. Neuro-

sci. 14, 417–428.

[10] Hauner, K. K., Mineka, S., Voss, J. L., and Paller, K. A. (2012) Exposure ther-

apy triggers lasting reorganization of neural fear processing. Proc. Natl.

Acad. Sci. USA 109, 9203–9208.

[11] Wang, S. H., de Oliveira Alvares, L., and Nader, K. (2009) Cellular and sys-

tems mechanisms of memory strength as a constraint on auditory fear

reconsolidation. Nat. Neurosci. 12, 905–913.

[12] Wiltgen, B. J., Sanders, M. J., Anagnostaras, S. G., Sage, M. R., and

Fanselow, M. S. (2006) Context fear learning in the absence of the hippo-

campus. J. Neurosci. 26, 5484–5491.

[13] Kelley, J. B., Balda, M. A., Anderson, K. L., and Itzhak, Y. (2009) Impairments

in fear conditioning in mice lacking the nNOS gene. Learn. Mem. 16, 371–

378.

[14] Hawkins, R. D., Son, H., and Arancio, O. (1998) Nitric oxide as a retrograde

messenger during long-term potentiation in hippocampus. Prog. Brain Res.

118, 155–172.

[15] Lu, Y. F., Kandel, E. R., and Hawkins, R. D. (1999) Nitric oxide signaling con-

tributes to late-phase LTP and CREB phosphorylation in the hippocampus.

J. Neurosci. 23, 10250–10261.

[16] Lewin, M. R., and Walters, E. T. (1999) Cyclic GMP pathway is critical for

inducing long-term sensitization of nociceptive sensory neurons. Nat. Neu-

rosci. 2, 18–23.

[17] M€uller, U. (1996) Inhibition of nitric oxide synthase impairs a distinct form

of long-term memory in the honeybee, Apis mellifera. Neuron 16, 541–549.

[18] Itzhak, Y., Anderson, K. L., Kelley, J. B., and Petkov, M. (2012) Histone acety-

lation rescues contextual fear conditioning in nNOS KO mice and acceler-

ates extinction of cued fear conditioning in wild type mice. Neurobiol.

Learn. Mem. 97, 409–417.

[19] Kelley, J. B., Anderson, K. L., and Itzhak, Y. (2010) Pharmacological modula-

tors of nitric oxide signaling and contextual fear conditioning in mice. Psy-

chopharmacology 210, 65–74.

[20] Pugh, C. R., Tremblay, D., Fleshner, M., and Rudy, J. W. (1997) A selective

role for corticosterone in contextual-fear conditioning. Behav. Neurosci. 111,

503–511.

[21] Thompson, B. L., Erickson, K., Schulkin, J., and Rosen, J. B. (2004) Corticos-

terone facilitates retention of contextually conditioned fear and increases

CRH mRNA expression in the amygdala. Behav. Brain. Res. 149, 209–215.

[22] Hui, G. K., Figueroa, I. R., Poytress, B. S., Roozendaal, B., McGaugh, J. L., et al.

(2004) Memory enhancement of classical fear conditioning by post-training

injections of corticosterone in rats. Neurobiol. Learn. Mem. 81, 67–74.

[23] Roozendaal, B., Hui, G. K., Hui, I. R., Berlau, D. J., McGaugh, J. L., et al.

(2006) Basolateral amygdala noradrenergic activity mediates corticosterone-

induced enhancement of auditory fear conditioning. Neurobiol. Learn. Mem.

86, 249–255.

[24] Marchand, A. R., Barbelivien, A., Seillier, A., Herbeaux, K., Sarrieau, A.,

et al. (2007) Contribution of corticosterone to cued versus contextual fear in

rats. Behav. Brain Res. 183, 101–110.

[25] Kelley, J. B., Anderson, K. L., Altmann, S. L., and Itzhak, Y. (2011) Long-term

memory of visually cued fear conditioning: roles of the neuronal nitric oxide

Itzhak et al. 569

synthase gene and cyclic AMP response element-binding protein. Neuro-

science 174, 91–103.

[26] Sweatt, J. D. (2009) Experience-dependent epigenetic modification in the

central nervous system. Biol. Psychiatry 65, 191–197.

[27] Jenuwein, T., and Allis, C. D. (2001) Translating the histone code. Science

293, 1074–1080.

[28] Guan, J. S., Haggarty, S. J., Giacometti, E., Dannenberg, J. H., Joseph, N.,

et al. (2009) HDAC2 negatively regulates memory formation and synaptic

plasticity. Nature 459, 55–60.

[29] Levenson, J. M., O’Riordan, K. J., Brown, K. D., Trinh, M. A., Molfese, D. L.,

et al. (2004) Regulation of histone acetylation during memory formation in

the hippocampus. J. Biol. Chem. 279, 40545–40559.

[30] Vecsey, C. G., Hawk, J. D., Lattal, K. M., Stein, J. M., Fabian, S. A., et al. (2007)

Histone deacetylase inhibitors enhance memory and synaptic plasticity via

CREB:CBP-dependent transcriptional activation. J. Neurosci. 27, 6128–6140.

[31] Guan, Z., Giustetto, M., Lomvardas, S., Kim, J. H., Miniaci, M. C., et al.

(2002) Integration of long-term-memory-related synaptic plasticity involves

bidirectional regulation of gene expression and chromatin structure. Cell

111, 483–493.

[32] Agalioti, T., Chen, G., and Thanos, D. (2002) Deciphering the transcriptional

histone acetylation code for a human gene. Cell 111, 381–392.

[33] Koshibu, K., Gr€aff, J., Beullens, M., Heitz, F. D., Berchtold, D., et al. (2009)

Protein phosphatase 1 regulates the histone code for long-term memory. J.

Neurosci. 29, 13079–13089.

[34] Itzhak, Y., Liddie, S., and Anderson, K. L. (2013) Sodium butyrate-induced

histone acetylation strengthens the expression of cocaine-associated con-

textual memory. Neurobiol. Learn. Mem. 102, 34–42.

[35] Bouton, M. E. (2002) Context, ambiguity, and unlearning: sources of relapse

after behavioral extinction. Biol. Psychiatry 52, 976–986.

[36] Bouton, M. E. (2004) Context and behavioral processes in extinction. Learn.

Mem. 11, 485–494.

[37] Havermans, R. C., and Jansen, A. T. (2003) Increasing the efficacy of cue

exposure treatment in preventing relapse of addictive behavior. Addict.

Behav. 28, 989–994.

[38] Carter, B. L., and Tiffany, S. T. (1999) Meta-analysis of cue-reactivity in

addiction research. Addiction 94, 327–340.

[39] Powell, J., Gray, J., and Bradley, B. (1993) Subjective craving for opiates:

evaluation of a cue exposure protocol for use with detoxified opiate addicts.

Br. J. Clin. Psychol. 32, 39–53.

[40] Siegel, S., and Ramos, B. M. (2002) Applying laboratory research: drug

anticipation and the treatment of drug addiction. Exp. Clin. Psychopharma-

col. 10, 162–183.

[41] Nader, K. (2003) Memory traces unbound. Trends Neurosci. 26, 65–72.

[42] Power, A. E., Berlau, D. J., McGaugh, J. L., and Steward, O. (2006) Anisomy-

cin infused into the hippocampus fails to block “reconsolidation” but

impairs extinction: the role of re-exposure duration. Learn. Mem. 13, 27–34.

[43] Milad, M. R., and Quirk, G. J. (2002) Neurons in medial prefrontal cortex sig-

nal memory for fear extinction. Nature 420, 70–74.

[44] Santini, E., Muller, R. U., and Quirk, G. J. (2001) Consolidation of extinction

learning involves transfer from NMDA-independent to NMDA-dependent

memory. J. Neurosci. 21, 9009–9017.

[45] Hyman, S. E. (2005) Addiction: a disease of learning and memory. Am. J.

Psychiatry 162, 1414–1422.

[46] Hyman, S. E., Malenka, R. C., and Nestler, E. J. (2006) Neural mechanisms

of addiction: the role of reward-related learning and memory. Ann. Rev.

Neurosci. 29, 565–598.

[47] Nestler, E. J. (2004) Molecular mechanisms of drug addiction. Neurophar-

macology 47, 24–32.

[48] Torregrossa, M. M., Corlett, P. R., and Taylor, J. R. (2011) Aberrant learning

and memory in addiction. Neurobiol. Learn. Mem. 96, 609–923.

[49] McDonald, R. J., and White, N. M. (1993) A triple dissociation of memory

systems: hippocampus, amygdala, and dorsal striatum. Behav. Neurosci.

107, 3–22.

[50] White, N. M. (1996) Addictive drugs as reinforcers: multiple partial actions

on memory systems. Addiction 91, 921–949.

[51] Tuesta, L. M., and Zhang, Y. (2014) Mechanisms of epigenetic memory and

addiction. EMBO J. 33, 1091–1103.

[52] White, N. M., and Milner, P. M. (1992) The psychobiology of reinforcers.

Annu. Rev. Psychol. 43, 433–471.

[53] Newlin, D. B. (1992) A comparison of drug conditioning and craving for

alcohol and cocaine. Recent Dev. Alcohol 10, 147–164.

[54] Childress, A. R., Mozley, P. D., McElgin, W., Fitzgerald, J., Reivich, M., et al.

(1999) Limbic activation during cue-induced cocaine craving. Am. J. Psychi-

atry 156, 11–18.

[55] Robbins, S. J., Ehrman, R. N., Childress, A. R., and O’Brien, C. P. (1999)

Comparing levels of cocaine cue reactivity in male and female outpatients.

Drug Alcohol Depend. 53, 223–230.

[56] Robinson, T. E., and Berridge, K. C. (1993) The neural basis of drug craving: an

incentive-sensitization theory of addiction. Brain Res. Brain Res. Rev. 18, 247–291.

[57] Stewart, J. (2000) Pathways to relapse: the neurobiology of drug- and

stress-induced relapse to drug-taking. J. Psychiatry Neurosci. 25, 125–136.

[58] White, N. M., and Carr, G. D. (1985) The conditioned place preference is

affected by two independent reinforcement processes. Pharmacol. Biochem.

Behav. 23, 37–42.

[59] Shalev, U., Grimm, J. W., and Shaham, Y. (2002) Neurobiology of relapse

to heroin and cocaine seeking: a review. Pharmacol. Rev. 54, 1–42.

[60] Sanchis-Segura, C., and Spanagel, R. (2006) Behavioural assessment of

drug reinforcement and addictive features in rodents: an overview. Addict.

Biol. 11, 2–38.

[61] Kelley, J. B., Anderson, K. L., and Itzhak, Y. (2007) Long-term memory of

cocaine-associated context: disruption and reinstatement. Neuroreport 18,

777–780.

[62] Mueller, D., and Stewart, J. (2000) Cocaine-induced conditioned place pref-

erence: reinstatement by priming injections of cocaine after extinction.

Behav. Brain. Res. 115, 39–47.

[63] Mueller, D., Perdikaris, D., and Stewart, J. (2002) Persistence and drug-

induced reinstatement of a morphine-induced conditioned place preference.

Behav. Brain. Res. 136, 389–397.

[64] Parker, L. A., and McDonald, R. V. (2000) Reinstatement of both a condi-

tioned place preference and a conditioned place aversion with drug primes.

Pharmacol. Biochem. Behav. 66, 559–561.

[65] Itzhak, Y., and Martin, J. L. (2002) Cocaine-induced conditioned place prefer-

ence in mice: induction, extinction and reinstatement by related psychosti-

mulants. Neuropsychopharmacology 26, 130–134.

[66] Gawin, F. H., and Ellinwood, E. H., Jr. (1989) Cocaine dependence. Annu.

Rev. Med. 40, 149–161.

[67] Marlatt, G. A., Baer, J. S., Donovan, D. M., and Kivlahan, D. R. (1988) Addic-

tive behaviors: etiology and treatment. Annu. Rev. Psychol. 39, 223–252.

[68] Siegel, R. K. (1984) Changing patterns of cocaine use: longitudinal observa-

tions, consequences, and treatment. NIDA Res. Monogr. 50, 92–110.

[69] Ahmed, S. H., and Koob, G. F. (1998) Transition from moderate to excessive

drug intake: change in hedonic set point. Science 282, 298–300.

[70] Ahmed, S. H., Walker, J. R., and Koob, G. F. (2000) Persistent increase in

the motivation to take heroin in rats with a history of drug escalation. Neu-

ropsychopharmacology 22, 413–421.

[71] Anker, J. J., Holtz, N. A., and Carroll, M. E. (2012) Effects of progesterone

on escalation of intravenous cocaine self-administration in rats selectively

bred for high or low saccharin intake. Behav. Pharmacol. 23, 205–210.

[72] Kitamura, O., Wee, S., Specio, S. E., Koob, G. F., and Pulvirenti, L. (2006)

Escalation of methamphetamine self-administration in rats: a dose-effect

function. Psychopharmacology 186, 48–53.

[73] Knackstedt, L. A., and Kalivas, P. W. (2007) Extended access to cocaine self-

administration enhances drug-primed reinstatement but not behavioral sen-

sitization. J. Pharmacol. Exp. Ther. 322, 1103–1109.

[74] Oleson, E. B., and Roberts, D. C. (2008) Parsing the addiction phenomenon:

self-administration procedures modeling enhanced motivation for drug and

escalation of drug intake. Drug Discov. Today Dis. Models 5, 217–226.

[75] Pacchioni, A. M., Gabriele, A., and See, R. E. (2011) Dorsal striatum media-

tion of cocaine-seeking after withdrawal from short or long daily access

cocaine self-administration in rats. Behav. Brain Res. 218, 296–300.

IUBMB LIFE

570 Memory Strength and Maladaptive Behaviors

[76] Itzhak, Y., and Anderson, K. L. (2012) Changes in the magnitude of drug-

unconditioned stimulus during conditioning modulate cocaine-induced

place preference in mice. Addict. Biol. 17, 706–716.

[77] Conrad, K. L., Louderback, K. M., Milano, E. J., and Winder, D. G. (2013)

Assessment of the impact of pattern of cocaine dosing schedule during con-

ditioning and reconditioning on magnitude of cocaine CPP, extinction, and

reinstatement. Psychopharmacology 227, 109–116.

[78] Zhao, M. G., Toyoda, H., Lee, Y. S., Wu, L. J., Ko, S. W., et al. (2005) Roles

of NMDA NR2B subtype receptor in prefrontal long-term potentiation and

contextual fear memory. Neuron 47, 859–872.

[79] Rescorla, R. A., and Wagner, A. R. (1972) “A theory of Pavlovian condition-

ing: variations in the effectiveness of reinforcement and nonreinforcement.”

In Classical Conditioning II (Black, A. H., and Prokasy, W. F., eds). pp. 64 –

99, Appleton-Century-Crofts, New York.

[80] Schultz, W. (2007) Multiple dopamine functions at different time courses.

Annu. Rev. Neurosci. 30, 259–288.

[81] Schultz, W. (2011) Potential vulnerability of neuronal reward, risk, and deci-

sion mechanisms to addictive drugs. Neuron 69, 603–617.

[82] Waelti, P., Dickinson, A., and Schultz, W. (2001) Dopamine responses com-

ply with basic assumptions of formal learning theory. Nature 412, 43–48.

[83] Bayer, H. M., and Glimcher, P. W. (2005) Midbrain dopamine neurons

encode a quantitative reward prediction error signal. Neuron 47, 129–141.

[84] Tsai, H. C., Zhang, F., Adamantidis, A., Stuber, G. D., Bonci, A., et al. (2009)

Phasic firing in dopaminergic neurons is sufficient for behavioral condition-

ing. Science 324, 10870–10874.

[85] Wanat, M. J., Willuhn, I., Clark, J. J., and Phillips, P. E. (2009) Phasic dopa-

mine release in appetitive behaviors and drug addiction. Curr. Drug. Abuse

Rev. 2, 195–213.

[86] Balda, M. A., Anderson, K. L., and Itzhak, Y. (2006) Adolescent and adult

responsiveness to the incentive value of cocaine reward in mice: role of

neuronal nitric oxide synthase (nNOS) gene. Neuropharmacology 51,

341–349.

[87] Itzhak, Y., and Anderson, K. L. (2007) Memory reconsolidation of cocaine-

associated context requires nitric oxide signaling. Synapse 61, 1002–1005.

[88] Maurice, D. H., Ke, H., Ahmad, F., Wang, Y., Chung, J., et al. (2014) Advances in tar-

geting cyclic nucleotide phosphodiesterases. Nat. Rev. Drug Discov. 13, 290–314.

[89] Liddie, S., Anderson, K. L., Paz, A., and Itzhak, Y. (2012) The effect of phos-

phodiesterase inhibitors on the extinction of cocaine-induced conditioned

place preference in mice. J. Psychopharmacol. 26, 1375–1382.

[90] Sierra-Mercado, D., Padilla-Coreano, N., and Quirk, G. J. (2011) Dissociable

roles of prelimbic and infralimbic cortices, ventral hippocampus, and baso-

lateral amygdala in the expression and extinction of conditioned fear. Neu-

ropsychopharmacology 36, 529–538.

[91] Millan, E. Z., Marchant, N. J., and McNally, G. P. (2011) Extinction of drug

seeking. Behav. Brain Res. 217, 454–462.

[92] Sandk€uhler, J., and Lee, J. (2013) How to erase memory traces of pain and

fear. Trends Neurosci. 36, 343–352.

[93] Reichelt, A. C., and Lee, J. L. (2013) Memory reconsolidation in aversive

and appetitive settings. Front. Behav. Neurosci. 7, 118.

[94] Moghaddam, B. (2002) Stress activation of glutamate neurotransmission in

the prefrontal cortex: implications for dopamine-associated psychiatric dis-

orders. Biol. Psychiatry 51, 775–787.

[95] Morales, M., and Root, D. H. (2014) Glutamate neurons within the midbrain

dopamine regions. Neuroscience, 2014 May 27;36, 282C:60–68. doi: 10.1016/

j.neuroscience.2014.05.032. [Epub ahead of print].

[96] Sattler, R., Xiong, Z., Lu, W. Y., Hafner, M., MacDonald, J. F., et al. (1999)

Specific coupling of NMDA receptor activation to nitric oxide neurotoxicity

by PSD-95 protein. Science 284, 1845–1848.

[97] Nott, A., Watson, P. M., Robinson, J. D., Crepaldi, L., and Riccio, A. (2008)

S-Nitrosylation of histone deacetylase 2 induces chromatin remodelling in

neurons. Nature 455, 411–415.

[98] Alarc�on, J. M., Malleret, G., Touzani, K., Vronskaya, S., Ishii, S., et al. (2004)

Chromatin acetylation, memory, and LTP are impaired in CBP1/2 mice: a

model for the cognitive deficit in Rubinstein-Taybi syndrome and its amelio-

ration. Neuron 42, 947–959.

Itzhak et al. 571