Seediscussions,stats,andauthorprofilesforthispublicationat:https://www.researchgate.net/publication/311825391

Towardarationalandmechanisticaccountofmentaleffort

ArticleinAnnualReviewofNeuroscience·January2017

DOI:10.1146/annurev-neuro-072116-031526

CITATION

1

READS

829

7authors,including:

Someoftheauthorsofthispublicationarealsoworkingontheserelatedprojects:

DevelopingtoolsandtheoriesforhelpingpeoplemakebetterdecisionsViewproject

Resource-rationalityViewproject

AmitaiShenhav

BrownUniversity

26PUBLICATIONS742CITATIONS

SEEPROFILE

SebastianMusslick

PrincetonUniversity

6PUBLICATIONS2CITATIONS

SEEPROFILE

FalkLieder

UniversityofCalifornia,Berkeley

26PUBLICATIONS148CITATIONS

SEEPROFILE

MatthewMBotvinick

PrincetonUniversity

64PUBLICATIONS17,812CITATIONS

SEEPROFILE

AllcontentfollowingthispagewasuploadedbyFalkLiederon22December2016.

Theuserhasrequestedenhancementofthedownloadedfile.Allin-textreferencesunderlinedinblueareaddedtotheoriginaldocument

andarelinkedtopublicationsonResearchGate,lettingyouaccessandreadthemimmediately.

1

Annu.Rev.Neurosci.2017.40:X--X

doi:10.1146/annurev-neuro-072116-031526

Copyright©2017byAnnualReviews.Allrightsreserved

SHENHAVETAL.

MECHANISMSANDRATIONALITYOFMENTALEFFORT

TOWARDARATIONALANDMECHANISTICACCOUNTOFMENTALEFFORT

AmitaiShenhav,1,2SebastianMusslick,3FalkLieder,4WouterKool,5ThomasL.Griffiths,6JonathanD.Cohen,3,7andMatthewM.Botvinick8,91DepartmentofCognitive,Linguistic&PsychologicalSciences,BrownUniversity,Providence,RhodeIsland02912;email:amitai_shenhav@brown.edu2BrownInstituteforBrainScience,BrownUniversity,Providence,RhodeIsland029123PrincetonNeuroscienceInstitute,PrincetonUniversity,Princeton,NewJersey085444HelenWillsNeuroscienceInstitute,UniversityofCalifornia,Berkeley,California947205DepartmentofPsychology,HarvardUniversity,Cambridge,Massachusetts021386DepartmentofPsychology,UniversityofCalifornia,Berkeley,California947207DepartmentofPsychology,PrincetonUniversity,Princeton,NewJersey085408GoogleDeepMind,LondonM1c4AG,UnitedKingdom9GatsbyComputationalNeuroscienceUnit,UniversityCollegeLondon,LondonW1T 4JG,UnitedKingdom■ Abstract In spite of its familiar phenomenology, the mechanistic basis for mental effort remains poorlyunderstood. Althoughmost researchers agree thatmental effort is aversive and stems from limitations in ourcapacitytoexercisecognitivecontrol, it isunclearwhatgivesrisetothose limitationsandwhytheyresult inanexperienceof controlas costly.Thepresenceof thesecontrol costsalso raises furtherquestions regardinghowbesttoallocatementalefforttominimizethosecostsandmaximizetheattendantbenefits.Thisreviewexploresrecent advances in computationalmodeling and empirical research aimed at addressing these questions at thelevelofpsychologicalprocessandneuralmechanism,examiningboththelimitationstomentaleffortexertionandhowwemanagethoselimitedcognitiveresources.Weconcludebyidentifyingremainingchallengesfortheoreticalaccountsofmentaleffortaswellaspossibleapplicationsoftheavailablefindingstounderstandingthecausesofandpotentialsolutionsforapparentfailurestoexertthementaleffortrequiredofus.

Keywordsmotivation,cognitivecontrol,decisionmaking,reward,prefrontalcortex,executivefunction

2

1.INTRODUCTION

Allhighlyabstractconceptions,unaccustomedreasons,andmotivesforeigntotheinstinctivehistoryoftherace…prevail,whentheyeverdoprevail,witheffort;andthenormal…sphereofeffortisthusfoundwherevernon-instinctivemotivestobehavioraretoruletheday.(James1891,p.536)

Cognitiveeffortisamongthemostfamiliarandintuitivefixturesofmentallife.Differenttaskstransparentlydemanddifferentlevelsofcognitiveexertion,withsuccessorfailuredependingonhowhardwetry.Insomecases,difficultiespromptustoapplyourselvesmoreintently.Inothers,wedisengage,judgingthedemandedeffortnottobeworthit,orperhapsexperiencingourselvestobedepletedorfatigued(Botvinick&Braver2015,Hockey2011,Kurzbanetal.2013,Westbrook&Braver2015).

Giventhisseeminglyimmediateavailabilitytointrospection,mentaleffortissurprisinglydifficulttopindownasanobjectofscientificstudy.Whatexactlyismentaleffort,fromanobjectiveratherthanintrospectivepointofview?Whatexactlyisgoingonwhenwetryharderonacognitivetaskordecidethatthistryingisnotworthit?Whatisbeingconservedwhenweconserveourcognitiveresources,andhowdowedecidethemannerinwhichthoseresourcesgetallocated?Andhowcanweidentifytheneuralmechanismsunderlyingsuchasubjectiveconstruct?Ouraiminthepresentarticleistoreviewsomeareasofrecentprogressinaddressingthesequestions.

Toconvertmentaleffortintoanapproachableobjectofscientificstudy,ausefulfirststepistooperationalizeitnotinpurelysubjectiveconativetermsbutinsteadintermsofinformationprocessing.Drawingonpreviouswork(Bonner&Sprinkle2002,Camerer&Hogarth1999,Hockey1997,Kahneman1973),weadoptthefollowingworkingdefinition:Effortiswhatmediatesbetween(a)thecharacteristicsofatargettaskandthesubject’savailableinformation-processingcapacityand(b)thefidelityoftheinformation-processingoperationsactuallyperformed,asreflectedintaskperformance.Thefirsttwofactors,taskcharacteristicsandcapacity,determinewhatlevelofperformanceisattainableinprinciple.Effortreferstothesetofinterveningprocessesthatdeterminewhatlevelofperformancewillinfactberealized;thequalityofthisperformanceisquantifiedthroughsuchmeasuresasresponselatencyandaccuracy.Drawingonthefamiliaranalogybetweenmentalandphysicaleffort,wecansaythattaskcharacteristicsandinformation-processingcapacityareanalogoustotheweightofanobjectandthephysicalstrengthofapersontryingtoliftit,andthattaskperformanceisanalogoustotheswiftnessofthelift.Effort,then,isthethingthatmediatesbetweenweightandstrength,ontheonehand,andtheactualliftoutcomeontheother.

3

Inthecaseofmentaleffort,theheavyliftaccomplishedthrougheffortcantakeavarietyofforms,manyofwhichwediscussfurtherbelow:theoverridingofdefaultactionsorhabits(Miller&Cohen2001),engagementincomplicatedmentalfeatssuchasreasoning(Kahneman2003),workingmemorymaintenance(Braver2012),andswitchingbetweentaskswithdifferentkindsofdemands(Monsell2003).Andthecumulativeeffectofsuchliftingcandetermineimportantlifeoutcomes,includingacademicsuccess,socialcompetence,andabilitytocopewithenvironmentalstressors(Caseyetal.2011,Duckworthetal.2007,Mischeletal.1989,Tangneyetal.2004).

However,thesedownstreameffectsdonotimmediatelytellushoweffortdoesitswork.Whatexactlyisthenatureofthismediator?Ifphysicaleffortregulatestheengagementofmuscles,whatisitthatcognitiveeffortisregulating?

Aplausibleanswertothisquestionhasemergedthroughseveraldecadesofresearchonperformanceincognitivetasks.Thisworkhasshownthatinformationprocessingfallsalongacontinuumofautomaticity(Shiffrin&Schneider1977),withsomeprocesses(typicallyheavilypracticedones)abletobedeployedmorereflexivelyandwithlessthreatofinterferencefromotherongoingthoughts.Processesontheotherendofthiscontinuumaresaidtorequireincreasingcommitmentsofcognitivecontroltoreconfigureinformationprocessingawayfromdefault(i.e.,moreautomatic)settings(Botvinick&Cohen2015,Cohenetal.1990).Thenotionofeffortwasinfactcentraltotheearliestcharacterizationsofautomaticprocessingandcontrol-dependentprocessing,withtheformerdescribedaseasyandeffortless,andthelatteraseffortful.Thus,cognitivecontrolmaybeviewedastheforcethroughwhichcognitiveeffortisexerted.

Althoughthispointhelpstofirmupadefinitionforcognitiveeffort,itleavesopenonemorecriticalissue.Ifcognitiveeffortregulatesthedegreetowhichcognitivecontrolisengaged,howisthetargetlevelofcontrolchosen?Thenotionofeffortimpliesadecisionproblem:Howmuch(andwhatform)ofcontrolshouldbeallocated,givencurrentcircumstances?

Thissetofquestionsprovidesthecentralfocusofourpresentreview.Inparticular,weexploreresearchaimedatunderstandingmentaleffortasadomainofdecisionmaking,focusinginparticularonrecentapproachesthatidentifycognitiveeffortastheoutputofreward-basedchoice.Accordingtosuchapproaches,individualsweighthebenefitsofcognitivecontrolagainstsomeinherentcost,thenatureofwhichwediscussnext.Thecoreofthisreviewfocusesoncomputationalcognitiveapproachestounderstandingthiscost-benefitanalysis,withtheaimofprovidingaframeworkforinvestigatingassociatedphenomenologyandunderlyingneuralsubstrates.Weconcludebysummarizingthosesubstratesandconsideringhowthesehelpexplainthesubjectiveexperienceofeffortandhowthiscanbequantified.

4

2.WHYISCONTROLCOSTLY?

Ourworkingdefinitionofmentaleffortexposesanintriguingriddle:Whyshouldtherebeanymediatingfactorbetweencognitivecapacity,ontheonehand,andperformanceontheother?Why,inotherwords,don’tpeoplealwayssimplyperformatthehighestlevelofwhichtheyarecapable?Theintuitiveanswersuggestedbyintrospectionisthatweareconstitutivelyreluctanttomobilizeallavailablecognitiveresources.Thatis,mentaleffortisinherentlyaversiveorcostly.Inadditiontoaccountingforitsphenomenology,theideathatcontroliscostlyhelpstoexplainwhyincentivesarefoundregularlytoimprovecognitiveperformance,suggestingthatindividualscanincreasetheircontrolallocationwhenhigherincentivesareonoffer(i.e.,theyarenotconstrainedbyability)butholdbackfromdoingso,owingtotheaversivenessoftheeffortrequired(Botvinick&Braver2015).Forexample,participantsrespondfasterandmoreaccuratelywhenexpectinggreaterrewardfornamingthecolorofaStroopstimulus(e.g.,whenthewordGREENissetinredtype)(Krebsetal.2010).Similareffectshavebeenfoundwithtaskdemandsincludingselectiveattention(Engelmannetal.2009,Padmala&Pessoa2011)andtaskswitching(Aartsetal.2010,Umemoto&Holroyd2014).Evenperformanceonintelligencetests,traditionallyassumedtobeoneofthepurestmeasuresofcognitiveability,isaffectedbyincentivelevels(Duckworthetal.2011).

Additionalevidenceforthepresenceofcognitiveeffortcostscomesfromworkonthedemand-selectiontask(DST).IntheDST,participantsfacearecurringchoicebetweentwooptions,eachassociatedwithdifferentlevelsofdemandforcognitiveeffort(e.g.,higherversuslowerfrequenciesoftaskswitching).Thekeyfindingfromthissetoftasksisthatparticipantsgenerallypreferthecourseofactionassociatedwiththefewestcognitiveeffortdemands(Dunnetal.2016,Kooletal.2010,McGuire&Botvinick2010).Thisisconsistentwithfindingsthatparticipantsdemandgreaterrewardstoengageintasksthatdemandincreasinginhibitorycontrol(Dixon&Christoff2012)orworkingmemorymaintenance(Westbrooketal.2013).Inotherwords,cognitiveeffortisexperiencedascarryingdisutility(i.e.,assomethingtobediscountedfromtheexpectedreward),anobservationthathasbeenfurthersubstantiatedbyfindingsthatcognitiveeffortevokesnegativeemotions(Dreisbach&Fischer2015,Inzlichtetal.2015,Spuntetal.2012),negativelybiaseslearningofstimulus-rewardassociations(Cavanaghetal.2014),anddiscountsneuralresponsestotherewardpresentedafterapersoncompletesaneffortfultask(Botvinicketal.2009a).

Theseandotherfindingssubstantiatetheintuitionthatcontrolisregisteredascostlyandlaythegroundworkfortreatingcontrolcostsasacentralexplanatoryvariableintheoriesofcontrolallocation.However,beforeexaminingthesetheoriesingreaterdepth,itisworthfirstconsideringwhyafunctionsoseeminglyimportantascognitivecontrolmightbeencodedascostlyatall.

Therearetwobroadcategoriesofexplanationthathavebeenofferedinresponsetothisquestion,thatareinfactcloselyrelatedtooneanother:intrinsiccosts,andopportunitycosts.Thefirstsuggeststhattheallocationofcontrolitselfcarriesacost,and

5

thusthereisalimitonhowmuchcontrolcanbeallocatedatonetime.Onthisaccount,theinvestmentofcontrolmayregisterasmentaleffortinordertoindexthatcost,andensureaproperevaluationoftheworthofthatinvestment.Thesecondtypeofexplanationfollowsnaturallyfromthefirst:giventhatthecapacityforcontrol-dependentprocessingislimitedbyitscost,thenallocatingcontroltoonesetofprocessesmeansforgoingpursuitofothersthatmayalsohavevalue.Onthisaccount,thedurationofacontrol-demandingprocessposesanopportunitycost(Kurzbanetal.2013),thatmayalsoregisterasmentaleffort.Inthesectionsthatfollow,weconsiderthesetwotypesofexplanation.

2.1.IntrinsicCostsofControl.Thenotionoflimitedcapacityofcontrolwasadefiningfeatureintheearliestconceptualizationsofcontrolledprocessing(e.g.,Posner&Snyder1975,Shiffrin&Schneider1977).However,itbegsafundamentalquestionthatcontinuestovexresearchinthisarea:Whyisthecapacityforcontrolsolimited?Prominenttheoreticalaccountsofferthreepossibleexplanations:limitedmetabolicresourcesinthebrain,constraintsonthecapacitytomaintaintask-relevantinformation,and/orinterferencethatarisesfromtheuseofsharedrepresentationsformultiplepurposes(Figure1).Ineachofthesecases,thecostofcontrolplaysaroleinprotectingalimitedresource.Whatdistinguishesthetheoriesisthenatureoftheprotectedresourceitself.

Figure1.Schematicsummaryofpossiblecontrollimitations.Differentaccountsofthepotentialsourcesofcontrolcostsareshownforanexamplecaseofadrivertryingtoattendmultiplestreamsofinformation.Resource-basedaccounts(green)proposethatcontrolcostsreflectthelimitationsofacentralmetabolicresourcethatdepleteswithextendeduseofcognitivecontrol.Controlcapacity--basedaccounts(blue)proposethatcontrolcostsreflectanupperboundonthe

Constraintsduetointerferenceintask

processingpathways

Constraintsduetolimitedcapacityofcontrolsystem

Constraintsduetometabolicresourcesdepletedbycontrolledprocessing

Glucose,glycogen

Taskperformance

Localprocessing

Cogni/vecontrol

Environmentals8muli

6

controlsignalsthatcanbedeployedorcontrol-relevantinformationthatcanbestoredatanygiventime.Interference-basedaccounts(red)proposethatcontrolcostsreflecttheriskstoperformanceassociatedwiththeover-allocationofcontrol.Theserisksresultfromcrosstalkthatcanarisefromoverlapamongthepathwaysrequiredtoprocesstask-relevantstimuli,crosstalkthatitistheverypurposeofcontroltoavoid.Assuggestedbythevisual,theseaccountsarenotmutuallyexclusiveofoneanother.

2.1.1.MetabolicConstraints

Byanalogytotheexertionofphysicaleffort,cognitiveefforthasbeenlinkedhypotheticallytoalimitedphysiologicalresourcethatdepleteswithuse,muchlikeamuscledepletesenergy(oracquirestoxicbyproducts)asittranslatescontractionintoforce(Baumeister&Heatherton1996,Baumeisteretal.1998,Muravenetal.1998).Accordingtothisidea,exertionofcognitiveeffortislimitedbyaresourcethatdepletesinproportiontotheamountanddurationofexertionandthereforeencouragesindividualstojudiciouslyallocatetheiravailablereserves(Figure1,green).Forinstance,anindividualthatengagesinacontrol-demandingtaskoveranextendedperiodoftimewouldfindherselfimpairedatasubsequenttaskthatrequirescontrolorself-regulation(e.g.,choosingahealthymeal).Aspredictedbythisresource-basedaccount,initialevidencesuggestedthatexperimentalparticipantsindeedexertlesscognitiveeffortontasksthatfollowsomeamountofeffortexertion(i.e.,post-depletion)relativetotasksthatfollowminimaleffortexertion(Haggeretal.2010).

However,thisaccountofcontrolcostsraisestwoquestionsthatremainunderdebate.First,whatistheresourcebeingdepleted?Researchershypothesizedinitiallythatitmaybebloodglucose,showingforinstancethatparticipantswerelessdepletedwhenadministeredasweeteneddrink(Gailliotetal.2007).However,subsequentexperimentsprovidedstrongevidenceagainstthishypothesis(reviewedinKurzbanetal.2013;see,e.g.,Moldenetal.2012,Vadilloetal.2016)andsuggestedthat,totheextentglucoseimprovescontrol,itdoessothroughanincreaseinmotivationtoperformthesubsequenttaskratherthanareplenishmentofaphysiologicalresource(Hockey2011,Inzlicht&Schmeichel2012).Moreover,thebrain’sglucoseutilizationisheavilyweightedtowardprocessesthatdonotdepleteinthismanner(e.g.,vision)andthemarginalincreaseinconsumptionforcontrol-demandingtasks–atleastsomeofwhichseemfarlesscomputationallydemanding(suchastwo-digitarithmeticvs.recognizingaface)—isestimatedtoberelativelysmall(Kurzbanetal.2013).

Somehavesuggestedthattheseconcernsaboutglucosecanbeavoidedbyalternateresourcemobilizationaccountsthataremoresensitivetocontrolledprocessing,focusingforinstanceonanindividual’seffortstomaximizeastrocyticglycogen(astoredformofglucose)(Christie&Schrater2015)ortominimizebuildupoftheneurotoxinamyloid-βintheinterstitialfluid(Holroyd2015),buttheseproposalshaveyettobetestedempirically.Alltheseaccountsmustalsoaddressasecondopenquestionpertainingtothetimescaleoverwhichputative

7

resourcesdeplete.Classicfindingshavesuggesteddepletionofcognitivecontroloccursoverrelativelyshortperiodsoftime(i.e.,followinglessthananhourofmentaleffortexertion).However,recentmeta-analysesandreplicationattemptshavecalledsuchfindingsintoquestion(Carteretal.2015,Haggeretal.2016).Collectively,thesesuggestthat,totheextentcontroldepletesorfatigues,itmaydosoonlyoverlongertimescales(Blainetal.2016).Wereturntopotentialsourcesoftheselongertimescalecontrolcostslater(e.g.,boredom),butfornowsimplyreiteratethattheresourcemobilizationaccountssuggestthatthesecostsareoperativeatshortertimescalesaswell(e.g.,individualtrialsofatask).

2.1.2.StructuralCapacity:LimitationsonStorageandMaintenance

Asecondlineofresource-basedcontrolcostaccountssuggeststhatcontrolcostsmayarisefromcomputationallimitationsinthecapacityforcontrolledprocessing,ratherthanthedepletionofanykindofmetabolicresource.

Thetraditional,andstilldominant,accountexplainsthisbymakingtwokeyassumptions:(a)Controlreliesonacentralizedmechanism,and(b)assuggestedabove,thecapacityofthissystemislimited.Theseassumptionsaretypicallyjustifiedbyarguingthatcontrolisdependentonworkingmemorytorepresentthecontextinformation(e.g.,instructions,intentions,taskconditions,goals)usedbythecontrolsystemtoguidebehavior(e.g.,Anderson1983,Cohenetal.1990).This,inturn,linkstheconstraintsoncontroltothewell-knownlimitationsofworkingmemorycapacity(e.g.,Cowan2012,Luck&Vogel1997,Miller1956)(Figure1,blue).Thus,thecapacitylimitationsofcognitivecontrolcanbetracedtothefactorsthatlimitworkingmemorycapacity,ofwhichseveralhavebeenproposed:aresourcelimitationinactivelymaintainedworkingmemoryrepresentations,intermsofdiscreteslots(Cowanetal.2012,Luck&Vogel1997)orcontinuousresources(Maetal.2014);interferencebetweentherepresentationsheldinworkingmemory(Nairne1990,Oberauer&Kliegl2006);and/orpassivedecay(Jensen1988,Page&Norris1998)(foracomparativereviewoftheseaccounts,seeOberaueretal.2016).

However,theseexplanationshaveyettobejustifiedadequatelyintermsoftheunderlyingmechanisms(e.g.,theneuralmechanismsinvolved)and,wherethishasbeenattempted(e.g.,Elmoreetal.2011,Maetal.2014),thefocushasbeenonsimpleformsofshort-termmemory(e.g.,visualmemory)andnotonsystemsmoredirectlyinvolvedincognitivecontrol.Moregenerally,itseemsoddtoimaginethatthecontrolsystem---onesocriticaltoadaptivebehaviorandwithaccesstosuchvastresources(therearebillionsofneuronsinthehumanprefrontalcortexalone)---wouldbesubjecttosuchastultifyinglimitation:theinability,inmanyinstances,tocarryoutmorethanasinglecontrol-dependenttaskatatime.Evolutionanddevelopmentwouldhavetoberatherpoorengineersindeedtoarriveatthissolution,ifstructuralresourcesavailabletothecontrolmechanism(s)weretheonlyconsiderationinvolved.(Foranormativeconsiderationoftheconstraintsonworkingmemoryitself,seeElman1993,Toddetal.2009.)

8

2.1.3.RepresentationalCapacity:LimitationsArisingfromInformation-ProcessingPathways

Analternativetoanaccountattributingcapacitylimitstothecontrolsystemitselfwassuggestedbyearlyattentiontheoristsandreferredtoasthemultipleresourceshypothesis(Navon&Gopher1979;seealsoAllport1980,Allportetal.1972,Logan1985,Wickens1984).Thisproposedthatrestrictionsincontrol-dependentbehaviorreflectcrosstalkarisingfromlocalbottlenecksinprocessing,whendifferenttaskscompetetousethesamesetofrepresentationsorapparatusfordifferentpurposes.Asatrivialexample,evenwithlimitlesscapabilityforcontrol,itwouldbeimpossibletosaythewordsgreenandredatthesametimebecausewehaveonlyonesetofvocalchords,onemouth.However,theconstraintsneedn’tariseonlyineffectorsystems(e.g.,theremaybeonlyasinglephonologicalsystemthatdrivesmultitaskingconstraintsonspeech),andtheconstraintsontheseeffectorsclearlycan’texplainmorecommonexamplesofmulti-taskingfailures,suchasourinabilitytocarryouttwomentalarithmeticproblemsatthesametime(seeShaffer1975foramoreinterestingexampleandclassicexperimentaldemonstrationoftheproblemposedbycross-talkinvolvinginternalrepresentations).Fromthisperspective,restrictionsoncontrol-dependentprocessingreflecttheverypurposeofcontrol---tolimitthedeleteriouseffectsofcrosstalkintheprocessingsystemoverwhichcontrolpresides(Figure1,red)---ratherthananintrinsiclimitationofthecontrolsystemitself.

Thisaccountofcapacityconstraintsincontrol-dependentprocessing---intermsofrepresentationandcomputationalpropertiesoftheprocessingsystem,ratherthanstructuralpropertiesofthecontrolsystem---givesrisetoanotherpairofquestions,bothofwhichhavebeenaddressedbyrecentcomputationalwork.Thefirstquestioniswhethercollisionsinprocessingthatgiverisetocrosstalkarereallyaseriousprobleminasystemaslargeasthebrain.Simulationstudies(Fengetal.2014),followedbyrecentanalyticwork(Musslicketal.2016b),indicatethatevenmodestamountsofoverlapamongprocessingpathwayscanimposedramaticandnearlyscale-invariantconstraintsonhowmanyprocessescanbeexecutedatonetime.Suchconstraintsonparallelprocessinghavebeenshowntoholdevenincasesinwhichmultitaskingisexecutedasrapidsequentialswitchesratherthanperformancethatisstrictlyparallel(Musslicketal.2016a),undertheassumptionthatsequentialtasksbleedintooneanother(referredtoastasksetinertia)(Allportetal.1994,Allport&Wylie1999).Thus,itisatleastplausiblethateveninaverylargenetwork,pathwayoverlap(i.e.,theshareduseofrepresentationsbydifferentprocesses)quicklyproducesbottlenecksthatdemandmanagementbytheinterventionofacontrolsystem,andthattheselocalbottlenecks,ratherthantheconstraintsonacentralizedcontrolmechanism,mayexplainlimitsinthecapacityforcontrolledprocessing.Inotherwords,ourlimitedcapacityforcontrolledprocessingmayreflectthepurposeofcontrolratherthanaconstraintonitsabilitytooperate.

Thedeleteriousimpactofpathwayoverlaponprocessingraisesasecondquestion:Ifthebottleneckstheycreatearesoproblematic,whynotavertthisproblembydiminishingtheshareduseofrepresentations?Insightgainedfromthe

9

studyoflearningandrepresentationinneuralnetworksprovidesadirectanswertothisquestion:Sharedrepresentationsupportsinferenceandgeneralizationandiscriticaltothediscoveryanduseofabstractstructure.Thisinsightdrovetheconnectionistrevolutioninpsychologyinthe1980s(Rumelhart&McClelland1986)andisdrivingthecurrentexplosionofinterestindeeplearningnetworkswithinthemachinelearningcommunity(Bengioetal.2013,Caruana1998,LeCunetal.2015).

Thus,theuseofsharedrepresentationimposesatrade-offbetweenitsvalueforlearningandabstractionononehandandtheconstraintsitimposesonthesimultaneousexecutionofmultipleprocessesontheother.Musslicketal.(2016a)haveexploredthistrade-offdirectly.Theyhaveshownthatwhennetworksaretrainedtoperformavarietyoftasks,thereisastrongbiastowardtheemergenceofrepresentationsthataresharedacrosstaskswithsimilarrequirements,andforcontrolrepresentationstodevelopthatdisambiguatethesharedrepresentationsappropriately,accordingtotaskcontext.Furthermore,thisbiasconsiderablyfacilitateslearninginlargetaskspaces.However,thiscomesatthecostofseverelydegradedperformanceifanyofthetasksinvolvedmustbeperformedconcurrently.Additionaltrainingcanovercomethislimitationbyseparatingtherepresentationsforthedifferenttasks,whichalsodiminishestheirrelianceoncontrol(Garner2015).Theseobservationsconcurwithavast,longstandingcognitivepsychologicalliteratureonthetrajectoryfromcontrolledtoautomaticprocessingduringskillacquisition(Cohenetal.1990,Graybiel2008,Shiffrin&Schneider1977).

Musslicketal.(2016a)describetheirobservationsintermsofafundamentalcontinuumofcomputationalarchitectures,withthoseatoneendthatmakeuseofindependent(sometimesreferredtoasembarrassing)parallelismtosupportconcurrentmultitasking,andarchitecturesattheotherendthatexploitsharedrepresentationstosupportinteractiveparallelismintheserviceofabstractinferenceandefficientlearning.Fromtheperspectiveofsuchacontinuum,thecapacityconstraintsincontrolledprocessingreflectthebrain’schoice---inthosesituationsthatdemandrapidlearning,theflexibilityaffordedbyabstractinferenceandgeneralization,orboth---toexploitthevalueofsharedrepresentation,atthecostoflimitsonconcurrenttaskexecution.Assuggestedabove,mentaleffortcanthenbeviewedasanexplicitindicator(computationalandsubjective)ofthiscost---thatis,thecostassociatedwithsituationsinvolvingprocessingconfigurationsthatdemandtheengagementofcontroltoavertcrosstalk.

2.2OpportunityCostsofControlAlthoughunderstandingthesourceoftheconstraintsoncontrolled-processingremainsanimportantpriorityforresearch,theconstraintitselfsufficestoimposeacloselyrelatedcost:thatoftime.Engagingthecontrolsystemintheserviceofonecontrol-demandingtaskmeansforgoingothersthatcouldhavebeenperformedoverthatsameperiod.Thus,inadditiontoindexingthedegreeofinvestmentincontrolitself,mentaleffortmayalso

10

reflecttheopportunitycostofthatinvestment,asawayofensuringthattheyarerespectedwhenmakingdecisionsabouthowtoallocatecontrol(Kurzbanetal.2013).

Theideathatmentaleffortreflectstheopportunitycostsassociatedwithallocatingavaluablebutlimitedresource---thecapacityforcontrol---isintuitivelyappealing.Italsosuggeststhattheperceivedeffortmayscalewiththedurationoftheinvestment.Thisideaaccordswellwithworkonhowpeopleassessthevalue(andcost)ofcomputation,andhowthisinfluencestheirdecisionsaboutstrategiestopursueinproblemsolvingandotherbehaviors—alineofworkthathasanintimaterelationshiptothecostofcontrol.Wediscussthisindetailinthesectionthatfollows.

3.HOWSHOULDWEALLOCATECOGNITIVEEFFORT?

Asdiscussedabove,thebrain’sinherentcapacitylimitations---irrespectiveoftheircause---provideabasisforunderstandingmentaleffortcosts:Ifaresourceisvaluablebutlimited,itshouldbeconservedwheneverpossiblesothatitcanbeusedjudiciously.Thatis,cognitiveeffortshouldbeexpendedtotheextentitisworthit.Thisideaiscentraltoadecades-oldliteratureontheoriesofboundedrationality,whichemphasizethathumancognitionhastomakedowithlimitedinformation,littletime,andboundedcognitiveresources,andhasbeenfurtherdevelopedintotheoriesofboundedoptimality,whichspecifytheoptimalwaytousetheselimitedresources(seethesidebartitledBoundedRationalityandBoundedOptimality).

Recentaccountshavebuiltuponthistheoreticalgroundingtoproposeseveralwaysinwhichcomplexcognitiveprocessescanbeselectedsoastomaximizerewardswhileminimizingthecostsassociatedwithmentaleffort.Inthissection,wefocusontwocomplementaryoptimizationapproachesthathavebeendevelopedinparallelwithinresearchonstrategyandalgorithmselectionandwithinresearchoncognitivecontrol.

BOUNDEDRATIONALITYANDBOUNDEDOPTIMALITY

People’sjudgmentsanddecisionssystematicallyviolatethenormativeprinciplesoflogic,probabilitytheory,andexpectedutilitytheory(Tversky&Kahneman1974).Theoriesofboundedrationality(Simon1956,1982;Gigerenzer2008,Simon1982,Todd&Gigerenzer2012)attributethesesuboptimalitiestopeople’srelianceonsimplifyingheuristicsinlieuofmoreexhaustivedeliberation.Thesetheoriesarguedthatsuchcognitivefrugalityreflectspeople’smentallimitationsaswellastheirlimitedtimeandinformation.

Whereasearlypsychologicaltheoriesofboundedrationalitywerelargelyqualitative,researchinartificialintelligencesubsequentlydevelopedamathematicallyprecise,normativetheoryofhowboundedagentsshouldallocatetheircomputationalresources.Accordingtothetheoryofboundedoptimality(Russell&Subramanian1995),theobjectiveofrationalinformationprocessingistomaximizetheagent’sexpectedrewardperunittimeoverthelongterm,subjecttotheconstraintsoftheagent’sperformance-limitedhardware.

11

Thenotionofboundedoptimalityhasinspiredcomputationalmodelsofhumancognition,accordingtowhichthebrainmakesoptimaluseofitsfiniteresources(Gershmanetal.2015,Griffithsetal.2015,Lewisetal.2014).Fromthisperspective,limitationsinthebrain’scapacityforparallelprocessing(Fengetal.2014,Musslicketal.2016b)anditsfiniteinformation-processingspeedgiverisetoopportunitycostsforeachcognitiveoperation:Committingtooneorasetofcomputationsprecludesthesimultaneousexecutionofothervaluableones(cf.Kurzbanetal.2013).Asaresult,peoplehavetotradeofftheexpectedqualityoftheirchosencomputationsagainstthenumberofdecisionsthatcanbemadeperunittime,andinmanycasestheoptimalamountofdeliberationforeachindividualdecisionissurprisinglylow(Vuletal.2014).Thelimitedeffortthatpeopleinvestincertaindecisionsandjudgmentsmayreflectthisoptimalityprinciple(Liederetal.2012,F.Lieder,T.L.Griffiths,Q.J.M.Huys&N.D.Goodman,submittedmanuscript,Vuletal.2014;butseealsoOudetal.2016).

3.1.TheValueofComputation

Toachieveboundedoptimality,peoplemayperformacost-benefitanalysistoselectthecognitivestrategywiththebesttradeoffbetweeneffortandaccuracy(Beach&Mitchell1978,Payneetal.1988).Researchinartificialintelligencehasprovidedamathematicallyprecisedefinitionofwhatconstitutesthisoptimaltradeoffforagivensetofcomputeralgorithms(i.e.,sequencesofcomputations)(Russell&Wefald,1991).Inbrief,thebestalgorithmshouldmaximizethevalueofcomputation(VOC),whichisdefinedastheexpectedutilitygainedbyengagingthosecomputationsminustheexpectedcostofthecomputationalresourcesitwillconsume(e.g.,CPUcyclesandmemory).Suchmethodsfordecidinghowtoallocatecomputationalresources(termed‘rationalmetareasoning’)havebeendevelopedtoenableintelligentsystemstointeractwiththeirenvironmentinrealtimetomakeoptimaluseoftheirlimitedtimeandfinitecomputationalresourcesbyselectingtheircomputationsadaptively(Hay,etal.,2012).

Liederandcolleagueshaverecentlyappliedthisoptimalityprincipletocognition,proposingthatindividualsmightsimilarlyselectcognitivemechanisms(e.g.,decisionstrategies)basedontheirrelativeVOC(Griffithsetal.2015,Liederetal.2012,Lieder,Plunkett,etal.,2014;Lieder&Griffiths,2015).Underthisformulation,peopleshouldrationallytradeoffthequalityofaselectedcognitivestrategyagainstthecostofthecomputationsitentails(Liederetal.2014).Specifically,mentaleffortshouldbeallocatedtoachievetheoptimaltrade-offbetweentheexpectedutilityofitsoutcomeandtheopportunitycostoftherequiredtime.Thissimpleprinciplecanaccountfortheadaptiveflexibilitywithwhichpeopleswitchbetweendifferentcognitivemechanismsdependingontheproblemtobesolved(F.Lieder&T.L.Griffiths,submittedmanuscript).

Theseresearchershaveproposedthatpeoplelearntoselectcognitivemechanismsinawaythatapproximatesrationalmetareasoningefficiently(Lieder&

12

Griffiths2015,Liederetal.2014);thatis,theylearntopredicttheVOCofcandidatecognitiveoperationsfromfeaturesoftheproblemtobesolvedandselectthesequenceofoperationswiththehighestpredictedVOC(seeFigure2a).Suchafeature-basedlearningmechanismcanaccountforadaptivechangesinstrategyselectionacrossabroadarrayoftaskdomains,includingdecisionmaking,problemsolving,andmentalarithmetic(Lieder&Griffiths2015,F.Lieder&T.L.Griffiths,submittedmanuscript).Forexample,thisaccountwasabletopredictpeople’sadaptivechoicesofsortingstrategywhenfacedwithanout-of-orderlistmuchmoreaccuratelythanpreviousmodelsofstrategyselection(Figure2b)(Liederetal.2014).Overall,thesefindingssuggestthatwhenselectingbetweensequencesofcognitiveoperations,peopleinvestmentaleffortrationallywithrespecttotheirmentalmodelofthe(time)costsandrewardsforpotentialstrategies.

Figure2.(a)Thevalueofcomputation(VOC)modellearnshowfeaturesoftheenvironment(e.g.,thelengthandsortednessofalistofnumbers)predicttheeffectivenessandefficiencyofcandidatecomputations(e.g.,strategiesforsortingthatlist),basedontheirexpectedrewardandtimecost(thedifferencebetweenwhichdeterminestheirVOC).(b)TheVOCmodelperformed

13

substantiallybetteratpredictingactualchoicesofsortingstrategythanpreviousmodelsofstrategyselection(namedStrategyChoiceAndDiscoverySimulation[SCADS],StrategySelectionLearning[SSL],andReinforcementLearningfromCognitiveStrategies[RELACS]).AdaptedfromLiederetal.(2014).(c)Accordingtotheexpectedvalueofcontrol(EVC)theory,agivencognitivecontrolsignalsetting(consistingofbothitsidentityandintensity)determinestheexpectedpayoffsandcostsforcontrol.Theoptimalcontrolsignalsettingsmaximizethedifferencebetweenthesetwoquantities,ortheEVC.AdaptedwithpermissionfromShenhavetal.(2013).(d)EVCmodelsimulationscapturetheimprovementsinbehavioralperformance(higheraccuracyandfasterresponses)observedwhenvaryingtheincentivesforapicture-wordStrooptask(Musslicketal.2015,Padmala&Pessoa2011).Participantsperformingthistaskwereinstructedtoindicatewhethertheimagedisplayedabuildingorhousewhileignoringthecontentoftheoverlaidtext.AdaptedwithpermissionfromPadmala&Pessoa(2011)andMusslicketal.(2015).

3.2.TheExpectedValueofControl

TheVOCmodeldescribeshowindividualsselectbetweensetsofsequentialoperationsbasedontheexpectedrewardaswellasthecostsassociatedwiththetimerequired.However,asnotedabove,cognitivecontrolisgenerallyconceivedasfallingalongacontinuum;onecanapplyvaryingdegreesofcontrol(e.g.,attention)tothetaskathand,withconcomitantchangesinperformance.Moreover,peopleexperiencehigherlevelsorintensitiesofcontrolexertionasmorecostlyoraversive,independentlyoftheassociatedtimecosts(Dixon&Christoff2012,Kooletal.2010,Westbrooketal.2013).Thus,acost-benefitanalysisisnecessarytodeterminenotonlywhattypesofcontrolledprocessesareworthinvestinginbutalsohowmuchcontrolisworthinvestingineach,basedonthereturnsexpectedforagivenlevelofcontrol.Inspiredbyreinforcementlearning(RL)modelsofactionselectionandmotorcontrol(Sutton&Barto1998,Wolpert&Landy2012),Shenhavandcolleagues(2013)recentlydevelopedatheorythatformalizesthisideaofacost-benefitanalysisformaximizingtheexpectedvalueofcontrol(EVC).

TheEVCtheoryproposesthatcontrolsignalsarespecifiedalongtwodimensions:anidentity(e.g.,whattoattend,suchasthecolorofaStroopstimulus)andanintensity(e.g.,howstronglytoattend,relativetoadefaultorautomaticlevel).Adjustingtheintensitiesofcontrolsignalsshouldinfluencethelikelihoodofobtainingreward,avoidingpunishment,orboth(e.g.,basedonprovidingacorrectversuserroneousresponse),aswellastheefficiencyofdoingso(e.g.,howlongittakestorespond).Collectively,thesefactorsdefinetheoverallrateofrewardreceipt(rewardperunittime),whichisakeyvariableanimalsseektomaximize(Bogaczetal.2006,Nivetal.2007).Inadditiontodiscountingthevalueofcontrolbythetimespentonthetask,thetheoryimportantlyalsoassumesanintrinsiccostordisutilityassociatedwithincreasingcontrolintensity.TheEVCisdefinedasthedifferencebetweentheexpectedgains(e.g.,rewardrate)andtheexpectedcontrolcostassociatedwithagivenconfigurationofcontrolsignals(Figure2c).OptimalcontrolallocationcanbeachievedbyselectingthecontrolsignalconfigurationthatmaximizesEVCinthecurrentsituation.Recently,acomputationalimplementationoftheEVCtheoryhasbeenshowntoaccountforavarietyofphenomena

14

associatedwiththeallocationofcontrol(Musslicketal.2015),includingsequentialadaptationeffectssuchaspost-errorslowingandincentive-drivenimprovementinperformanceoninhibitorycontroltasks(Figure2d).

EVCisoneofafamilyofrecenttheoriesthathaveappliedRLapproachestotheallocationofcontrol(Frank&Badre2012,Holroyd&McClure2015,O'Reilly&Frank2006,Toddetal.2009,Vergutsetal.2015).Forinstance,onerecentmodelusestemporaldifferencelearningtoestimatethevalueandeffortcostsofbothcognitiveandmotoractions(Vergutsetal.2015)andcombinesthetwoestimatestodeterminewhetherornottoincreasethegain(signal-to-noiseratio)ofeitherformofactionselection.BuildingonahierarchicalextensionofRL(HRL)---wherebyanagentcanlearnthevalueofindividualactionsaswellastemporallyextendedsequences(Botvinicketal.2009b)---Holroydandcolleagues(Holroyd&McClure2015,Holroyd&Yeung2012)haveproposedanothermodel,accordingtowhichcontrolmaybeusedtomitigatetheshort-termcostofphysicallyeffortfulobstaclesinfavoroflonger-termrewardsthatwillbeobtainedaftercompletingthecurrentsequenceofactions.Inotherwords,theHRLmodelselectshigh-levelactions(calledoptions)tomaximizelong-termvalue(e.g.,walkacrosscampustomeetafriend),andcontrolprotectsthoseoptionsfrombeingdelayedoroverturnedinthefaceofeffortfulobstaclesalongtheway.OthermodelshaveaddressedthecontrolofbehaviorbyusingRLasthebasisforselectingtheinformationthatisallowed(gated)intoworkingmemory(Alexander&Brown2015,Frank&Badre2012,O'Reilly&Frank2006,Toddetal.2009),enablingthemaintenanceofappropriatehigher-levelgoalsandtheassociatedmobilizationofcontrolmechanismsgivenfeedbackfromtheenvironment.

Theseaccountsdrawuponsimilarbasicprinciplesoflearningandactionselectionandarethereforepotentiallycomplementary,addressingdifferentcomponentmechanisms,levelsofmechanisticdetail,orboth.AdistinctiveelementoftheEVCtheoryisitsfocusonthecostofcontrolasafactorintheselectionprocess.ThisallowstheEVCtheorytonotonlymakecontactwithbroaderresearchintorationalmetareasoninganditshistoricalpredecessors,butalsotomakedetailedcontactwiththelarge,butheretoforeratherqualitative,literaturelinkingeffortwithincentivemotivation(reviewedinBotvinick&Braver2015).

3.3.ValueofComputationandExpectedValueofControl

Inadditiontodifferentlyemphasizingdifferentkindsofcontrolcosts(i.e.,thosethatscalewithtimealoneorinadditiontothosethatscalewithlevelofcontrolengagement),theVOCandEVCframeworksalsomaximizethevalueofdifferentkindsofpotentialoutputs.TheVOCframework,withitsoriginsintheartificialintelligenceliterature,framesthecentralcost-benefitanalysisunderlyingeffortintermsofcomputation.TheEVCframework,incontrast,framesitintermsofcognitivecontrol.Thismayappeartobeadiscrepancybetweenthetwotheories.However,analignmentcanbeestablishedbyconsideringtheamountofcomputationrequiredforcontrolledversusautomatic

15

processing.Incomparisonwithautomaticprocessing,controlledprocessingrequiresaricherrepresentationoftaskortemporalcontext.Automaticprocessing,bydefinition,yieldsdefaultbehaviors;tooverridethese,theprocessingsystemmustinstantiatearepresentationofcontext(suchas,forexample,arepresentationofrecentlyreceivedverbalinstructions).Settingupthisrepresentationandderivingappropriateactionsfromitrequirescomputation,andstillmorecomputationisrequirediftheappropriatecontextrepresentationmustbediscoveredthroughsearchordeliberation.Thishighlightsthattherepresentationalcostofcognitivecontroliscloselyrelatedtothecostofcomputation.RecentresearchbyOrtega&Braun(2011,2013)formalizesthispoint,relatingittothelargerthemeofboundedrationality.

4.WHATARETHENEURALMECHANISMSFORTRACKINGCONTROLCOSTSANDALLOCATINGCONTROL?

Thetheoreticalperspectiveswehavereviewedaboveprovideaformallyrigorousframeworkforexaminingtheneuralsubstratesunderlyingmentaleffortallocation.Thesetheoriessuggestthatneuralcircuitsmediatingcontrolallocationshouldbesensitivetothepossiblewayscontrolcouldbeallocatedatagiventimeandthepotentialrewardsgained(orpunishmentavoided)byengagingcontrol,andthecostsincurredbytherequisitecontrol.Furthermore,inactivatingthesecircuitsshouldresultinmotivationaldeficits(i.e.,impairedeffortallocation).Thesepredictionshavebeenlargelyborneoutbyresearchintothecircuitryforcognitivecontrolandinparticulartheroleofthedorsalanteriorcingulatecortex(dACC)withinthatcircuit.

4.1.EffortandtheExecutiveNetwork

Abroadnetworkofcorticalstructureshasbeenimplicatedintasksthatrequireanindividualtoexertcognitiveeffort(Dosenbachetal.2008,Duncan2010,Power&Petersen2013,Shenhavetal.2013).ThisincludesdACC,anteriorinsula(AI),lateralprefrontalcortex(lPFC),andlateralparietalcortex.Theregionsinthiscontrol-relatednetworkaremoreengagedwhenanindividualmustperformataskthatdemandssustainedattention,maintenanceofinformationinworkingmemory,and/oroverridingofprepotentresponses;theyarerelativelydisengagedwhenperformingmorehabitualand/orexternallyguidedbehaviors.However,althoughtheinvolvementofthisnetworkinresearchoncognitivecontrolisrobust,thefunctionalrolesoftheconstituentsofthiscircuitareheavilydebated,particularlywithrespecttopotentialrolesindecisionsabouttheallocationofcontrol,ascontrastedwiththeexecutionofcontrol.GiventheparticularrelevanceofdACC,AI,andlPFCtothisquestion,andthequestionofcontrolcostsignaling,wereviewrelevantfindingsconcerningthesecorticalstructuresandtheirinteractionswithsubcorticalstructuresandneuromodulatorysystems.

ResearchershavelongvieweddACC(and,inparticular,anteriormidcingulatecortex)(Shackmanetal.2011,Vogt2016)asplayingaroleindetermininghow

16

physicalactions,cognitiveactions,orbotharedeployedoralteredbasedonavailableevaluativesignals.EarlyviewsfromthecognitivecontrolliteraturesuggestedthatdACCmayeffectthesechangesbycapitalizingonexternalsignalsthatanerrorhadbeencommitted(Holroyd&Coles2002),internalsignalsofconflictbetweenpotentialresponses(Botvinicketal.2001),and/orinternalestimatesofthelikelihoodofcommittinganerrorinagivencontext(Brown&Braver2005).Consistentwiththesesuggestions,dACChasbeenshowntosignaleachofthesequantities(Ridderinkhofetal.2004;Shackmanetal.2011;Shenhavetal.2013,2016;butseeNieuwenhuisetal.2007)aswellasotherpotentialindicatorsofcontroldemandssuchassurprise(Cavanagh&Frank2014,Wesseletal.2012).However,patternsofactivityindACCandbetweenthisregionandotherssuggestabroaderandmorenuancedrolethanonethatsimplyindicateshowmuchcontrolmightbedemandedataparticularpointintime(Heilbronner&Hayden2016,Shenhavetal.2016).Thesepatternssuggestrolesineffortavoidance(Waltonetal.2007),reward-baseddecisionmaking(Rushworthetal.2004,2011),andmotivation(Holroyd&Yeung2012,Stuss2011).TheapparentinvolvementofdACCinthesedifferentfunctionscanbereadilyexplainedthroughthelensofthecontrolallocationmodelsdescribedintheprevioussections.

4.2.AnExpectedValueofControlPerspectiveonDorsalAnteriorCingulateCortex

IthasbeenproposedthatdACCintegratessignalsrelevanttotheEVCandspecifiestodownstreamregionsthetypesandintensitiesofcontrolthatwouldmaximizethisquantity(Shenhavetal.2013,2016).Assuggestedabove,thismonitoringanddecision-makingprocessisinformedbysignalsthatindicatethelikelihoodofacertainoutcome(e.g.,acorrectresponse)givenaparticularallocationofcontrol,thetimerequiredtoobtainthatoutcome,andtherewardorpunishmentassociatedwiththatoutcome.ThisaccountsfordACC’sassociationwithindicatorsofcognitivedemandmentionedabove(e.g.,errors,conflict,surprise),whichcanserveasproxiesforperformancecosts(timeanderrorlikelihood).ItalsoaccountsforfindingsofdACCsignalsindicatingthevaluesofpotentialoutcomes(Heilbronner&Hayden2016,Kapingetal.2011,Kouneiheretal.2009).Moreover,thepredictionthatEVCincorporatestheintrinsiccostofcontrolisconsistentwithfindingsthatdACCtrackshowaversivecontroldemandsaretoanindividual,includingtheirexperiencedfrustration(Spuntetal.2012),theirpreferencesagainstperformingthetask(McGuire&Botvinick2010),andhowmuchtheydevaluerewardsassociatedwiththecognitivelyeffortfultask(Botvinicketal.2009a,Cavanaghetal.2014).Interestingly,dACChasalsobeenfoundtotrackthecostofphysicaleffort(Croxsonetal.2009,Hillman&Bilkey2010,Prévostetal.2010),suggestingitmayplayasuperordinateroleincomputingeffort-sensitivecost-benefitanalysesacrossdomains.

WithinthecontextoftheEVCtheory,dACChasbeenfurtherproposedtooutputsignalsspecifyingboththetypesandamountsofcontroltoallocateinordertomaximizeEVC;thesesignalshavetheeffectoflicensingtheeffortrequiredbytheallocatedcontrol.ThispredictionissupportedbyevidencethatdACCisableto

17

differentiatebetweentypesofcontrolthatareneededinagivensituation(Kapingetal.2011,Shenetal.2014);thatchangesindACCactivityduringmonitoringpredictsubsequentcontroladjustments(e.g.,responseslowing,attentionalshifts)(Kernsetal.2004,Shenhavetal.2016,Ullspergeretal.2014);andthatcausalmanipulationsofdACCcaninfluencetheseadjustments(Reinhart&Woodman2014,Shethetal.2012;seealsoMansourietal.inpress)or,inextremecases,moredrasticallyinfluenceone’swillingnesstoengageincontrol-demandingtasks(i.e.,one’sdecisiontodeemthosetasksworththeeffort)(Holroyd&Yeung2012,Parvizietal.2013,Stuss2011,Waltonetal.2007).Forinstance,dACC-lesionedratsarelesslikelytopursuethegreateroftworewardsifdoingsorequiresovercominganeffortfulobstacle(Waltonetal.2007,Holroyd&McClure2015).

TheproposedroleofdACCinspecifyingEVC-maximizingcontrolsignalsisbroadlyconsistentwithmanyotheraccountsofthisregion’sroleinintegratingrelevantevaluativesignalstohelpguideadaptivebehavior(reviewedinCavanagh&Frank2014;Heilbronner&Hayden2016;Shenhavetal.2013,2016;Ullspergeretal.2014).Forinstance,theHRLmodelbyHolroydandcolleagues(Holroyd&McClure2015,Holroyd&Yeung2012)proposesthatdACClearnsthevalueofextendedsequencesandonthatbasisdetermineswhethertodiscountthecostofeffortfulobstaclesthatpreventlower-levelactionselectionregionsofstriatumfromperseveringoverfutureactionswithinthatsequence.Vergutsandcolleagues(2015)similarlyproposethatdACClearnsthecost-discountedvalueofadjustingthegainonactioninagivencontextandmodifiesprocessingaccordingly.AndAlexander&Brown(2015)proposethatregionsofdACCservetotunepredictionsabouttheerrorsthatwillresultfromfailingtoactappropriately,failingtomaintaintheappropriateinformationinworkingmemory,orboth,leadingtoadaptiveimprovementsindecisionsaboutwhichinformationtogate.AlthoughacomprehensivecomparisonofthesemodelsandtheirabilitytoaccountfordACCfunctionisbeyondthescopeofthecurrentreview,thesetheoriesshowcollectivelythatanappropriatelynuancedmodelofcontrolallocationmayprovideamoreparsimoniousaccountofthevarietyofsignalsthathavebeenobservedinthisregion,withouthavingtopositadditionaluniquefunctionsassociatedwitheachsignal(Shenhavetal.2016).

4.3.ExpectedValueofControlandtheBroaderExecutiveNetwork

Twobrainregionsthatarecommonlyincludedwithinthesameexecutive/controlnetworkasdACCaretheAIandlPFC.Thesehavebeenproposedasinputstoandoutputsofcontrolallocationdecisions,respectively(Bushetal.2000,Caietal.2016,Shackmanetal.2011,Shenhavetal.2013,Ullspergeretal.2014).LikedACC,AIhasbeenshowntorespondphasicallytoabroadarrayofsalienteventsthatmaysignaltheneedtoadaptcontrol(includingrewardsanderrors)(Menon&Uddin2010)andalsodisplayssustainedelevatedresponsesoverthecourseoftaskperformance(possiblyrelatedtothemaintenanceofcontrol)(Dosenbachetal.2006).However,thesetworegions(whichsharereciprocal

18

connections)differintheirrelativepatternsofconnectivitywithsensoryinputsversusmotoroutputs,withAIsharingmoreconnectionswiththeformerthanthelatter(Craig2009).Accordingly,whilesubstantialevidenceofdissociationisstilllacking,basedontheirrelativetimingandpatternsofactivationacrosstheliterature,ithasbeensuggestedthatdACCmayplayamoredirectroleineffectingchangesincontrolallocationwhileAImaysignalsalientstatesthatbearonthosecontrolallocationdecisions(e.g.,internalandexternalsignalsassociatedwitherrorcommission)(Craig2009,Magnoetal.2006,Medford&Critchley2010,Ullspergeretal.2010,2014,Caietal.2016).

ResearchershaveshownlPFCplaysakeyroleintheexecutionorregulationofcertaincontrolpolicies,andtheythereforeargueitistheexecutorofspecifickindsofcontrolsignalsthathavebeenspecifiedbydACC(e.g.,maintainingtasksets)(Botvinicketal.2001,Holroyd&Yeung2012,Ridderinkhofetal.2004,Shenhavetal.2013).Thisproposedfunctionalrelationshiphasbeensupportedbystudiesthatexaminethetimescaleofprocessingacrossthesetworegionsduringcontrolmonitoring,adjustment,andexecution(Kapingetal.2011,Oehrnetal.2014,Tangetal.2016,Womelsdorfetal.2010).LikedACC,lPFChasalsobeenfoundtobeassociatedwithpreferencestoavoidcognitiveeffort(McGuire&Botvinick2010)andwithsignaturesofcognitivefatigue(Blainetal.2016,Tanakaetal.2014).Thismaysuggestaroleinsignalingcontrolcosts,perhapsatamoreabstractorheuristiclevel(cf.McGuire&Botvinick2010)butcouldalsosuggestthatlPFCprovidesamoresensitivereadoutofthelevelofcognitiveeffort(i.e.,controloutput)beingexertedatagiventime(Wangetal.2016).

AlthoughregionsoflPFCappeartobethenodesoftheexecutivenetworkmostcloselyassociatedwiththeregulationofcontrol,itisimportanttonotethatcontrolsignalsareheterogeneousandtheirregulationisthereforeunlikelytobethesolepurviewoflPFC.Rather,theexecutionofcontrolsignalsislikelymediatedbyseveralofdACC’sdownstreamtargetsthatcanimplementdifferenttypesofcontrolsignals(Shenhavetal.2013,Ullspergeretal.2014),includingglobalmodulatorychangesinresponsethreshold(drivenbysubthalamicnucleus)(Cavanagh&Frank2014,Cavanaghetal.2011,Keukenetal.2015)andinthegainofneuralprocessing(drivenbylocuscoeruleus)(Aston-Jones&Cohen2005,Eldaretal.2013,Jepma&Nieuwenhuis2011).

Itisimpossibletodiscusstheroleofcorticalandsubcorticalcircuitsineffortallocationwithoutalsoconsideringthecentralroleofthemidbraindopaminergicsystem(reviewedinCools2016,Salamoneetal.2009,Westbrook&Braver2016).Atshortertimescales,dopaminergiccircuitshavebeenshowntodiscountphasicresponsestorewardsbythelevelsofphysicalorcognitiveeffortrequired(Botvinicketal.2009a,Pasquereau&Turner2013,Varazzanietal.2015;butseealsoGanetal.2010).Atlongertimescales,dopaminergicinputtocortex,andtodACCinparticular,hasbeenshowntobecausallynecessaryforengagementineffortfulbehavior(Holroyd&McClure2015,Salamoneetal.2009,Waltonetal.2007),particularlywhentheeffortrequirestheagenttoovercomeabiastowardasalientdefaultalternative(Nicola2010).Forinstance,pharmacologicalattenuationof

19

dopaminelevels,ordamagetothewhitemattertractsconnectingthenucleusaccumbensandcingulatecortex,rendersanimalslesswillingtoexpendefforttoachieveagreaterreward.

ThemotivationalimpairmentsresultingfromdopaminemanipulationaresimilartothosediscussedabovethatresultfromdamagetodACCand,often,adjacentregionsofsupplementarymotorarea(SMA).VariabilityinstructuralandfunctionalconnectivitybetweendACCandSMAhasalsobeenimplicatedinmoresubtlemanifestationsofapathy,andtheirrelationshiptodecisionsaboutphysicaleffortinvestment(Bonnelleetal.2015).Mostnotably,transcranialmagneticstimulationofSMAhasbeenfurthershowntoattenuatetheaversiveexperienceofaphysicallydemandingtask,renderingparticipantsmorewillingtoexerteffortforlessreward(Zenonetal.2015).Collectively,thesefindingssuggestthatinteractionswithinandamongmedialprefrontalcortexanddopaminergiccircuitsarecriticalfortransformingevaluativeinputsintoeffortinvestments,bothinthephysicalandcognitivedomain(seealsoHoskingetal.2014).However,inthefinalsectionbelow,wediscussimportantquestionsthatremainregardingthedegreeofneuralandcomputationaloverlapthatexistsacrossthesedomains.

5.CHALLENGESANDFUTUREDIRECTIONS

Asourreviewshows,thereisanexcitingconfluenceofresearchonthecomputationalandneuralbasisofcontrolcostsandourabilitytoallocatecontrolwhileaccountingforthesecosts.Yetseveralimportantquestionsremaintobeaddressed.

5.1.OperationalizingandMeasuringCognitiveEffort

First,althoughoverwhelmingevidencesuggeststhatcontroliscostly,itisfarfromclearhowthosecostsshouldbeoperationalized(i.e.,whataretheirconstituents)or,moreimportantly,howtheyshouldbemeasured.Controlcostshavebeeninferredfromavarietyofmeasuresincludingresponsetimes(RTs)(Anderson1996,Liederetal.2014,Ratcliff1978,Sternberg1969),avoidantpreferences(Kooletal.2010,McGuire&Botvinick2010,Westbrooketal.2013),affectivepriming(Dreisbach&Fischer2015),pupildiameter(Kahneman&Beatty1966),contractionofspecificfacialmuscles(corrugatorsupercilli)(Elkins-Brownetal.2015),sympatheticarousal(Critchleyetal.2003),andneuralactivitymeasuredfromdACC(Cavanagh&Frank2014,Cavanaghetal.2014,McGuire&Botvinick2010,Spuntetal.2012)andotherregions(Blainetal.2016,McGuire&Botvinick2010;reviewsinInzlichtetal.2015,Westbrook&Braver2015).However,noneofthesemeasureshasbeenshowntobeselectivetocontrolcosts;rather,manyhavebeenshowntoindexother,moregeneral,factorssuchassympatheticarousal;evenaseeminglydirectmeasuresuchasdemandavoidancecanbesusceptibletoancillaryfactorsrelatedtoexperimentaldemandandone’sabilityormotivationtodetectthepresenceofcontrolcostdifferencesacrosstasks(Goldetal.2014).Moreover,someofthesemeasuresmaybesensitivetoboththecostsofcontrolandthecontrolbeingallocated,thusconfoundingtheirinterpretationasreflectingoneortheother.Forinstance,longerRTscouldreflecta

20

moredifficulttask(i.e.,indicatinghighercost)orlesseffort(i.e.,indicatingafailuretoallocateadequatecontrol).

Complicatingmattersfurther,thereisampleevidencethatcognitiveeffortcanhavepositiveassociationsinadditiontotheaversiveonesdescribedaboveandthatthismaydifferconsiderablyacrossindividuals.Forexample,certaincontextsorpersonalitytraits[e.g.,needforcognition(Cacioppo&Petty1982),learnedindustriousness(Eisenberger1992)]willleadanindividualtoassociatementaleffortexertionpersewithreward,independentoftheoutcomeoftheeffort.Moreover,itiswellknownthatpeoplefindtasksrequiringlowlevelsofcognitiveengagementtobeboringandthereforeaversive,andtheyinsteadseektofindanoptimalmidpointbetweentoolittleandtoomuchcognitiveeffort(Nakamura&Csikszentmihalyi2002).Whereastheformerexampleslinkrewardmoredirectlytoeffortexertion---leadingtopredictionsthataredifficulttodissociatefromintensity-basedcontrolcostpredictions---boredomhasbeenproposedtoinsteadbeareactiontolowlevelsofinformationorarousalincertainenvironments(Eastwoodetal.2012;Geanaetal.2016a,b;Zakay2014)andisthereforepotentiallymorereadilydissociatedfromcontrolcosts.Dissociatingthesecostandrewardfunctionswillrequirecarefulexperimentaldesignandmeasurementofparticipant-specificestimatesofrewardandeffort.

Theseconcernsregardingmeasurementandoverlappingcostandrewardfunctionshighlighttheimportanceofgeneratingpreciseandquantitativepredictionsregardingthefactorsinfluencingcontrolallocationandconstrainingthesepredictionswithmultipleclearlyspecifiedpredictorvariables.Inadditiontotheaforementionedmeasures,thisnaturallyincludesmeasuringself-reportedsubjectiveexperiencesofeffort.Itremainsanopenquestionwhetherconsciousawarenessisaprerequisitefortreatingtheexertionofcontrolascostly(cf.Desenderetal.2014,Dunnetal.2016,Mulertetal.2005,Naccacheetal.2005)andwhethercognitivecontrolevenfunctionsinparttoregulatethesenegativeexperiences(Inzlichtetal.2015).Regardless,subjectivemeasurescanundoubtedlyhelpconstrainatheorybyidentifyingsimilaritiesanddifferencesbetweentask-andcontext-relatedvariablesthattriggerdifferentkindsofavoidantreactionsandtherebydefineatopologyofsubjectiveexperiencescharacterizedbytermssuchasdifficult,frustrating,tiring,stressful,challenging,andboring(cf.Saundersetal.2015,Spuntetal.2012).Theymayalsohelpteaseapartthecomponentsofcontrolcoststhatrelatetoeffortpersefromotheraversivereactionsassociatedwithindividualcontroldemands,suchaserrorsandconflictoruncertainty.

5.2.DisentanglingCompetingModelsandMechanisms

Anothercriticalstepisformodelsofcontrolcostsandcontrolallocationtobecomparedwithoneanotherdirectly,whichinturnwillrequiremodelstoreconciledifferencesinrelevantterminology.Thisprocessislikelytorevealsimilaritiesorevensubstitutabilityinindividualalgorithms,aswellassubstantivedifferencesatthelevelsoftheory,

21

implementation,orboth.Thegoalshouldbetogeneratecommonbenchmarksforthesevariedmodelsintermsofbothselectionandexecutionofcontrolandtoperformdedicatedexperimentsthatexaminewhichmodelbestpredictshowpeopleallocatementaleffort.Forinstance,futureresearchshouldevaluatemorecarefullytheextenttowhichcontrolcostincreases(a)monotonicallywithcontrolsignalintensityandduration(Musslicketal.2015,Shenhavetal.2013);(b)withtherichnessofcontextrequiredtosupportnondefaultresponses(asindexedbythedifferencebetweenthedistributionofpotentialresponsesforthecontrolledversusautomaticpolicy)(Ortega&Braun2011,2013);and/or(c)withthevalueofopportunitiesforegonewhileengagedincandidatecomputations(Kurzbanetal.2013,F.Lieder&T.L.Griffiths,submittedmanuscript,Liederetal.2014).

Suchmodelcomparisonwillfurtheraideffortstointerprettheroleofneuralmechanismsthathavebeenimplicatedincontrolallocationandgeneratemoreprecisepredictionsregardingthetime-courseofactivationacrossregionsinresponsetosignalsthatdemandachangeincontrolsettings.ThiswillinturnhelptosettlelongstandingdebatesregardingthefunctionalroleorrolesofdACCandotherregionsacrossresearchintodecisionmaking,cognitivecontrol,andaffectiveprocessing(Heilbronner&Hayden2016;Holroyd&Yeung2012;Shackmanetal.2011;Shenhavetal.2013,2016).ItwillfurtherhelptoconstrainpredictionsforagiventheoryregardinghowtointerpretdACC’softenunderdeterminedengagementinaparticularcontext---forinstance,whetheritreflectsthecosts,demands,and/oroutputofcontrol(Shenhavetal.2013).

5.3.RelationshipBetweenDifferentFormsofEffort

Attemptstounderstandthecomputationalandneuralunderpinningsofcognitiveeffortfrequentlydrawconnectionstothosesameunderpinningsforphysicalormotoreffort.Thismaybeunavoidablegiventheempiricallinksbetweenthephenomenologyandmechanismsofthesetwoformsofeffort(Marcoraetal.2009,Schmidtetal.2012),butitpromptsaveryimportantquestion:Ifphysicaleffortisinfactassociatedwithphysicalresourcedepletion(Cabanac2006),andcognitiveeffortturnsoutnottobe(Inzlicht&Schmeichel2012,Kurzbanetal.2013),thenwhyaretheresimilarunderpinningsforthesetwo?Aprovocativepossibilityisthatthecostlycomponentofphysicaleffortinfacthaslittletodowithphysicalresources(Marcora2009,Marcora&Staiano2010,cf.Huangetal.2012)butratherisasimilarcontrolcostasforcognitivecontrol---thecostofovercomingamoreautomaticoption(e.g.,morehabitualbehavior).Giventhefundamentalnatureoftherelationshipbetweenthesetwoformsofeffort,futureworkshouldbettercharacterizetheirrespectivecostfunctionsandthebasisforanysimilaritiesamongthese.Thisworkshouldalsoaimtocomparethedegreetowhichdifferentspeciesdiscountrewardsforasimilartypeofcognitiveorphysicaleffort.

5.4.ClinicalandPolicy-RelatedApplications

22

Appropriateallocationofcognitiveeffortiscentraltoourabilitytothriveashumans,particularlygiventhedemandsofmodernenvironments.Individualswhoarewillingand/orabletoexertcontrolmoreconsistentlyandinspiteofapparentobstaclesareabletoperformbetterinacademicandworkenvironments(Duckworthetal.2007,Eigstietal.2006,Mischeletal.1989).Conversely,varietiesofimpairmentsincontrolallocationresultingfromdisorderssuchasmajordepression,schizophrenia,addiction,obsessive-compulsivedisorder,andattention-deficitdisordercanbeparticularlydebilitating(Cools2016,Salamoneetal.2016,Holroyd&Umemoto2016,Westbrook&Braver2015).

Yetevenhigh-functioningindividualsarepronetofailingtoexertthecontrolrequiredtooverridehabitual,impulsive,andothershort-sightedresponsetendencies;suchself-controlfailureshaveseriousramificationsforthehealth,safety,relationships,educationalattainment,andfinancesoftheseindividuals(Duckworthetal.2007,Heatherton&Wagner2011,Mischeletal.1989)andeventheevolutionofourspecies(Cohen2005,Tomlinetal.2015).Fromtheperspectiveofboundedoptimality,someofthesefailuresmightbeinevitableincomplexenvironmentswithtoomanymisleadingtemptationsbecausepeople’scognitiveresourcesarefiniteandtheirtimeislimited.Indeed,toleratingoccasionalself-controlfailuresmaybemoreboundedlyoptimalthanensuringsuchfailuresneveroccur.

Accordingtothisview,peopleshouldinsteadrestructuretheirenvironmentsothatgooddecisionscanresultfromsimpleheuristicsratherthanrequiringextensivecognitiveoperations(Gigerenzer2008,Lieder&Griffiths2016).Oneexampleofthisapproachthathasprovedeffectiveistoimposedefaultoptionsforcertaindecisionsthatmaximizethetypicaldecisionmaker’slong-termrewards(e.g.,retirementsavings),apolicyreferredtoaspaternalisticlibertarianism(Thaler&Sunstein2008).Anotherapproachistoaligntheimmediaterewardsofeachchoicewithitslong-termvalue,potentiallyenablingpeopletorelyonautomatic,short-sighteddecisionmechanismsinsteadofhavingtooverridethemandinsteadengageeffortfullong-termplanning(Lieder&Griffiths2016).Theseapproachesofferusefuldirectionsforpoliciesaimedatimprovingpeople’sabilitytodeploycognitiveeffortadaptivelywithincomplexenvironments.

Formalmodelsofcontrolallocationofferacriticalpathforwardinunderstandingthemechanismsbywhichindividualssucceedorfailatachievingthedesiredreturnontheircognitiveeffortinvestment.Thesemodelscanprovideinsightintohowtomakethecontrolallocationmostappropriatetothetaskathand---forinstance,byimprovingstrategiesforlearningaboutthevalueofexertingcognitiveeffortandhowitdependsondifferentattributesofthetaskorsituation.Inaddition,suchmodelsmayalsoprovideinsightsintohowtomakecontrollesscostly,forinstancebyallowingpeopletorelymoreonprocessesthatarewelllearnedandoverlapminimallywithotherprocessesthatmayneedtobeengaged.Thesemodelsmightthereforefacilitatethedesignofinterventionstoimprovetheallocationofcognitivecontrolandrestructuretheenvironmenttomaximizethelikelihoodthatattemptsatcontrolsucceedintheirgoals.Advancesinmodelingandempiricallyvalidatingsuchinterventionswillredoundtothebenefitofour

23

understandingofthemechanismsunderlyingcognitiveeffortallocationand,moregenerally,toansweringtheage-oldquestionofwhatmakesithardtothinkandwhatwecandoaboutit.

DISCLOSURESTATEMENT

Theauthorsarenotawareofanyaffiliations,memberships,funding,orfinancialholdingsthatmightbeperceivedasaffectingtheobjectivityofthisreview.

ACKNOWLEDGMENTS

TheauthorswouldliketothankCeydaSayaliandAndrewWestbrookforhelpfulcommentsonanearlierdraftofthisreview.

LITERATURECITED

AartsE,RoelofsA,FrankeB,RijpkemaM,FernándezG,etal.2010.Striataldopaminemediatestheinterfacebetweenmotivationalandcognitivecontrolinhumans:evidencefromgeneticimaging.Neuropsychopharmacology35:1943--51

AlexanderWH,BrownJW.2015.Hierarchicalerror representation:acomputationalmodelofanteriorcingulateanddorsolateralprefrontalcortex.NeuralComput.27:2354--410

Allport A, Styles EA, Hsieh S. 1994. Shifting intentional set: exploring the dynamic control oftasks. In Attention and Performance XV: Conscious and Nonconscious InformationProcessing,ed.CUmiltà,MMoscovitch,pp.421--52.Cambridge,MA:MITPress

AllportA,WylieG.1999.Task-switching:positiveandnegativeprimingoftask-set.InAttention,Space, and Action: Studies in Cognitive Neuroscience, ed. GW Humphreys, J Duncan, ATreisman,pp.273--96.NewYork:OxfordUniv.Press

AllportDA.1980.Attentionandperformance. InCognitivePsychology:NewDirections ed.GIClaxton,pp.112--53.London:Routledge

AllportDA,AntonisB,ReynoldsP.1972.Onthedivisionofattention:adisproofof thesinglechannelhypothesis.Q.J.Exp.Psychol.24:225--35

AndersonJR.1983.TheArchitectureofCognition.Cambridge,MA:HarvardUniv.Press

AndersonJR.1996.ACT:asimpletheoryofcomplexcognition.Am.Psychol.51:355--65

Aston-Jones G, Cohen JD. 2005. An integrative theory of locus coeruleus-norepinephrinefunction:adaptivegainandoptimalperformance.Annu.Rev.Neurosci.28:403--50

Baumeister RF, Bratslavsky E,MuravenM, TiceDM. 1998. Ego depletion: Is the active self alimitedresource?J.Pers.Soc.Psychol.74:1252--65

BaumeisterRF,HeathertonTF.1996.Self-regulationfailure:anoverview.Psychol.Inq.7:1--15

24

BengioY,CourvilleA,VincentP.2013.Representationlearning:areviewandnewperspectives.IEEETrans.PatternAnal.Mach.Intell.35:1798--828

BlainB,HollardG,PessiglioneM.2016.Neuralmechanismsunderlyingthe impactofdaylongcognitiveworkoneconomicdecisions.PNAS113:6967--72

Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD. 2006. The physics of optimal decisionmaking:aformalanalysisofmodelsofperformanceintwo-alternativeforced-choicetasks.Psychol.Rev.113:700--65

BonnelleV,Manohar S, Behrens T,HusainM. 2015. Individual differences in premotor brainsystemsunderliebehavioralapathy.Cereb.Cortex26:807--19

Bonner SE, Sprinkle GB. 2002. The effects of monetary incentives on effort and taskperformance: theories, evidence, and a framework for research. Account. Organ. Soc.27:303--45

Botvinick MM, Braver T. 2015. Motivation and cognitive control: from behavior to neuralmechanism.Annu.Rev.Psychol.66:83--113

Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. 2001. Conflict monitoring andcognitivecontrol.Psychol.Rev.108:624--52

Botvinick MM, Cohen JD. 2015. The computational and neural basis of cognitive control:chartedterritoryandnewfrontiers.Cogn.Sci.38:1249--85

BotvinickMM,HuffstetlerS,McGuireJ.2009a.Effortdiscountinginhumannucleusaccumbens.Cogn.Affect.Behav.Neurosci.9:16--27

Botvinick MM, Niv Y, Barto AC. 2009b. Hierarchically organized behavior and its neuralfoundations:areinforcementlearningperspective.Cognition113:262--80

Braver TS. 2012. The variable nature of cognitive control: a dual mechanisms framework.TrendsCogn.Sci.16:105--12

Brown JW, Braver TS. 2005. Learned predictions of error likelihood in the anterior cingulatecortex.Science307:1118--21

BushG,LuuP,PosnerM.2000.Cognitiveandemotionalinfluencesinanteriorcingulatecortex.TrendsCogn.Sci.4:215--22

CabanacM.2006.Exertionandpleasurefromanevolutionaryperspective.InPsychobiologyofPhysicalActivity,ed.EOAcevedo,PEkkekakis,pp.79--89.Champaign,IL:Hum.Kinet.

CacioppoJT,PettyRE.1982.Theneedforcognition.JPers.Soc.Psychol.42:116--31

CaiW,ChenT,RyaliS,KochalkaJ,LiC-SR,MenonV.2016.Causalinteractionswithinafrontal-cingulate-parietalnetworkduringcognitivecontrol:convergentevidencefromamultisite--multitaskinvestigation.Cereb.Cortex26:2140--53

CamererC,HogarthR. 1999. Theeffectsof financial incentives inexperiments: a reviewandcapital-labor-productionframework.J.RiskUncertain.19:7--42

25

CarterEC,KoflerLM,ForsterDE.2015.Aseriesofmeta-analytictestsofthedepletioneffect:Self-controldoesnotseemtorelyonalimitedresource.J.Exp.Psychol.Gen.144:796--815

CaruanaR.1998.Multitasklearning.InLearningtoLearn,ed.SThrun,LPratt,pp.95--133.NewYork:Springer

Casey BJ, Somerville LH, Gotlib IH, Ayduk O, Franklin NT, et al. 2011. Behavioral and neuralcorrelatesofdelayofgratification40yearslater.PNAS108:14998--5003

CavanaghJF,FrankMJ.2014.Frontalthetaasamechanismforcognitivecontrol.TrendsCogn.Sci.18:414--21

Cavanagh JF, Masters SE, Bath K, Frank MJ. 2014. Conflict acts as an implicit cost inreinforcementlearning.Nat.Commun.5:5394

CavanaghJF,WieckiTV,CohenMX,FigueroaCM,SamantaJ,etal.2011.Subthalamicnucleusstimulation reverses mediofrontal influence over decision threshold. Nat. Neurosci.14:1462--67

ChristieST,SchraterP.2015.Cognitivecostasdynamicallocationofenergeticresources.Front.Neurosci.9:289

Cohen JD. 2005. The vulcanization of the human brain: a neural perspective on interactionsbetweencognitionandemotion.J.Econ.Perspect.19:3--24

Cohen JD, Dunbar K,McClelland JL. 1990. On the control of automatic processes: a paralleldistributedprocessingaccountoftheStroopeffect.Psychol.Rev.97:332--61

CoolsR.2016.Thecostsandbenefitsofbraindopamineforcognitivecontrol.WileyInterdiscip.Rev.Cogn.Sci.7:317--29

CowanN.2012.WorkingMemoryCapacity.NewYork:PsychologyPress

CowanN,Rouder JN,BlumeCL, Saults JS. 2012.Models of verbalworkingmemory capacity:Whatdoesittaketomakethemwork?Psychol.Rev.119:480--99

CraigAD.2009.Howdoyoufeel—now?Theanteriorinsulaandhumanawareness.Nat.Rev.Neurosci.10:59--70

CritchleyHD,MathiasCJ,JosephsO,O’DohertyJP,ZaniniS,etal.2003.Humancingulatecortexandautonomiccontrol:convergingneuroimagingandclinicalevidence.Brain126:2139--52

Croxson PL, WaltonME, O'Reilly JX, Behrens TEJ, RushworthMFS. 2009. Effort-based cost--benefitvaluationandthehumanbrain.J.Neurosci.29:4531--41

Desender K, VanOpstal F, Van den Bussche E. 2014. Feeling the conflict: the crucial role ofconflictexperienceinadaptation.Psychol.Sci.25:675--83

DixonML, Christoff K. 2012. The decision to engage cognitive control is driven by expectedreward-value:neuralandbehavioralevidence.PLOSONE7:e51637

Dosenbach NUF, Fair DA, Cohen AL, Schlaggar BL, Petersen SE. 2008. A dual-networksarchitectureoftop-downcontrol.TrendsCogn.Sci.12:99--105

26

DosenbachNUF,VisscherKM,PalmerED,MiezinFM,WengerKK,etal.2006.Acoresystemfortheimplementationoftasksets.Neuron50:799--812

Dreisbach G, Fischer R. 2015. Conflicts as aversive signals for control adaptation. Curr. Dir.Psychol.Sci.24:255--60

DuckworthAL,PetersonC,MatthewsMD,KellyDR.2007.Grit:perseveranceandpassion forlong-termgoals.JPers.Soc.Psychol.92:1087--101

Duckworth AL, Quinn PD, Lynam DR, Loeber R, Stouthamer-Loeber M. 2011. Role of testmotivationinintelligencetesting.PNAS108:7716--20

DuncanJ.2010.Themultiple-demand(MD)systemoftheprimatebrain:mentalprogramsforintelligentbehaviour.TrendsCogn.Sci.14:172--79

DunnTL,LutesDJ,RiskoEF.2016.Metacognitiveevaluationintheavoidanceofdemand.J.Exp.Psychol.42:1372--87

EastwoodJD,FrischenA,FenskeMJ,SmilekD.2012.Theunengagedmind:definingboredomintermsofattention.Perspect.Psychol.Sci.7:482--95

Eigsti I, ZayasV,MischelW, ShodaY,AydukO,et al. 2006.Predicting cognitive control frompreschooltolateadolescenceandyoungadulthood.Psychol.Sci.17:478--84

EisenbergerR.1992.Learnedindustriousness.Psychol.Rev.99:248--67

Eldar E, Cohen JD, Niv Y. 2013. The effects of neural gain on attention and learning. Nat.Neurosci.16:1146--53

Elkins-BrownN,SaundersB,InzlichtM.2015.Error-relatedelectromyographicactivityoverthecorrugator supercilii is associatedwith neural performancemonitoring. Psychophysiology53:159--70

Elman, J. L. (1993). Learning and development in neural networks: the importance of starting small. Cognition, 48(1), 71–99.

ElmoreLC,MaWJ,MagnottiJF,LeisingKJ,PassaroAD,etal.2011.Visualshort-termmemorycomparedinrhesusmonkeysandhumans.Curr.Biol.21:975--79

Engelmann JB, Damaraju E, Padmala S, Pessoa L. 2009. Combined effects of attention andmotivationonvisualtaskperformance:transientandsustainedmotivationaleffects.Front.Hum.Neurosci.3:4

FellowsLK,FarahMJ.2005. Isanteriorcingulatecortexnecessaryforcognitivecontrol?Brain128:788--96

Feng SF, Schwemmer M, Gershman SJ, Cohen JD. 2014. Multitasking versus multiplexing:toward a normative account of limitations in the simultaneous execution of control-demandingbehaviors.Cogn.Affect.Behav.Neurosci.14:129--46

FrankMJ,BadreD.2012.Mechanismsofhierarchical reinforcement learning incorticostriatalcircuits1:computationalanalysis.Cereb.Cortex22:509--26

27

GailliotMT,BaumeisterRF,DeWallCN,Maner JK,PlantEA,etal.2007.Self-control reliesonglucose as a limited energy source: Willpower is more than a metaphor. J. Pers. Soc.Psychol.92:325--36

Gan, J.O.,Walton,M. E.,& Phillips, P. E.M. 2010.Dissociable cost and benefit encoding offuturerewardsbymesolimbicdopamine.NatureNeuroscience,13:25--27.

GarnerKG,DuxPE.2015.Trainingconquersmultitaskingcostsbydividingtaskrepresentationsinthefrontoparietal-subcorticalsystem.PNAS112:14372--77

GeanaA,WilsonRC,DawND,CohenJD.2016a.Boredom,information-seekingandexploration.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.1751--56

GeanaA,WilsonRC,DawND,CohenJD.2016b.Information-seeking,learningandthemarginalvaluetheorem:anormativeapproachtoadaptiveexploration.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.1793--98

Gershman SJ, Horvitz EJ, Tenenbaum JB. 2015. Computational rationality: a convergingparadigmforintelligenceinbrains,minds,andmachines.Science349:273--78

Gigerenzer G. 2008.Rationality forMortals: How People Copewith Uncertainty. Oxford, UK:OxfordUniv.Press

GoldJM,KoolW,BotvinickMM,HubzinL,AugustS,WaltzJA.2014.Cognitiveeffortavoidanceanddetectioninpeoplewithschizophrenia.Cogn.Affect.Behav.Neurosci.15:145--54

GraybielAM.2008.Habits,rituals,andtheevaluativebrain.Annu.Rev.Neurosci.31:359--87

GriffithsTL,LiederF,GoodmanND.2015.Rationaluseofcognitiveresources:levelsofanalysisbetweenthecomputationalandthealgorithmic.Top.Cogn.Sci.7:217--29

HaggerMS,ChatzisarantisNLD.2016.Amultilabpreregisteredreplicationoftheego-depletioneffect.Perspect.Psychol.Sci.11:546--73

HaggerMS,WoodC,StiffC,ChatzisarantisNLD.2010.Egodepletionandthestrengthmodelofself-control:ameta-analysis.Psychol.Bull.136:495--525

HayN,RussellS,TolpinD,ShimonySE.2012.Selectingcomputations:theoryandapplications.Uncertain. Artif. Intell. Proc. Twenty-Eighth Conf., Catalina Isl., Aug. 15--17, pp. 346--55.Corvallis,OR:AUAIPress

Heatherton TF, Wagner DD. 2011. Cognitive neuroscience of self-regulation failure. TrendsCogn.Sci.15:132--39

Heilbronner SR, Hayden BY. 2016. Dorsal anterior cingulate cortex: a bottom-up view.Annu.Rev.Neurosci.39:149--70

HillmanKL,BilkeyDK.2010.Neurons in the rat anterior cingulate cortexdynamically encodecost-benefitinaspatialdecision-makingtask.J.Neurosci.30:7705--13

HockeyGRJ.1997.Compensatorycontrolintheregulationofhumanperformanceunderstressandhighworkload:acognitive-energeticalframework.Biol.Psychol.45:73--93

28

Hockey GRJ. 2011. A motivational control theory of cognitive fatigue. In Cognitive Fatigue:Multidisciplinary Perspectives on Current Research and Future Applications, ed. PLAckerman,pp.167--87.Washington,DC:Am.Psychol.Assoc.

HolroydCB.2015.Thewastedisposalproblemofeffortfulcontrol.InMotivationandCognitiveControl,ed.TSBraver,pp.235--60.NewYork:Routledge

Holroyd CB, Coles MGH. 2002. The neural basis of human error processing: reinforcementlearning,dopamine,andtheerror-relatednegativity.Psychol.Rev.109:679--709

HolroydCB,McClureSM.2015.Hierarchical controlovereffortfulbehaviorby rodentmedialfrontalcortex:acomputationalmodel.Psychol.Rev.122:54--83

HolroydCB,UmemotoA.2016.Theresearchdomaincriteriaframework:thecaseforanteriorcingulatecortex.Neurosci.Behav.Rev.71:418--43

Holroyd CB, Yeung N. 2012. Motivation of extended behaviors by anterior cingulate cortex.TrendsCogn.Sci.16:121--27

Horvitz E, Rutledge G. 1991. Time-dependent utility and action under uncertainty. In Proc.SeventhConf.Uncertain.Artif.Intell.,LosAngeles,pp.151--58.Corvallis,OR:AUAIPress

HoskingJG,FlorescoSB,WinstanleyCA.2014.Dopamineantagonismdecreaseswillingnesstoexpend physical, but not cognitive, effort: a comparison of two rodent cost/benefitdecision-makingtasks.Neuropsychopharmacology40:1005—15

Huang,H.J.,Kram,R.,Ahmed,A.A.2012.ReductionofMetabolicCostduringMotorLearningofArmReachingDynamics.J.Neurosci.32:2182—2190.

InzlichtM, BartholowBD,Hirsh JB. 2015. Emotional foundations of cognitive control.TrendsCogn.Sci.19:126--32

InzlichtM,SchmeichelBJ.2012.What isegodepletion?Towardamechanisticrevisionoftheresourcemodelofself-control.Perspect.Psychol.Sci.7:450--63

JamesW.1891.ThePrinciplesofPsychology,Volume2.London:Macmillan

Jensen AR. 1988. Speed of information processing and population differences. In HumanAbilitiesinCulturalContext,ed.SHIrvine,JWBerry,pp.105--45.Cambridge,UK:CambridgeUniv.Press

Jepma M, Nieuwenhuis S. 2011. Pupil diameter predicts changes in the exploration--exploitationtrade-off:evidencefortheadaptivegaintheory.J.Cogn.Neurosci.23:1587--96

KahnemanD.1973.AttentionandEffort.EnglewoodCliffs,NJ:Prentice-Hall

KahnemanD.2003.Aperspectiveonjudgmentandchoice:mappingboundedrationality.Am.Psychol.58:697--720

KahnemanD,BeattyJ.1966.Pupildiameterandloadonmemory.Science154:1583--85

KahnemanD,TverskyA.1979.Prospecttheory:ananalysisofdecisionunderrisk.Econometrica47:263--91

29

Kaping D, VinckM, Hutchison RM, Everling S,Womelsdorf T. 2011. Specific contributions ofventromedial,anteriorcingulate,andlateralprefrontalcortexforattentionalselectionandstimulusvaluation.PLOSBiol.9:e1001224

Kerns JG,Cohen JD,MacDonaldAW,ChoRY, StengerVA,CarterCS.2004.Anterior cingulateconflictmonitoringandadjustmentsincontrol.Science303:1023--26

Keuken MC, van Maanen L, Bogacz R, Schäfer A, Neumann J, et al. 2015. The subthalamicnucleusduringdecision-makingwithmultiplealternatives.Hum.BrainMapp.36:4041--52

KoolW,McGuire JT, Rosen ZB, BotvinickMM. 2010. Decisionmaking and the avoidance ofcognitivedemand.J.Exp.Psychol.:Gen.139:665--82

Kouneiher F, Charron S, Koechlin E. 2009. Motivation and cognitive control in the humanprefrontalcortex.Nat.Neurosci.12:939--45

Krebs RM, Boehler CN,WoldorffMG. 2010. The influence of reward associations on conflictprocessingintheStrooptask.Cognition117:341--47

Kurzban R, Duckworth A, Kable JW,Myers J. 2013. An opportunity costmodel of subjectiveeffortandtaskperformance.Behav.BrainSci.36:661--79

LeCunY,BengioY,HintonG.2015.Deeplearning.Nature521:436--44

LewisRL,HowesA,SinghS.2014.Computationalrationality: linkingmechanismandbehaviorthroughboundedutilitymaximization.Top.Cogn.Sci.6:279--311

Lieder F, Griffiths TL. 2015.When to usewhich heuristic: a rational solution to the strategyselectionproblem.Proc.37thAnnu.Conf.Cogn.Sci.Soc.,Austin,TX,pp.1362--67

LiederF,GriffithsTL.2016.Helpingpeoplemakebetterdecisionsusingoptimalgamification.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.2075--80

LiederF,GriffithsTL,GoodmanND.2012.Burn-in,bias,andtherationalityofanchoring.Adv.NeuralInf.Process.Syst.25(NIPS2012),LakeTahoe,NV,pp.2690–2798

LiederF,PlunkettD,Hamrick JB,Russell SJ,HayNJ,GriffithsTL.2014.Algorithmselectionbyrationalmetareasoning as amodel of human strategy selection.Adv.Neural Inf. Process.Syst.27(NIPS2014),Montreal,pp.2870--78

Logan GD. 1985. Skill and automaticity: relations, implications, and future directions.Can. J.Psychol.39:367--86

LuckSJ,VogelEK.1997.Thecapacityofvisualworkingmemoryforfeaturesandconjunctions.Nature390:279--81

Ma WJ, Husain M, Bays PM. 2014. Changing concepts of working memory. Nat. Neurosci.17:347--56

MagnoE,FoxeJJ,MolholmS,RobertsonIH,GaravanH.2006.Theanteriorcingulateanderroravoidance.J.Neurosci.26:4769—73

30

Mansouri, F. A., Egner, T., & Buckley, M. J. (in press). Monitoring Demands for ExecutiveControl: Shared Functions between Human and Nonhuman Primates. Trends inNeurosciences.

Marcora SM. 2009. Perception of effort during exercise is independent of afferent feedbackfromskeletalmuscles,heart,andlungs.J.Appl.Physiol.106:2060--62

MarcoraSM,StaianoW.2010.The limit toexercise tolerance inhumans:mindovermuscle?Eur.J.Appl.Physiol.109:763--70

Marcora SM, Staiano W, Manning V. 2009. Mental fatigue impairs physical performance inhumans.J.Appl.Physiol.106:857--64

McCarthyJ.1959.Programswithcommonsense.Proc.Symp.Mech.ThoughtProcess.,Vol.1,pp.75--91.London:HerMajesty’sStation.Off.

McCarthy J, Hayes PJ. 1969. Some philosophical problems from the standpoint of artificialintelligence.InReadingsinArtificialInteligence.,ed.BLWebberNJNilsson,pp.431–50,LosAltos,CA:MorganKaufmannPublishers,Inc.

McGuire JT, BotvinickMM.2010. Prefrontal cortex, cognitive control, and the registrationofdecisioncosts.PNAS107:7922--26

Medford N, Critchley HD. 2010. Conjoint activity of anterior insular and anterior cingulatecortex:awarenessandresponse.BrainStruct.Funct.214:535--49

MenonV,UddinLQ.2010.Saliency,switching,attentionandcontrol:anetworkmodelofinsulafunction.BrainStruct.Funct.214:655--67

Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu. Rev.Neurosci.24:167--202

MillerGA.1956.Themagicalnumberseven,plusorminustwo:somelimitsonourcapacityforprocessinginformation.Psychol.Rev.63:81--97

MischelW,ShodaY,RodriguezML.1989.Delayofgratificationinchildren.Science244:933--38

Molden DC, Hui CM, Scholer AA, Meier BP, Noreen EE, et al. 2012. Motivational versusmetaboliceffectsofcarbohydratesonself-control.Psychol.Sci.10:1137--44

MonsellS.2003.Taskswitching.TrendsCogn.Sci.7:134--40

MulertC,MenzingerE,LeichtG,PogarellO,HegerlU.2005.Evidenceforacloserelationshipbetweenconsciouseffortandanteriorcingulatecortexactivity.Int.J.Psychophysiol.56:65--80

Muraven M, Tice DM, Baumeister RF. 1998. Self-control as limited resource: regulatorydepletionpatterns.J.Pers.Soc.Psychol.74:774--89

MusslickS,DeyB,ÖczimderK,PatwaryMMA,WillkeTL,CohenJD.2016a.Parallelprocessingcapabilityversusefficiencyofrepresentation inneuralnetworks.Presentedat15thNeuralComput.Psychol.Workshop,Aug.8--9,Philadelphia

31

MusslickS,DeyB,ÖzcimderK,PatwaryMMA,WillkeTL,CohenJD.2016b.Controlledversus.automaticprocessing: a graph-theoretic approach to theanalysisof serial versus.parallelprocessing in neural network architectures. Proc. 38th Annu. Conf. Cogn. Sci. Soc.,Philadelphia,pp.1547--52

Musslick S, Shenhav A, Botvinick MM, Cohen JD. 2015. A computational model of controlallocationbasedontheexpectedvalueofcontrol.Presentedat2ndMultidiscip.Conf.Reinf.Learn.Decis.Mak.,June7--10,Edmonton,Can.

NaccacheL,DehaeneS,CohenL,HabertM-O,Guichart-GomezE,etal.2005.Effortlesscontrol:executive attention and conscious feeling of mental effort are dissociable.Neuropsychologia43:1318--28

NairneJS.1990.Afeaturemodelofimmediatememory.Mem.Cogn.18:251--69

Nakamura J, Csikszentmihalyi M. 2002. The concept of flow. In Handbook of PositivePsychology,ed.CRSnyder,SJLopez,pp.89--105.Oxford,UK:OxfordUniv.Press

Navon D, Gopher D. 1979. On the economy of the human-processing system. Psychol. Rev.86:214--55

Nicola SM. 2010. The flexible approach hypothesis: unification of effort and cue-respondinghypothesesfortheroleofnucleusaccumbensdopamineintheactivationofreward-seekingbehavior.J.Neurosci.30:16585--600

Nieuwenhuis S, Schweizer TS, Mars RB, Botvinick MM, Hajcak G. 2007. Error-likelihoodpredictioninthemedialfrontalcortex:acriticalevaluation.Cereb.Cortex17:1570--81

NivY,DawND, JoelD,DayanP.2007.Tonicdopamine:opportunity costsand thecontrolofresponsevigor.Psychopharmacology191:507--20

Oaksford M, Chater N. 2007. Bayesian Rationality: The Probabilistic Approach to HumanReasoning.Oxford,UK:OxfordUniv.Press

OberauerK,FarrellS,JarroldC,LewandowskyS.2016.Whatlimitsworkingmemorycapacity?Psychol.Bull.142:758--99

OberauerK,KlieglR.2006.Aformalmodelofcapacitylimitsinworkingmemory.J.Mem.Lang.55:601--26

Oehrn CR, Hanslmayr S, Fell J, Deuker L, Kremers NA, et al. 2014. Neural communicationpatternsunderlyingconflictdetection,resolution,andadaptation.J.Neurosci.34:10438--52

O'ReillyRC,FrankMJ.2006.Makingworkingmemorywork:acomputationalmodeloflearningintheprefrontalcortexandbasalganglia.NeuralComput.18:283--328

Ortega PA, Braun DA. 2011. Information, utility and bounded rationality.Proc. 4th Int. Conf.Artif.Gen.Intell.,MountainView,CA,Aug.3--6,pp.269--74.Berlin:Springer-Verlag

OrtegaPA,BraunDA.2013.Thermodynamicsasatheoryofdecision-makingwithinformation-processingcosts.Proc.R.Soc.A469:20120683

32

OudB,Krajbich I,MillerK,CheongJH,BotvinickM,FehrE.2016. Irrational timeallocation indecision-making.Proc.R.Soc.B283:20151439

Padmala S, Pessoa L. 2011. Reward reduces conflict by enhancing attentional control andbiasingvisualcorticalprocessing.J.Cogn.Neurosci.23:3419--32

PageM,NorrisD.1998.Theprimacymodel:anewmodelof immediateserialrecall.Psychol.Rev.105:761--81

ParviziJ,RangarajanV,ShirerWR,DesaiN,GreiciusMD.2013.Thewilltopersevereinducedbyelectricalstimulationofthehumancingulategyrus.Neuron80:1359--67

Pasquereau B, Turner RS. 2013. Limited encoding of effort by dopamine neurons in a cost-benefittrade-offtask.J.Neurosci.33:8288--300

Payne, J.W., Bettman, J. R., & Johnson, E. L. (1988). Adaptive strategy selection in decisionmaking. Journal of Experimental Psychology: Learning,Memory, and Cognition, 14, 534–552.

PosnerMI, SnyderCRR.1975.Attentionand cognitive control. In InformationProcessingandCognition:TheLoyolaSymposium,ed.RLSolso.Hillsdale,NJ:ErlbaumAssoc.

Power JD, Petersen SE. 2013. Control-related systems in the human brain. Curr. Opin.Neurobiol.23:223--28

Prévost C, Pessiglione M, Météreau E, Cléry-Melin M, Dreher J. 2010. Separate valuationsubsystemsfordelayandeffortdecisioncosts.J.Neurosci.30:14080--90

RatcliffR.1978.Atheoryofmemoryretrieval.Psychol.Rev.85:59--108

Reinhart RMG, Woodman GF. 2014. Causal control of medial-frontal cortex governselectrophysiological and behavioral indices of performance monitoring and learning. J.Neurosci.34:4214--27

RidderinkhofKR,UllspergerM,CroneEA,NieuwenhuisS.2004.Theroleofthemedialfrontalcortexincognitivecontrol.Science306:443--47

Rumelhart DE, McClelland JL. 1986. PDP models and general issues in cognitive science. InParallelDistributedProcessing:Explorations in theMicrostructureofCognition,Vol.1,ed.DERumelhart,JLMcClelland,pp.110--46.Cambridge,MA:MITPress

RushworthMFS,NoonanMP,BoormanED,WaltonME,BehrensTE.2011.Frontalcortexandreward-guidedlearninganddecision-making.Neuron70:1054--69

RushworthMFS,WaltonME,KennerleySW,BannermanD.2004.Actionsetsanddecisions inthemedialfrontalcortex.TrendsCogn.Sci.8:410--17

RussellSJ,SubramanianD.1995.Provablybounded-optimalagents.J.Artif.Intell.Res.2:575--609

RussellSJ,WefaldE.1991.Principlesofmetareasoning.Artif.Intell.49:361--95

SalamoneJD,CorreaM,FarrarAM,NunesEJ,PardoM.2009.Dopamine,behavioraleconomics,andeffort.Front.Behav.Neurosci.3:13

33

Salamone JD, Yohn SE, López-Cruz L, SanMiguel N, CorreaM. 2016. Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology.Brain139:1325--47

SaundersB,MilyavskayaM, InzlichtM.2015.Whatdoescognitivecontrol feel like?Effectiveand ineffective cognitive control is associated with divergent phenomenology.Psychophysiology52:1205--17

SchmidtL,LebretonM,Cléry-MelinM-L,DaunizeauJ,PessiglioneM.2012.Neuralmechanismsunderlyingmotivationofmentalversusphysicaleffort.PLOSBiol.10:e1001266

ShackmanAJ,SalomonsTV,SlagterHA,FoxAS,WinterJJ,DavidsonRJ.2011.Theintegrationofnegative affect, pain and cognitive control in the cingulate cortex. Nat. Rev. Neurosci.12:154--67

ShafferLH.1975.Multipleattentionincontinuousverbaltasks.InAttentionandPerformanceVed.PMARabbitt,SDornic,pp.157--67.London:Academic

ShenC,ArdidS,KapingD,WestendorffS,EverlingS,WomelsdorfT.2014.Anterior cingulatecortex cells identify process-specific errors of attentional control prior to transientprefrontal-cingulateinhibition.Cereb.Cortex25:2213--28

ShenhavA,BotvinickMM,CohenJD.2013.Theexpectedvalueofcontrol:anintegrativetheoryofanteriorcingulatecortexfunction.Neuron79:217--40

ShenhavA,Cohen JD,BotvinickMM.2016.Dorsal anterior cingulate cortex and the valueofcontrol.Nat.Neurosci.19:1286--91

Sheth SA, Mian MK, Patel SR, Asaad WF, Williams ZM, et al. 2012. Human dorsal anteriorcingulatecortexneuronsmediateongoingbehaviouraladaptation.Nature488:218--21

Shiffrin RM, Schneider W. 1977. Controlled and automatic information processing: II.Perceptuallearning,automaticattending,andageneraltheory.Psychol.Rev.84:127--90

SimonHA.1956.Rationalchoiceandthestructureoftheenvironment.Psychol.Rev.63:129--38

SimonHA.1982.ModelsofBoundedRationality,Vol.3:EmpiricallyGroundedEconomicReason.Cambridge,MA:MITPress

Spunt RP, Lieberman MD, Cohen JR, Eisenberger NI. 2012. The phenomenology of errorprocessing:thedorsalACCresponsetostop-signalerrorstracksreportsofnegativeaffect.J.Cogn.Neurosci.24:1753--65

SternbergS.1969.Memory-scanning:mentalprocessesrevealedbyreaction-timeexperiments.Am.Sci.57:421--57

Stuss DT. 2011. Functions of the frontal lobes: relation to executive functions. J. Int.Neuropsychol.Soc.17:759--65

SuttonRS,BartoAG.1998.ReinforcementLearning:AnIntroduction.Cambridge,MA:MITPress

TanakaM, Ishii A,WatanabeY. 2014.Neural effects ofmental fatigue causedby continuousattentionload:amagnetoencephalographystudy.BrainRes.1561:60--66

34

Tang H, Yu H-Y, Chou C-C, Crone NE,Madsen JR, et al. 2016. Cascade of neural processingorchestratescognitivecontrolinhumanfrontalcortex.eLife5:4671

Tangney JP,BaumeisterRF,BooneAL. 2004.High self-control predicts goodadjustment, lesspathology,bettergrades,andinterpersonalsuccess.J.Personal.72:271--324

Thaler RH, Sunstein CR. 2008. Nudge: Improving Decisions about Health, Wealth, andHappiness.NewHaven:YaleUniv.Press

Todd MT, Niv Y, Cohen JD. 2009. Learning to use working memory in partially observableenvironmentsthroughdopaminergicreinforcement.Adv.NeuralInf.Process.Syst.21:1689--96

Todd PM, Gigerenzer G. 2012. Ecological Rationality: Intelligence in the World. Oxford, UK:OxfordUniv.Press

Tomlin D, Rand DG, Ludvig EA, Cohen JD. 2015. The evolution and devolution of cognitivecontrol:thecostsofdeliberationinacompetitiveworld.Sci.Rep.5:11002

Tversky A, Kahneman D. 1974. Judgment under uncertainty: heuristics and biases. Science185:1124--31

UllspergerM,DanielmeierC,JochamG.2014.Neurophysiologyofperformancemonitoringandadaptivebehavior.Physiol.Rev.94:35--79

UllspergerM,HarsayHA,WesselJR,RidderinkhofKR.2010.Consciousperceptionoferrorsanditsrelationtotheanteriorinsula.BrainStruct.Funct.214:629—43

Ullsperger, M., Danielmeier, C., & Jocham, G. 2014. Neurophysiology of Performance Monitoring and Adaptive Behavior. Physiological Reviews. 94: 35–79.

UmemotoA,HolroydCB.2014.Task-specificeffectsofrewardontaskswitching.Psychol.Res.698--707

VadilloMA,GoldN,OsmanM.2016.Thebitter truthabout sugarandwillpower: the limitedevidentialvalueoftheglucosemodelofegodepletion.Psychol.Sci.27:1207--14

VarazzaniC,San-GalliA,GilardeauS,BouretS.2015.Noradrenalineanddopamineneuronsinthereward/efforttrade-off:adirectelectrophysiologicalcomparisoninbehavingmonkeys.J.Neurosci.35:7866--77

Verguts T, Vassena E, Silvetti M. 2015. Adaptive effort investment in cognitive and physicaltasks:Aneurocomputationalmodel.Front.Behav.Neurosci.9:57

Vogt BA. 2016. Midcingulate cortex: Structure, connections, homologies, functions anddiseases.J.Chem.Neuroanat.74:28--46

VonNeumannJ,MorgensternO.1944.TheTheoryofGamesandEconomicBehavior.Princeton,NJ:PrincetonUniv.Press

VulE,GoodmanND,GriffithsTL,TenenbaumJB.2014.Oneanddone?Optimaldecisionsfromveryfewsamples.Cogn.Sci.38:599--637

35

WaltonM,RudebeckPH,BannermanD,RushworthMFS.2007.Calculatingthecostofactinginfrontalcortex.Ann.N.Y.Acad.Sci.1104:340--56

Wang C, Trongnetrpunya A, Samuel IBH, Ding M, Kluger BM. 2016. Compensatory neuralactivityinresponsetocognitivefatigue.J.Neurosci.36:3919--24

WesselJR,DanielmeierC,MortonJB,UllspergerM.2012.Surpriseanderror:commonneuronalarchitecturefortheprocessingoferrorsandnovelty.J.Neurosci.32:7528--37

Westbrook A, Braver TS. 2015. Cognitive effort: a neuroeconomic approach. Cogn. Affect.Behav.Neurosci.15:395--415

Westbrook A, Braver TS. 2016. Dopamine does double duty in motivating cognitive effort.Neuron89:695--710

WestbrookA,KesterD,BraverTS.2013.Whatisthesubjectivecostofcognitiveeffort?Load,trait,andagingeffectsrevealedbyeconomicpreference.PLOSONE8:e68210

Wickens DD. 1984. Processing resources in attention. In Varieties of Attention, ed. RParasuraman,DRDavies,JBeatty,pp.63--102.NewYork:Academic

WolpertDM,LandyMS.2012.Motorcontrolisdecision-making.Curr.Opin.Neurobiol.22:996--1003

WomelsdorfT,JohnstonK,VinckM,EverlingS.2010.Theta-activityinanteriorcingulatecortexpredictstaskrulesandtheiradjustmentsfollowingerrors.PNAS107:5248--53

ZakayD.2014.Psychologicaltimeasinformation:thecaseofboredom.Front.Psychol.5:917

ZenonA, SidibeM,Olivier E. 2015.Disrupting the supplementarymotorareamakesphysicaleffortappearlesseffortful.J.Neurosci.35:8737--44

View publication statsView publication stats