ImprovingappropriateurgentGPcancerreferrals

AfinalreportbytheBehaviouralInsightsTeam

August2018

2

Tableofcontents

Executivesummary 3

Introduction 5

Background 5

Overviewofliteraturereview 6

BarrierstoimprovingGPreferrals 6

BehaviouralinsightsforGPreferral 7

Socialnormsandfeedback 7

Reminders 8

Personalisation,SalienceandMessenger 8

Thetrial 9

Intervention 10

Experimentaldesign 11

Results 14

Outcomemeasures 14

Potentialsavings 17

Conclusion 18

Annex1.Treatmentletters 20

Annex2.Resultswithjustthesimplecontrolgroup 25

Annex3.Literaturereview 25

Annex4.Technicalappendix 38

Endnotes 41

3

Executivesummary

Almost half of people who get cancer are diagnosed late which makes anytreatmentlesslikelytosavepeople’slives.1ThemajorityofcancersareidentifiedbyGPs,butemergencyhospitaladmissions remainamajor route todiagnosis.2Estimates suggest that earlier diagnosis could improve the survival chances of52,000patientsandsavetheNHS£210millionayear.3

UrgentGPreferral(whenaGPrefersapatientwithsuspectedcancertobeseenby a specialistwithin twoweeks) is a key tool for improvingearlydetectionofcancer.However, the use of urgent cancer referral varies substantially acrossEngland,evenwhenaccountingfordemographicfactors.However,PublicHealthEngland(PHE)currentlyusestheEnglandaverageofcrudereferralsrateandtheage-and-sex standardised ratio as benchmarks to measure performance andimproveoutcomes.Basedonthismetrics,thereisacasetoincreaseappropriatereferralsinGreaterManchesterwheretheoverallreferralrateisslightlylowerthantheEnglandaverage(3003per100,000individualsvs3164in2016/2017).4

Thetrial

The Greater Manchester Health and Social Care Partnership (GMHSCP)commissioned theBehavioural InsightsTeam (BIT) toexplore andevaluatehowbehavioural insights could be applied to improve the appropriate urgent GPreferral.

We conducted a stepped-wedge randomised trial between August 2017 andJanuary2018with244GPpracticeswhichhadanurgentGPreferralratebelowtheEnglandaverage.Wetestedtheimpactofasetofthreebehaviourally-informedletters on the urgent GP referral rates. Within the stepped-wedge design,practiceswere randomly allocated to amonth duringwhich they received theintervention (the three letters).Non-treatedpractices thatdidnot receive anyinterventionservedasourcontrolgroupinthemonthsbeforetheyweresenttheletters.

Ourmainbehaviouralapproachwastoprovidetheunder-referringGPpracticeswithsocialnormfeedbackontheirurgentGPreferralrate:howitcomparedwithotherGPpracticeinGM.Researchonsocialnormsshowsthatlettingindividualsknow how they compare to their peers can encourage them to adjust theirbehaviour.Wesentthree letterstoeachpractice:afirst letterannouncingthatthefeedbackwouldbesent,afeedbackletterandareminderletter.WesenttwocopiesofeachlettertoeveryGPpractice,onewassenttoanamedGPandtheotherwassenttothepracticemanager.

4

Results

WefoundastatisticallysignificantpositiveeffectofourinterventionontheurgentGPreferralrate.TheGPpracticesthatreceivedthesocialnormfeedbacklettersincreasedtheirurgentcrudeGPreferralby9.6%(p<0.05level),comparedtothecontrol group. Thismeans that each of the treated GP practices referred onaverage 1.5more patient permonth, compared to 14 patients referred in thecontrolgroup.Basedonouravailabledata,weestimatethiseffectpersistsforsixmonthsaftertheprovisionoffeedback.Theincreaseissubstantiallydrivenbyasizeable improvement inperformanceof the low referrers (i.e.practices in thebottom30%ofreferrersnationwide).Thesepracticesurgentlyreferred17%morepatientsthanbeforethetrial.

During the trial, our 244 under-referring GP practices urgently referred 1,281additionalpatientsofwhom90 canbe expected tohavebeendiagnosedwithcancer.Weestimate that ifall the244practices received the feedbackat thebeginningofthetrial,theywouldhavereferred2,196morepatientsanddiagnosed154morecancerpatientsinsixmonthsaftertheletterarrived.WeestimatethatGMHSCP could save £1.2m annually, thanks to the lower costs of early cancertreatment.Thesefindingsshowthatalow-intensityfeedbackinterventioncanbeacost-effectiveapproachtomeaningfullyimprovetheurgentGPcancerreferralrateofGPpracticesperformingbelow theEnglandaverage.We recommend to

5

GMHSCPtoexplorewaystoembedtheprovisionofsocialnormfeedbacktothebottom30%ofunder-referringpractices,tosustainthebenefitsofthistrial.

Introduction

The Greater Manchester Health and Social Care Partnership (GMHSCP)commissioned the Behavioural Insights Team (BIT) to explore how behaviouralinsightscouldbeusedtoencourageGPsto increasethenumberofappropriateurgentGPreferrals.Thisreportoutlinesthebackgroundtothispolicy issueanddescribes thedesign and resultsof a field experiment inwhichwe tested theimpactofsocialnormfeedbacklettersonpractices’urgentGPreferral,comparedtobusiness-as-usual,i.e.nocommunication.

Background

TheUKhasapoorrateofearlycancerdetectioncomparedtotherestoftheEU.5Currently every fifth cancer is detected late through emergency hospitaladmissions, rather than through the key early detection route - the so-calledurgentGPreferral.6Thisdiagnosisrouteensuresthaturgentlyreferredpatientswith suspected cancer are examined by a specialist within two weeks whichimproveschancesofearlytreatmentandrecovery.Thirty-onepercentofcancersarecurrentlydetectedthroughurgentGPreferralsbut22%arestillonlydetectedthroughemergencyhospitaladmission.7

ThereissubstantialvariationintheurgentreferralratesamongstdifferentClinicalCommissioning Groups (CCGs) in England.8 The differences cannot be fullyexplainedbythedifferencesinpopulationcompositionandtheprevalenceofriskfactors. In Greater Manchester, the overall referral rate is slightly below thenationalaverage(3003vs3164per100,000individualsasof2016/2017).9

Toaddressthe issueof latecancerdetection,NICEhaverecentlyupdatedtheirguidelines forGPs on urgent cancer referrals to secondary care.10 In order toincreasethenumberofreferrals,thecancer‘riskthreshold’basedonthepositivepredictivevalue(PPV)11ofsymptomshasbeenloweredfrom5%to3%.Inpractice,thismeansthataGPshouldreferallpatientswhohaveatleasta3%riskofcancer.For illustration, all patients aged over 40 with unexplained weight loss andabdominal pain should be examined by specialists. Complying with a 3% riskthresholdisadifficulttaskforGPsandnodataiscollectedonperformanceonthisindicator.Instead,PublicHealthEngland(PHE)currentlyusestheEnglandaverageofcrudereferralsrateandtheage-and-sexstandardisedtwo-weekwaitratioasbenchmarkstomeasureperformanceofpracticesandaimstoincreasereferralsbythosepracticeswhofallbelowthisaverage.

6

Emergingevidencesuggeststhaturgentreferralscanbeanefficienttoolforearlydetection.InacohortstudyofEnglishpractices,higherurgentreferralrateswereassociatedwith a lower cancermortality.Compared topracticeswithmediumreferral rates, the cancer mortality rate of GP practices who referred morepatientswas4%lower,whilstthosewhoreferredfewerpatientshada7%highermortalityrate.12Anotherrecentstudy found that theGPpractices thaturgentlyreferredmorepatientshadfewerlatestagecancers.13

Higherdemand forcancerassessmentand treatmentcouldputastrainon thesecondarycaresystemwithoutfurthersupportand investment in itsresources.AccordingtotheNHSCancerStrategy,thevolumeofsomecancertestsmaygrowby70-80%by202014whichraisesquestionsaboutthe localcapacitytoactuallydeliveronthenewguidelines.

InMarch2016,GMHSCPcommissionedBIT to runa trialsendingbehaviourally-informedletterstoGPstoimprovetheappropriateurgentGPreferralsofunder-referringGPpractices(i.e.thepracticeswithreferralratelowerthantheEnglandaverage).Therestofthisreportoutlinesourexploratorywork,thetrialdesignandresults.

Overviewofliteraturereview

To informthisproject,BITconducteda literaturereviewtoexplorebehaviouralprinciples that can be applied to encourage GPs to increase the number ofappropriateurgentGPreferrals.Weincludeaselectionofthekeyfindingsonthebarriers forGPreferralandonpromisingapproaches to influencingphysician’sbehaviourwhich informedour interventionhere.The full literature review is inAnnex3.

BarrierstoimprovingGPreferrals

Theliteratureidentifiesanumberofdoctor-centredbarriersforcompliancewithreferralrecommendations,suchas:

● Issueswithinformationoverloadorlowawareness15● Lowincentivesandinertia16● Acceptanceofandbeliefsabout the relevanceofnew recommendations

andtheirefficacyinimprovingpatients’well-being17● Organisationalconstraints18

Beyond these doctor-centred barriers, BIT’s exploratory work suggested thatfurtherbarrierswhichhamperchangeinthereferralbehaviourmaystemfromthesystemitself.Thesebarriersinclude:

7

● Conflictingmessages:19Thenew guidelinespromptGPs to increase theirurgentreferralrates,againstthebackgroundoftheoverarchingprioritytoreduce generalGP referrals to secondary care.20We believe thismixedmessaging could cause some confusion. If combined with a lack ofawarenessaboutthenewcancerreferralguidelines,thesystemmayinduceGPstodefaultintoagenerallylowlevelofreferrals.

● Secondarycareattitudes: Inparallel, secondarycare isunder significantstrain whichmight give the impression that higher cancer referrals areinappropriate.Forinstance,sometimeswhenreferralshadbeenmade,GPswerereprimandedandpatientssentbacktoprimarycareorputonawaitinglist,insteadofbeingexamined.21

BehaviouralinsightsforGPreferral

Socialnormsandfeedback

Feedbackhasbeenshowntobeareliabletoolforinfluencingdoctors’behaviour.22Theeffectsizetendstodependonthecontext,themethodoffeedbackandthespecific behaviour being targeted. Research has found feedback to be moreeffective in thecaseof lowbaselineperformance, lowcomplexityof the targetbehaviour (e.g. test or drug ordering),23 high feedback intensity (e.g. repeatedfeedbackindifferentformatsorfromaninfluentialmessenger)andincombinationwithotherinterventions.24

Social norms, especially ‘descriptive norms’ - describingwhat themajority ofpeopleareactuallydoing-offerapowerfulexampleforpeopletofollow.Thisisbecausepeopleareheavilyinfluencedbythebehaviourofthosearoundthem,aswellasbyimplicitorexplicitexpectationswithinaparticularsocietyorgroup.25

Feedbackdrawingonsocialnormsandlettingpeopleknowhowtheycomparetotheirpeers (peer comparisons) canbe very effective in encouragingpeople toadjusttheirbehaviour.Localnorms(indicatinghowgroupssimilartousbehave)2627andminoritynorms(indicatingthatfewpeoplebehavelikeus)28canbeevenmoreinfluential.Forinstance,inaBITtrial,weinformedlatetaxpayersthattheywereoneofthefewpeoplewhofailedtopaytheirtaxontime.Thismessageincreasedthetaxpaymentrateby16%.29

BIT has also successfully used social norm feedback in health. Unnecessaryantibiotic prescriptions decreased by 3.3% when we let GPs know that theirpracticewasprescribingatahigherratethanthemajorityofotherpractices intheirNHSLocalAreaTeam.30Providing feedbackhasprovedeffective inseveralother healthcare experiments aimed at reducing prescription31 or excessivediagnostictestuse.32

8

Inthistrial,weprovidedGPswithsocialnormfeedbackonhowtheircancerreferralratescompareto

theEnglandaverage.

Reminders

Timelyreminders-viaaletter,atextoraphonecall-canstoppeopleforgettingtodo something and also improve follow throughwith aparticular action. Forinstance,inatrialrunbyBITandtheCourtsService,sendingpeopletextmessageprompts topay theircourt fines tendaysbefore abailiff visit tripledpaymentrates.33

Inthehealthcarecontext,reminders-providerorpatientpromptsandcomputeralerts - have been one of themost effective tools for influencing clinicians’behaviour.34Somestudiessuggestedthatreminderscanachieveanaverageeffectofaround13%35ordeliverinthemajorityofcases.36

BIT has also found that reminders can influence healthcare professionals. Forinstance, sending weekly reminders to pharmacists significantly improved theengagementwith theSummaryCareRecords (whichprovidea fasteraccess tobasic clinical information about patients).37 In a recent trial with NHS Digital,introducing‘redalerts’forlowcapacityhospitalsontheE-referralportalsizeablyreducedthereferraltohospitalswithlongwaitingtimes.38

Personalisation,SalienceandMessenger

GPshave to take inandprocessa lotof informationonadailybasis.Withourlimited cognitive resources, people struggle to notice and remember what isimportantwhichcanaffecttheirabilitytomakeeffectivechoices.Wearemorelikelytofailtodosomethingwhenweworkunderpressureoraredistractedbyotherpressingcallsonourattention.39

Therefore, we need to find ways to make a particular message stand out.Behavioural science suggests that personalisation of communication - using aperson’sname,40referringtoherprofession41oraddingapersonalisedmessage-

Inthistrial,wesentthreeletters:anadvancedfeedbackletter,asocialnormfeedbackletter,includingareminderstickeranda

reminderletter.

9

canhelpattractpeople’sattention.Asimilarapproachistomakeamessagemoresalient by using eye-catching envelopes, ink42 or stamps.43 Using an influentialmessenger can also increase the effect of a message. People tend to weighinformationdifferentlydependingontheauthorityof44ortheirrelationshipwith45amessengerwhocanheavilyinfluencewhethertheyfollowthroughwithanaction.

Inthistrial,wepersonalisedourlettersbyusingGPsandpracticemanagers’namesandbyreferringtotheirlocalarea.

WeincreasedsalienceoflettersthroughhighlightingthecostsandbenefitsofurgentGPreferral.SomelocalGPcancerleads

endorsedourinterventiontoincreaseitslegitimacy.

Thetrial

InpartnershipwithGMHSCP,wedesignedandranastepped-wedgefieldtrial.Inthetrial,wecontactedthe250GPpracticesinGreaterManchesterwithanurgentGP referral ratebelow theEnglandaverage.Thesepracticeswere identifiedasunder-referrersusingPHE’sage-and-sexstandardisedreferralratioin2015/16.46

We tested the impact of a series of three behaviourally-informed letters onpractices’ urgent GP referral rates compared to business-as-usual (nocommunication).Themainelementofour interventionwastoprovideGPswithsocial norm feedback on their performance in urgent referral in 2015/2016,relativetootherGreaterManchesterpractices.Tobettertailorourmessage,wealsosplitoursampletotwogroups:lowreferrers(practicesinthebottom30%ofreferrers nationwide, N=121) and medium referrers (practices in the 20% ofreferrersclosertotheEnglandaverage,N=123).

Wesentlettersto250GreaterManchesterGPpractices.Wesentthreetreatmentlettersoverthecourseofamonth(oneevery5-7days)toonenamedGPandthepracticemanager. SoeachGPpractice received two copiesof the anticipatedfeedbackletter,thesocialnormfeedbackletterandthereminderletter(eachGPpracticereceivedatotalofsixletters,seeFigure1.andTable1.below).Intotalwesent1,500letters.

10

Figure1.Trialdesign

Intervention

Wesummarisetheinterventionsbelow.ThefullmaterialcanbefoundinAppendix1.

Table1:InterventionDetails

Condition Detailsofintervention

Anticipatedfeedbackletter

ThefirstletterinformedGPpracticesthattheywouldsoonreceivefeedbackontheirperformance.Theanticipationandconciseinformationonsocialimportanceoftheurgentreferralwasmeanttoincreasesalienceoftheissue.TheletterwassignedbytheGPCancerLeadforthegivenCCG.

"Almosthalfofpeoplewhodevelopcancerarediagnosedlate.InthenextfewweeksIwillbesendingyoufeedbackonhowyourpractice’stwoweekwait(suspectedcancer)referralratecomparestootherpracticesinyourarea."

Socialnormfeedbackletter,

Thesecondlettercontainedthefeedback,basedonthePHEage-and-sexstandardizedtwoweekwaitcancerreferralratiosin2015-2016.

11

includingremindersticker

● GPPracticeswithlowreferralrate

"Thevastmajority(70%)ofpracticesinGreaterManchesterhaveahighertwoweekwaitcancerreferralratethanyours.GPtwoweekwaitcancerreferralsarecriticalfordetectingcancerearly."

● GPpracticeswithmediumreferralrate

"ManypracticesinGreaterManchesterhaveahighertwoweekwaitcancerreferralratethanyours.GPtwoweekwaitcancerreferralsarecriticalfordetectingcancerearly."Theletteralsoincludedareminderstickerandanumberofsuggestionstohelpincreaseappropriatereferrals(suchasdiscussingthereferralwithcolleaguesorcompletingonlinetrainingmodulesonGatewayC).ItwassignedbytheGPCancerLeadforthegivenCCG.

Socialnormreminderletter

ThereminderletterwasdesignedtoreinforcethefeedbackmessagebyagaincontrastingaGPpractice’sperformancewiththereferralbehaviourofthemajorityofpractices.ThisletterwasalsosignedbytheGPCancerLeadforthegivenCCG:

● GPPracticeswithlowreferralrate"Recentlywewrotetoyoutotellyouthatyourpracticehadalowertwoweekwaitcancerreferralratethanthevastmajority(70%)ofpracticesinGreaterManchester."

● GPpracticeswithmediumreferralrate"RecentlywewrotetoyoutotellyouthatyourpracticehadalowertwoweekwaitcancerreferralratethanmanypracticesinGreaterManchester."

Wealsotoldpracticesthatthe‘dataonreferralwillcontinuetobecollected’hopingthattheongoingmonitoringwouldencouragethemtoact.

Experimentaldesign

We designed and conducted a stepped-wedge randomised trial with 250 GPpractices across 10CCGs47 inGreaterManchesterwith the urgentGP referralbelowtheEnglandaverage.Asourbaselinemeasureofurgentreferral,weusedPHE’s age-sex-standardised referral ratio in 2015/16.48 To ensure a balancedrandomisation,wesampledourtreatmentgroupstobeassimilaraspossible,so

12

astocontainGPpracticesfromalldifferentCCGsandboththelowandmediumreferrers.

GP practices were randomly allocated to receive the letter intervention in aparticularmonth between August 2017 and January 2018 (they all received iteventually).Priortotheirtreatmentmonth,non-treatedpracticesthatwerenotreceiving any intervention served as our control group. Of the 250 originallyrandomised practices, we received referral and demographic data for 244practiceswhichareourfinalanalysedsampleinthistrial.49

BITchosethistrialdesignbecausethesmallsamplesizemeantwewouldnothavebeenabletodetecttheeffectofan interventionusingastandardrandomised-controlled design. Instead, we eventually included all practices and onlyrandomisedtheorderinwhichtheyreceivedtheintervention.Anotheradvantageofstepped-wedgetrialisthatitallowsalltrialparticipantstoreceiveapotentiallybeneficial intervention. Inaway, it ismoresimilartoreal-worldpolicyroll-out,whenapolicymaybepilotedfirstbuteveryonewillreceiveiteventually.

Table2.Illustrationofstepped-wedgetrialdesign

Baseline Month1 Month2 Month3 Month4 Month5 Month6

40GPpractices

Control Treat Treat Treat Treat Treat Treat

40GPpractices

Control Control Treat Treat Treat Treat Treat

40GPpractices

Control Control Control Treat Treat Treat Treat

40GPpractices

Control Control Control Control Treat Treat Treat

40GPpractices

Control Control Control Control Control Treat Treat

40GPpractices

Control Control Control Control Control Control Treat

InAugust2017,westartedthetrialbysendingtreatmentletterstoleadGPsandpracticemanagersof41practices.Toevaluatetheintervention’simpact,wecouldthen compare the urgent referral in treated practices to the remaining 203practicesinoursample.Everymonth,about40practicesreceivedthelettersuntilallhadbeentreated(seeTable2).

13

Table3.UrgentGPreferralfinalstepped-wedge

Month0 Month1 Month2 Month3 Month4 Month5 Month6

Control 244 203 162 119 79 39 0

Treatment 0 41 82 125 165 205 244

Total 244 244 244 244 244 244 244

14

Results

WefoundastatisticallysignificantpositiveeffectofourinterventionontheurgentGPreferralrate,consistentacrosstwodifferentmeasures.Below,wepresenttheaggregate results forall treatedGreaterManchesterpractices.Givenour smallsamplesize,wecannotlookathowandwhethertheeffectvariesdependingonGPpracticecharacteristicsoracrossdifferentCCGs.

Outcomemeasures

WelookedatthestandardPHEurgentcancerreferralmeasure-thecrudereferralrate-aswellasthenumberofreferralspermonth.Inouranalysis,wealsotookintoaccountthepotentialinfluenceofseasonalityandthelocationinagivenCCG(alsoaproxyfordeprivation).

Togetherwiththeage-and-sexstandardisedratio,thesearecurrentlythemostrobustmethodsofmeasuringpractices’performance in the urgent cancerGPreferral. To calculate the standardised ratio, PHE uses a special internaldemographic dataset which could not be obtained in time to complete thisanalysis.Thedatacanbeproducedadditionallyonrequest.50

Wepresentthereferralintreatedpracticescomparedtoarobustcontrolgroup,composedoftwodatagroups.Asusual forstepped-wedge,weusethecontroldata generated during the trial period - i.e. the urgent cancer referral in GPpracticesjustbeforetheyweretreated.Secondly,wealsocombinedthisdatawithamodelofpractices’referralinpre-trialperiod(January-July2017).ThismorerigorousapproachenablesustogetobservationsontheGPpractices’referraloveralongertimeperiodandhencemorerobustlyexcludethattheeffectisduetoanyseasonal variation. In Annex 2we also include the resultswith just the simplecontrolgroup(observationsjustfromthetrialperiod);wegetessentiallythesameconclusions.

Crudereferralrate

ThecrudereferralrateisthenumberofurgentGPreferralsmultipliedby100,000anddividedbythelistsizeofthepractice.Inotherwords,thismeasuretellsuswhattheincidenceofcancerreferralsisinagivenGPpractice,weightedforitspopulationsize.

Wefoundthatprovidingsocialnormfeedbackincreasedthecrudereferralrateby9.6%,achangestatisticallysignificantatp<0.05level.Afterreceivingtheletters,theunder-referringGPpracticesreferred245patientsper100,000permonth,comparedto223.5peoplebeforethetreatment(Figure2).Inpractice,thismeans

15

thatafterreceivingthefeedback,thetreatedGPsinGreaterManchesterreferred21.5morepeopleper100,000amonththanbeforetheintervention.

Figure2:Treatmenteffectonthecrudecancerreferralratepermonth

We lookedathow thisresultcompares toPHECancerdataand found that thecurrentmonthlyaveragereferralacrossallGreaterManchesterpractices is250per100,000.51Thissuggeststhatourinterventioncouldhavehelpedtoreducethegapbetweenoursampleofunder-referringpracticesandtheaverageperformers.

Weexploredthishypothesis inouranalysisandfoundthatourfeedback lettersweremoreeffectiveforthelowreferrers(practicesinthebottom30%ofreferrersnationwide).GPsfromthelowreferringpracticesreferred17%morepatients(227up from 194per 100,000), achange significant atp <0.01 level (Figure3). Thissuggeststhatouroveralleffectwaslargelyduetothelowperformersimprovingtheir performance52 rather than an increase in referrals by the close-to-the-averageperformers.

16

Figure3:Treatmenteffectbyreferrergroupbymonth

Referralspermonth

Having lookedat thestandardPHEmeasure,wealso lookedatamuchsimplermeasuretounderstandtheaveragemonthlynumberofreferralsperpractice.WefoundthatwhenGPpracticesreceivedthesocialnormfeedback,theyreferred15.5patientspermonth,sooneandhalfmorepeople,comparedto14patientsreferred in the control group (Figure 4.). This represent an increase of 10.7%,statisticallysignificantatp<0.05level.

17

Figure4:TreatmenteffectonthenumberofurgentGPreferralsmadebymonth

Lookingattheeffectovertime,ourresultsacrossthetwomeasuresshowthatGPpractices changed their referral only after they received all three letters.OuranalysissuggeststhattreatedGPscontinuedtorefermorepatientsforsixmonths.Note thatwhilst theeffectcouldpersistsaftersixmonths,wewouldnotknowbecauseweonlyobservedour treatedpractices for themaximumperiodofsixmonths.

Duringthetrialperiod,our244under-referringGPpracticeswereinthetrialforanaverageof3.5months,andreferred1,281additionalpatientsofwhom90couldbeexpectedtobediagnosedwithcancer.53Overall,ifweprovidedthesocialnormfeedbacktoall244practicesinonego,weexpecttheywouldhavereferred2,196morepatientsandwhichequatesto154morecancerpatientsreceivinganearlydiagnosisoverthefollowingsixmonths.54

Potentialsavings

Wealsocalculatedthepotentialannualsavingsthat interventioncouldgeneratefor GreaterManchester and England.We have assumed that all the under-referringGPpracticeswouldreceivethefeedbackonlyonceand increasetheirreferraloverthefollowing6months,consistentwithourtreatmenteffect.

Toestimate the savings for theNHS,weassume, for simplicity’s sake, that thereferredcancerpatientswouldbediagnosedatstage1insteadofstage4.SuchanearlydiagnosiswouldhelptheNHStosave£7,847percancercase,accordingtoourcalculationsbasedonCRUK’sestimations.55

18

LookingatGreaterManchester,ifall244under-referrersreceivedfeedback,theywould identifyanadditional 154cancerpatientsover the following sixmonths.Assuming thesewere diagnosed at stage 1 instead of stage 4,GMHSCPwouldannuallysave£1,207,218.Thecostofsendingthelettersduringthetrialwasjust£1,220.

Thereare3,716GPsinEnglandwithbelowaveragereferrals.Ifwescaledupthisintervention to include them all we calculate they would refer 33,444 morepatients.Thiswouldleadto2,542additionalpatientsdiagnosedwithcancerinthenextsixmonths.56Assumingtheywerebeingdiagnosedatstage1ratherthanstage4,theNHScouldsaveatleast£19,945,854annually.Thisonlylooksatdirectcostsavings to the NHS, the gains in terms of improved outcomes from earliertreatmentwouldbevaluedsubstantiallyhigher.

Conclusion

Inthislargefieldexperimentwith244GPpractices,wefoundthatprovidingsocialnormfeedbacktounder-referringpracticesiseffectiveatincreasingthenumberof patients referred for suspected cancer. The crude urgent GP referral ratestatistically significantly increased by 9.6%, thanks to our set of threebehaviourally-informed letters. This increase was mostly due to changes inbehaviour by the low referrers (i.e. practices in the bottom 30% of referrersnationwide)whosent17%morepatientstoseespecialists,comparedtothepre-trialperiod. Inotherwords,our feedbackprovisionhelped to reduce the gapbetweentheunder-referrersandthemediumreferrersinGreaterManchester,bybringingthemclosertotheEnglandaverage.

Following the social norm feedback, a given GP practicemonthly referred onaverage 1.5morepatientwith suspectedcancerover the six followingmonths.Duringthetrial,the244GPpracticesreferred1,281additionalpatientsofwhom90arelikelytohavebeendiagnosedwithcancer.Annually,thiswouldamounttoalmost 2,200more referrals and 154 cancer patients diagnosed earlier inGM,assumingthelettersweresentonceperyear.IfwescaledthisinterventionuptoallEngland’sunder-referringGPpractices,weestimatethatGPswouldannuallyurgentlyrefer33,444morepatientsanddiagnose2,542additionalpatientsearlier.For comparison, thismeans thatGPswould diagnosemore new cancer casesannuallythanallthenewcervicalcancercasesinagivenyear.

Thefindingsofthistrialareencouraginginseveralways.Firstly,weshowedthatalow-intensityletterinterventioncanbeacost-effectiveapproachtomeaningfullyimprove theurgent cancer referral rateofGPpracticesperformingbelow theEnglandaverage.Secondly, the fact that the treatedpracticesonly referred 1.5

19

morepersonpermonthonaverageseemsproportionate,giventhattheaveragepracticeonlysees27cancercasesperyear.57ThissuggestsourletterspromptedGPstoconsidertheappropriatenessofurgentreferralinmarginalcases,withouta largehike inreferralcreatingamajorpressureonsecondarycare.Finally,theearlysuccessofourletterinterventionsuggeststhatthereisapotentialtoexploreother approaches to improve early cancer detection through the urgent GPreferral.Promisingareasforfurtherresearchcouldbe: interventionsseekingtoembed socialnorm feedback intoNHS systems,other timelier interventions toencourageGPs to improve their referral (suchas real-timecomputeralerts)orinterventionsencouragingpatientswithsuspectedcancertovisittheirGPearlyon.

Inthemeantime,werecommendthattheGMHSCPexploreswaystoembedtheprovisionofsocialnormfeedbacktothebottom30%ofunder-referringpractices,tosustaintheeffectofintervention.Weestimatethatthisinterventioncouldsaveupto£900,000annually,58thankstolowercostsofandbettersurvivalwithearlycancertreatment.

20

Annex1.Treatmentletters

Letter1–Allpractices

21

Letter2-Lowreferralpractices

22

Letter2-Mediumreferralpractices

23

Letter3-Lowreferralpractices

24

Letter3-Mediumreferralpractices

25

Annex2.Resultswithjustthesimplecontrolgroup

Figure5.Cruderateforpracticesbytreatment,usingthesimplecontrolgroup(observationsjustfromthetrialperiod)

Figure6.UrgentGPreferralspermonthforpracticesbytreatment,usingthesimplecontrolgroup(observationsjustfromthetrialperiod)

26

Annex3.Literature review

IntroductionThisreviewwasproducedbytheBehaviouralInsightsTeam(BIT)fortheGreaterManchester Health and Social Care Partnership (GMHSCP). The aim of thisliterature review is to draw upon existing research to identify ways in whichbehavioural insightsmaybeappliedto improveGPcancerreferrals.Thisreviewdiscusses behavioural biases and their potential solutions looking at how toencourageGPstoincreasethenumberofreferredpatients.

Theprinciplesdiscussed inthispaperaregroupedaccordingtotheBehaviouralInsightsTeam’s“EAST” framework,59which isbasedonBIT’sownworkand thewideracademicliterature.Theframeworksuggeststhatifyouwanttoencourageabehaviour,youshouldmakeitEasy,Attractive,SocialandTimely:

1.MakeitEasy-Harnessthepowerofdefaults-Reducethe‘hasslefactor’oftakingupaservice-Simplifymessages

2.MakeitAttractive-Attractattentionandcutthroughthe‘noise’ofeverydaylife-Designrewardsandsanctionsformaximumeffect

3.MakeitSocial-Showthatmostpeopleperformthedesiredbehaviour.-Usethepowerofnetworks-Encouragepeopletomakeacommitmenttoothers

4.MakeitTimely-Promptpeoplewhentheyarelikelytobemostreceptive.-Considertheimmediatecostsandbenefits-Helppeopleplantheirresponsetoevents

Background

Late cancer detection is amajor cause of poorer survival rates in theUK.60 Ifpatientswerediagnosedatanearlierstageupto10,000deathscouldbeavoidedevery year.61 Approximately one-third of the population will develop a cancerduringtheirlifetime.Earlysymptomidentificationusuallyhappensinprimarycare,but there remains a substantial variation in the urgent referral rates amongstdifferent Clinical Commissioning Groups (CCGs) in England62 (six-fold inScotland),63 which is not fully explained by the variation in population andprevalenceofriskfactors.InGreaterManchester,thetotalreferralrateremainsbelow the national average (2,742 vs 2,975 per 100,000 individuals as of

27

2015/2016).Thismeansthat inGreaterManchesterapproximately6,50064moreindividualswouldneedtohavebeenreferredin2015/16,ifreferralsweretohavebeenthesameastheEnglandaverage.65

Toaddressthe issuearound latecancerdetection,NICEhaverecentlyupdatedtheir guidelines forGPs relating to the threshold formaking suspected cancerreferralstosecondarycare.66InordertoincreasethenumberofGPreferrals,thecancer‘riskthreshold’basedonthepositivepredictivevalue(PPV)67ofsymptomshasbeen lowered from5% to3%. Inpractice, thismeansaGPshouldreferallpatientswhohaveonlya3%riskofcancer.For illustration,GPsshouldreferallpatients aged over 40 with unexplained weight loss and abdominal pain.Unfortunatelydata isnotavailableonwhetherGPpracticesarereferring in linewiththeNICEguidelines,giventhecomplexnatureofidentifyingwhetherpatientshave a 3% risk of cancer. At the least GreaterManchester should aim to bereferringattheEnglandaverage(whichwouldmeanmakingapproximately6,500morereferralsayear),howeverreferringinlinewiththeNICEguidelinescouldbealotlargerthanthis.

Evidencesuggeststhaturgentreferralscanbeanefficienttoolofearlydetectionasthereisapositivecorrelationbetweenthetwoweekwait(2ww)referralsandthedetectionrate. InacohortstudyofEnglishpractices,higherurgentreferralrateswerefoundtobeassociatedwithlowercancermortality:theGPpracticeswith high urgent referrals experienced a 4% decrease inmortality, while thepractices with low urgent referral rates had 7% increase, compared to theintermediatereferralpractices.68Themostsignificantpositiveimpactisexpectedto come from increased identification of “lung and pleural cancers, uppergastrointestinaltractcancerandlowergastrointestinaltractcancer”.69Yet,higherdemandforcancerassessmentandtreatmentcouldputastrainonthesecondarycaresystem,withoutfurthersupportandinvestmentinitsresources.Forinstance,ithasbeen suggested that thenumberofendoscopiesmay risebyup to 79%between2013/14and2019/20,whichraisesquestionsaboutthelocalcapacitytoactuallydeliveronthenewrecommendations.

Inordertoachievetheincreasedrateofreferralandcancerassessment,itwillbekeytoinfluenceGPsbehaviour.Thesecondimportantaspectisensuringpatientsattendthesubsequentsecondarycareappointment.Lastly,thereferralprocesscouldprovidea ‘teachablemoment’ toencourage individualswhoexperienceacancerscare toadopthealthybehaviours.GMHSCP is interested inall threeofthese stages and this document outlines the behavioural biases and potentialsolutionsforeachofthese.

28

The table below provides an overview of behavioural insights approaches thatcouldbeappliedatthesethreepointsinthecancerreferralprocess.Moredetailsontheseapproachesareoutlinedthroughoutthisreview.

Table1:OverviewofbehaviouralinsightsforGPcancerreferral

SECTION1:IncreasingGPcancerreferrals

Actualcompliancewiththenewguidelinesoncancerreferralsdependsonseveralfactors,bothpracticalandbehavioural,eitherattheGPlevelorduetofailuresofthe administrative system. This section sets out these barriers, followed bypotentialsolutionsofhow toovercome these issues,usingbehaviouralscienceapproaches.

Barrierstoincreasingreferrals

Clinicalguidelinesaregraduallyupdatedbasedonthenewestmedicalresearch.Yet,evidencesuggeststhatthetransferoftheseguidelinesintoanactualpracticeishaphazard.Evidence fromdifferentcountriessuggests thatguidelinesareonaveragefollowedinonly70%ofdecisions.70,71Asforcancerreferrals,compliancevariessubstantiallyacrossGPs.72

Themost frequently cited reasons forphysician’s low compliancewith clinicalguidelinesarethefollowing:

● Informationoverloadorawareness issues:73,74,75,76On topof a substantialworkload,GPsareexpected tokeepup todatewith lotsofnew/revised

InfluencingGPbehaviour

Easy ClearguidelinemessagingRemove‘frictioncosts’ofdiagnostics

Attractive AttractattentiontotheguidelinesLossframing

Social Auditandpeer-feedbackNamingandshaming

Timely RemindersImplementationintentions

29

recommendations,oftenunderseveretimeconstraint.Lackof familiaritycanbeabiggerissueforolderprofessionals,whomaytendtorelyontheirexpertisetoagreaterextent.7778

● Lowmotivationandinertia:79,80Especiallywithpreventiveguidelines,thereareoftenfewincentivesand/orpenaltiesinplacetoinfluencebehaviour81andphysiciansmaytendtodefaulttofollowtheirexistingroutines.8283

● Acceptanceandbeliefsabouttherelevanceofnewrecommendationsandtheir efficacy in improving patients’ well-being:84,85,86 Lack of trust inresearch underpinning the guidelines and disagreement with,87,88 orconfusionabout, theaimsofauthoritiesare important reasons fornon-compliance.Relatedtothisissomepatients’perceptionthatfollowingtheguidelineisunnecessaryintheircase.89,90

● Organisationalconstraints:91Lackof timeand insufficientstaff resourcescan be constraints,92,93,94 as well as the negative attitudes of peers orsuperiors95 (inthiscase,the (non)receptivenessofsecondarycareduetopressureontheirservices).96

Beyond the physician-centred barriers identified in the behavioural literature,BIT’sexploratoryworkundertakenaspartofthisproject,suggestsfurtherbarrierswhichhamperchangemaystemfromthesystemitself.Thesebarriersinclude:

● Conflictingmessages:97ThenewguidelinespromptGPs to increase theircancerreferralrates,againstthebackgroundoftheoverarchingprioritytoreduce referrals in general to secondary care.98We believe this mixedmessaging could cause serious confusion. If combined with a lack ofawarenessaboutthenewcancerreferralguidelines,thesystemmayinduceGPstodefaultintoagenerallylowlevelofreferrals.

● Secondarycareattitudes: Inparallel, secondarycare isunder significantstrain whichmight give the impression that higher cancer referrals areinappropriate.Insomesituationswhenreferralshavebeenmade,GPshavebeenreprimanded,patientssentbacktoprimarycareorputonawaitinglist,insteadofbeingseenthroughtheurgentreferralprocess).99

ApplicationofbehaviouralinsightsTakingthesebarriersintoaccount,BIThavegeneratedthefollowingideasofhowtoincreasecancerreferralrates.ThesehavedrawnoftheacademicliteratureandarestructuredbyBIT’sEASTframework.100

1. MAKINGITEASYSimplifymessages

30

Thehumancapacitytoprocess101andretain102informationislimited,thisabilityisreducedevenmorewhensomeonehashadtomakemultipledecisions.103GPshavealotofmedicalandadministrativetaskstoattendto-hence,guidelinesneedtobeasclearaspossible tobemosteffective.104Behaviouralresearchshows thatmore specific andprecise guidelines are twice as likely tobe implemented asguidelines that are vague and long.105Nevertheless, evenwith clear guidelines,passive dissemination alone has often proved to be ineffective.106,107 A furthercomplicationisthatthenewNICEguidelineservesasamerebenchmarkforaquitecomplexevaluationofapatient’shealth.Inotherwords,whilehavinga3%cancer‘riskthreshold’cansoundstraightforward,GPsneedtomakeacomplexdiagnosis,inorder to translate theguidance intopractice.Whilstdata isnotavailableonwhetherGPpracticesare referring in linewith theNICEguidelines, inGreaterManchester approximately 6,500108more individualswould need to have beenreferredin2015/26tobeinlinewithEngland’sreferralaverage.

So,whilecommunicationabouttheguidelinecanbesimplified,itsimplicationsforactualclinicaldecision-makingarecomplexandeffortful.109Simplifyingboththecontent (for instance, by including easily actionable steps) and access to theguidelines, to increase awareness and understanding, is the crucial first steptowardschangingGPdecision-makingbehaviour.

Reduce‘friction’

The behavioural literature shows that ‘friction’ factors which make a taskmarginally more effortful have a disproportionate impact on whether peoplecomplete an action.110 Inotherwords,processeswhich are a lotof ‘hassle’ tocomplete are less likely to getdone. Theway inwhich choices arepresented,knownasthechoicearchitecture111canhelptoeliminatethese‘frictioncosts’.Forinstance,presentingachoicewithaclearlypredefinedandlimitedsetofoptions112greatlyincreasesthelikelihoodofmakinganoptimaldecision.

Experimental evidence shows that this approach works well for clinician’sbehaviour:clearer,partlypre-filledprescriptionformswereshowntosignificantlyreduce medication error,113 while reducing the number of checkbox optionsimprovedlaboratorytestordering.114Reducinghasslebystreamliningdecisionscanalsohelpclinicianswithcomplexdiagnosticdecisions.Forexample,using‘decisiontrees’significantlyimprovedclinicians’accuracyinrecognisingsymptomsofheartattacks or cerebellar stroke.115 Medical checklists, with clear steps and easilyobservabletargets,helptocheckcompliancewithrecommendedpracticesandavoiderror.116Mostoftenusedinpreoperative117andoperativecare118orICU,119,120,121thechecklistshavebeenshowntoeffectivelyimprovediagnosticdecision-makinginprimarycare122too.

31

CompliancewiththeGPcancerreferralguidelinescouldbeenhancedbyprovidingariskfactorchecklist123(forexample,listingspecificsymptomsandriskfactorsforeach cancer type) or decision tree. Fortunately, some tools to simplifyinterpretationoftheguidelinesalreadyexist:suchasQCancer,124analgorithmthatprovidesacancerriskscorebasedonpatientsymptomsandriskfactors,andTheCancerSymptomMaps125whichindicatespossiblecancersbasedonsymptomsandriskfactors.Aninterventioncouldfocusonencouraginggreateruseofresourcessuchasthese.

2.MAKINGITATTRACTIVE PersonalisationandSalience

OvercomingGPspotential lackofawarenessabout thedetailsof thenewNICEguidelines requires their attention to be captured. The behavioural literaturerecommendsthatthiscanbeachievedthroughpersonalisationofcommunicationandincreasingsalienceofthedesiredaction’scostsandbenefits.126

Personalisationofcommunicationhasprovedsuccessful in increasing responserates in a range of situations. Ways of doing this have included: adding ahandwritten note (increasing survey response rate127 or sewerbillpayments128),using a person’s name in text messages (increasing court fines payment),129including a customisedmessage (increasingdoctors’ repaymentofoutstandingliabilities).130Othermethodsofattractingattentionhavealsobeenshown tobeeffective, such as using more attractive envelopes, ink131 or visually salientstamps.132Make communications toGPs personalised (e.g. through using theirnameandtailoringtheinformationtothem,theirGPpracticeorlocalarea)andmakingkeyinformationsalient(e.g.throughtheuseofboldingandcolours).

Framing

Thebehaviouralscienceliteraturesuggeststhatpeopleareinfluencedbythewayinwhichamessage is framed, forexamplewhether themessagehighlights thelossesorgainsofanaction.133

Peopletendtoexperiencealossastwiceaspowerfulasanequivalentgain134–aconcept known as loss aversion. 135 For instance, when hypothetical medicaldecisions were framed in positive terms (probability of living), as opposed tonegative terms (probability of dying), people were more likely to opt for aprocedure.136

In the healthcare context, there is an important distinction between illnessdetecting behaviours, for which a loss-framing is more effective; and healthenhancingbehaviour,whichismoreresponsivetogain-framing.137Inotherwords,

32

while the gain-framing helps stimulate preventive behaviours (such as physicalexercise,138 smoking cessation,139 sunscreen use140 or even some surgicalprocedures141), loss-framing is more effective at encouraging early detectionbehaviours(suchasthebreast,142cervical143orHIVscreening144).

AsGPcancerreferralsbelongamongdetectivebehaviours,usinga loss-framedmessage in communicationwithGPsmightbemore effective. Inpractice, theinformationaboutcancerreferraloutcomesshouldbepresentedintermsofthehumanloss(forpatients)orthefinancialloss(fortheNHS)thatGPscanavoidiflatecancerdetectionisprevented.Forinstance,GPscouldbemadeawarethatlow-referringpracticestendtohavehighermortalityrates.

ThenewcancerreferralNICEguidelinereducestheriskthresholdratiofrom5%to 3%, with the aim of increasing the number of patients receiving furtherassessment.ThischangeneedstobeunderstoodwithinthecontextoftheNHSCancerStrategy,whichestimatesthatthevolumeofsomecancertestsneedstogrowby70-80%overthenextfiveyears.145Asaresultofthelowerriskthreshold,GPsareexpectedtorefermorepeoplewithalowerlikelihoodofhavingcancer.Asaresult,theshareofpositivecancercasesamongreferrals(theconversionrate–theprecisionofreferral)isautomaticallysettofall.146ThisinturnmaynegativelyaffectGPsperceptionaboutthegainsandlossesassociatedwiththeirreferrals.Theymaygaintheimpressionthattheeffectivenessofincreasedreferralislow,while the higher flow of patients further encumbers the secondary care. Tocounteract this perception,GPs should be informed that the higher-referringpracticesachievebettermortalityoutcomesthanthelow-referringones.147

Physicians’ reluctance to increase referralsmay stem from their focus on thepresentcostsandunderestimationof thecostsof late treatment.148Hence,wecouldhighlightthecostsoflatecancerdetection:boththehumancosts,intermsofthelowerlikelihoodofsuccessfultreatmentandsurvival;andfinancialcoststotheNHS formoreextensive treatment.Previously, thiskindofa ‘costmessage’method was successful at encouraging physicians to reduce the number ofunnecessary diagnostic tests (reducing demand by 32%without compromisingpatientoutcomes),149ortoreducethenumberofmissedhospitalappointments(seebelowfordetail).150Asfortheurgentreferral,GPscouldbeinformedabouttheestimatedcostsofnon-referral(e.g.computedasadifferencebetweenthecostsofearlyandlatedetectedcancer).Arguably,thiswouldbeaweakermessagecomparedtoamoreexplicitcostofanextratestoramissedappointment.Thatbeingsaid,acostmessagestressingthehumanandfinancialcostsassociatedwithlatedetectedcancercouldbeconsidered.

33

3.MAKINGITSOCIALSocialnormsandfeedback

Feedback has been shown to be a reliable tool for influencing physicians’behaviour.151 The effect size seems to depend on the context, themethod offeedbackandthespecificbehaviourbeingtargeted.Feedbackhasbeenshowntobemoreeffectiveinthecaseoflowbaselineperformance,lowcomplexityofthetarget behaviour (e.g. test or drug ordering),152 high feedback intensity (e.g.repetitive feedback inboth verbalandwritten formats,and froman influentialmessenger - supervisor or colleague) and in combination with otherinterventions.153

Humansaresocialanimalsandthebehaviourofourpeersheavilyinfluencesouractions.Recentbehaviouralresearchpointsatthepotentialofusingadherencetosocialnormstoupscalethetraditionalformsoffeedback.Socialnormmessages,especially‘descriptivenorms’-whichmakepeopleawareofwhatthemajorityofpeoplearedoing-andhighlightwhenpeopleareoutlierstothesocialnormhaveprovedhighlyeffectiveinencouragingpeopletoadjusttheirbehaviour.154

BIThassuccessfullyusedsocialnormfeedbackanumberoftimes,mostrecentlytoalterphysicianbehaviour.UnnecessaryantibioticprescriptionsdecreasedwhenGPswere informedthattheirpracticewasprescribingatahigherratethanthemajority of other practices in their NHS Local Area Team.155 Peer-comparisonfeedback hasproved effective in several other experiments aimed at reducingprescription156orexcessivediagnostictestuse.157Inthiscontext,providingsocialnorm feedback to GPs on how their cancer referral rates differ from otherpracticescouldhelptochangebehaviour.

IntheGPreferralcontext,feedbackinterventionshavedeliveredsizeableresults.Thatbeingsaid,theseinterventionsusuallyfocusondecreasingreferral:providingregularfeedbackonbenchmarkreferralratesreducedreferralsby20%,158whilemore intensiveweeklypeer reviewdiscussions reduced referral ratesbyup to50%.159Itshouldbenotedthatpeerfeedbackhastobecarefullydesigned,mindfulofpotential‘backfireeffects’ifagivenphysician’sperformanceisalreadybetterthanaverage.TopreventaGPwho isalreadyreferringatthecorrectratefromworsening their performance, a further ‘injunctive norm’ could be used - e.g.acknowledgingandpraisingtheindividualforbetterthanaverageperformance.

‘NamingandShaming’

Publicly available socialnorm feedback,whichcan sometimesbepresented as‘namingandshaming’hasbeenproventobeeffectiveinsomehealthcarecontexts.For instance,usingpublicreportingto inflictreputationaldamagetoagroupof

34

poorlyperformingUShospitalsmotivatedqualityimprovementofservices.160Keyfor‘namingandshaming’toworkistodesignarankingsystemthatcanbeeasilyunderstood, iswidelydisseminatedandaccessibletothepublic,withfollow-upreportsonperformanceevolutionover time.161The ‘starsystem’, introduced torankhealthcareprovidersintheUK,managedtosatisfyallthesecriteria162-andhelped to substantially reduce hospitalwaiting times in England, compared toWaleswheresuchasystemwasnotputinplace.163Similarly,stoppingpublicationoftheschool leaguetables inWales leadtoasignificantdeterioration inschoolperformance.164Accessibilityofthiskindoffeedbackcanbeincreasedthankstothespreadofdigitalplatforms.

Thebehaviouralrationalebehind‘namingandshaming’istwo-fold.165Firstly,thepublicexercisesdirectpressuretochange,aspoorrankingincreasesthesalienceofproviders’accountability.Secondly,‘namingandshaming’worksonaprivate,internal level, too.166Research shows that ordinal rankingwithin a comparisongroupcanbemoreimportantthanrelativeorabsoluteperformance.167,168‘Namingandshaming’amplifiestheimportanceofrankingbyaspotlighteffect.169Itpointsatbehaviour inconsistentwith individual’sgeneralself-perceptionasmoralandhonest170 - or in this particular context, their perception of themselves ashardworkingpublicservants.

Yet,‘namingandshaming’canbackfireinthesituationswhentheadjustmentofperformance is not fully within the discretion of a ‘shamed’ individual.Unfortunately,GPreferralmayfallintothiscategory,becauseofseveralfactors.For instance, lownumberofcancercasesandrandomcasemix inagivenyearmakeitdifficulttoattributealowreferralsolelytoGPperformance.Patientswithsomebackgrounds(socio-economic,cultural)maybelesslikelytovisittheirGP,whilethesecondarycare incertainareasmaybe lessreceptiveandmorevocalabouttheurgentlyreferredpatients.

ForGPcancerreferrals,‘namingandshaming’maynotbeanappropriatemethodofbehaviourchangeifadjustmentofperformanceisnotfullywithinthediscretionof individual (e.g.due to the randomcasemix)and itmayhaveademoralisingeffect. Arguably, this kind of reasoning might have previously led to theabandonmentofpublishingofGPpractice leveldataoncancer referrals (eventhoughitstillremainsaccessible171).

4.MAKINGITTIMELY

Timing isoftenoverlooked,yet it isavitalaspectofthepolicy-makingprocess.People’s behaviour can vary significantly depending on when information isreceived.172Hence,timelypromptscouldhelpenhanceGPs’compliance,ifusedat

35

therightmoment.Asdiscussedpreviously,primarycarephysiciansarelikelytobeoverwhelmedbyinformation,potentiallylosingtrackofthechangeinguidelines-orsimply,nothavingitontheirmindatkeymoments.

Reminders

Thebehaviouralliteraturehavefoundtimelypromptstobeeffectiveinanumberof different situations: advance-warning text messages to improve court finepaymentbeforebailiffvisits,173usingthemomentindividualsarewritingtheirwillto increasecharitablegiving174ormilestonemoments to increaseattendanceofcancerscreening,forexample.175Reminders-viaaletter,atextoraphonecall-aregenerallysuccessful inhelpingpeople toremember toperformaparticularaction.

In the medical context, evidence shows that reminders - provider prompts,computer-assisted treatmentplans,andpatientprompts -are the singlemosteffective tool forchangingclinicians’behaviour:with themedian improvementeffectof13%,176theyalteredclinicianbehaviourin75%oftrialsaccordingtoonereview.177Inonetrial,researchersplacedpaperstickersintheshapeofabreastonthepatients’recordswithspace forGPstorecordmammographyreferralsandcompletion.ThestickeractedasareminderforGPsandturnedouttobemoreeffective in increasing referrals than both individual feedback and financialincentives.178Computer-basedremindersforclinicianshavealsoprovedeffectivefor various clinical decisions: preventive care (e.g. vaccination),179 obesitytreatment,180diabetesmanagement181andcancerscreening.182

For test ordering andmedication dosing, the size of improvement in processadherence ranges from 3-6%,183 if the reminders do not directly engage thephysician.But reminderswhich requireanactivechoice184,185canachieve threetimestheeffectofapassivelydisplayedmessagealone.186Aswellasrequiringanactivechoice,thepowerofreminderscanbeenhancedbytheuseofdeadlines.Deadlinesintroduceatimelyeffectandincreasetheperceptionofatimelimitedopportunity,andhenceenhancethedesirabilityofanaction. InthecaseofGPreferral,GPscouldbe instructedto improvetheirreferralnumbersbyacertaindeadline.

Reminders could be incorporated into the GP cancer referral process. Forexample,throughtheintroductionoftimelypromptsontheGPITsystem(EMIS)orthroughletterremindershighlightinghowtheirreferralratecomparestothenationalaverageandtheNICEobjectiveofincreasedreferrals.

Implementationintentions

36

Thereisoftenanimportantgapbetweenpeoplesettinganintentionandfulfillingtheirgoals.Behaviouralresearchsuggeststhatcreatingaconcreteplanofactionthatspecifieswhen,whereandwhichactionsneedtobetakencanhelpbridgethisgapbetweenintentionsandoutcomes.187Inpractice,thismeanspeopleshouldbeencouraged tocomeupwithaspecificplanofaction, ideallywrittendown.Evidenceshowsthatthiskindofadvancedplan-makingsignificantlyimprovestheuptake of influenza vaccinations188 and the attendance of colonoscopyappointments.189Anotherapproachistoidentifyspecificbarriersbeforehandandprepareaplantoovercomethem.Forinstance,ifoneobstacletoahealthydietisatendencytoimpulsivelychooseamoreunhealthyoptionforlunchinthecanteen,wecanplantopre-orderourlunchesinadvance.

For repetitive tasks, implementation intentions can help to substitute a priorundesirable habitwith a new habit.190 This creates an automated sequence ofbehaviour,activatedbysituationalcuesintheenvironment,whichtriggersthe‘if-then’implementationintention.Forinstance,animplementationintentionmaybe‘ifIamhungry,Iwilleatanapple’whichmaybedesignedtoovercomeahabitofeatingachocolatebartosatisfyhunger.

Todate, implementation intentionshavenot yetbeen tested in influencingGPbehaviour,buttheycouldbeusedaspartofareminderorfeedbackintervention.WithrespecttocompliancewiththeNICEguidelines,implementationintentionscouldhelpachievetheformationofnewhabits.

Summary

ThissectionsummarisesthebehaviouralinsightsapproachesoutlinedthroughoutthisliteraturereviewthatcouldbeappliedtoincreasingGPcancerreferrals.

GPsareexposedtoasubstantialamountof information,aswellasaheavyandcomplexworkload.Hence,aclearandintelligibleguidelineismuchmorelikelytogetnoticedandimplemented.‘Frictioncosts’couldbereducedbybreakingdowncomplexprocesses,narrowingdown theoptions tobe takenorcreatinghandychecklistsanddecisiontreestoguidephysiciansthroughoutthedecisionmakingprocessofwhethertomakeareferral.Thechoicearchitectureofthee-Referralsystem’sinterfacecouldalsobeadaptedtomaketheprocessintuitiveandeasy,howeverthefeasibilityofthiswouldneedtobeexplored.Lastly,weneedtobemindfulof thecurrentbehaviouraldefault favouring lowGP referralwhichmaypartiallybiasthecancerreferralratedownwards.

Simplified,personalisedand visuallycompellingcommunicationabout theNICEguidelines could be a first step for achieving increased referrals. Financialincentives are an effective tool, but have the potential to crowd out intrinsic

37

motivation or increase referrals irrespective of actual need. Loss framing hasproved successful with illness detection behaviours, hence it could enhancecompliance,forexamplebymakingthehumanandfinancialcostof latecancerdetectionsalient.

Feedbackcanbeareliableinstrumentforinfluencingphysicians’behaviour,evenif themagnitude of the effect canbe small.When lettingGPs know how theyperformcomparedtotheirpeers,apowerfultendencytowardsconformitymightpromptthemtoadjusttheirbehaviourtowardsthesocialnorm.

Reminders remain a reliable tool in achieving physician behavioural change.Automatically-generated computerised reminders are improving and becomingincreasinglysophisticatedandinsightful.GPcancerreferralscouldbeincreasedbyatargeteduseoftimelyprompts,especiallyiftheyrequireanactivechoice-i.e. physician engagement with the reminder or a deadline. Alongside this,implementation intentions could help to bridge GPs intention to follow theguidelineswiththeactualimplementation.

38

Annex4.Technicalappendix

Randomisation

ThetrialperiodlastedfromAugust2017untilJanuary2018.ForeachGPpractice,werandomlyselectedamonthduringthisperiodwhentheywouldreceivetheinterventionletters.Practicesareconsidered“treated”duringthismonththattheyreceivelettersandallfollowingmonths.

ThisrandomisationwasstratifiedbyCCGandwhetherornotthepracticewasinthebottom30%ofreferrersnationwide.

Balancechecks

Eachmonthduringthetrial,newpracticesreceivedthelettersandmovedfromcontroltotreatment.Wethereforecheckforimbalancesbetweentreatmentandcontrolonpracticelistcharacteristicsforeachmonthduringthetrialperiod.Thesecharacteristicsare:thenumberofpatientsonthepractice’slist;theproportionofpatientsonthelistthatareaged65andover;andtheproportionofpatientsonthelistthatarefemale.

Thetablebelowpresentsp-valuesfromt-testsofcomparisonsbetweentreatmentandcontrolpracticesforeachmonthofourtrialperiod.Wedefineimbalancesbetweentreatmentandcontrolasanyt-testwithap-valueoflessthan0.10.

WefindoneimbalanceinAugust(onagecomposition),andoneimbalanceinNovember(ongendercomposition).However,sincewemake15comparisonsbetweentreatmentandcontrol,wewouldexpecttheretobeatleastonesignificantdifferenceduetorandomchance.Moreover,thefactthateachpracticeappearsinbothtreatmentandcontrolatsomepointduringourtrialhelpsalleviatepotentialconcernsaboutimbalance.

Wearethereforeconfidentthatourrandomisationhasproducedgroupsthatarefairlycomparable.Toensureourresultsarerobusttothesefactorspresentedbelow,wecontrolfortheminouranalysis(seeAnalysismodelsectionbelow).

Month Listsize Proportion65+ Proportionfemale

August p=0.92 p=0.09 p=0.21

September p=0.88 p=0.79 p=0.14

October p=0.48 p=0.44 p=0.15

39

November p=0.17 p=0.66 p=0.05

December p=0.89 p=0.87 p=0.17

Analysismodel

Instepped-wedgetrials,onedrawbackisthattreatmentdeliveryisconfoundedwithtime.Inotherwords,earliermonthslikeAugustandSeptemberhaveveryfewpracticesthathavebeentreated,whilelatermonthshavemanypracticesthathavebeentreated.Iftherearetimetrendswhichaffectouroutcome,andifwedonotaccountforthesetimetrends,thetreatmenteffectweestimatemaybebiased.

Weaddresstheissueoftheconfoundingoftreatmentwithtimebycontrollingformonthfixedeffects.Thiscontrolsfortheprevailingconditionsatthetimetoaccountforthecorrelationbetweentreatmentandtime.Thewayitcontrolsforthesedifferencesisquiteflexible:othertechniques,likecontrollingfortimetrends,requiremoreassumptionsaboutthefunctionalformoftherelationshipbetweentheoutcomeandtime,andmaynotaccountforseasonaleffectswell.

Forourprimaryanalysis(outcome=crudereferralrate),weuseOrdinaryLeastSquaresregression.Ourmodeltakesthefollowingform:

Where:

𝑅𝑅"#isathecruderateforGPpracticeiatmontht;

𝑇"#isabinarytreatmentindicator,equaltooneiftheindividualGPpracticei received the intervention during or before month t (equal to zerootherwise).

𝐴"istheproportionofthepracticelistagedover65(forpracticei)

𝑆"istheproportionofthepracticelistthatisfemale(forpracticei)

𝑀isavectorofmonthdummyvariables(oneforeachmonth)

𝐶𝐶𝐺"isavectorofCCGdummyvariables(oneforeachCCG)

𝐸"isabinaryvariablecapturingwhetherpracticeifellintothebottom30%

!!"# = &# +()*"# + (+," + (-." + /0 + 112"3 +(45" + 6"#

40

ofGPpracticesintermsofreferrals1

𝛼#istheconstant,and

𝑒"#istheerrorterm.(Forstandarderrors,weuseheteroskedasticity-robuststandarderrors).

Foroursecondaryanalysis(outcome=referralspermonth),weuseaPoisson

regression.Asexploratoryanalysis,weuseanegativebinomialregressionmodel.

Inthesespecifications,weaddacontinuousvariabletotheequationstated

abovetoaccountforthelistsize(e.g.howmanypeopleareregisteredattheGP).

Data

Inouranalysis,wehaveoneobservationperpracticepermonth.

Allregressionsincludedatafromthepre-trialmonthsofJanuary–July2017.Weincludethisdatainordertoimproveourstatisticalpower.

Inallgraphs,wepresentthecontrolmeanfromthetrialperiod(e.g.August2017

–January2018).

1Notethatpracticesreceivedaslightlydifferentletterdependingonwheretheyfellwithinthiscategory;seeAnnex1formoreinformation.

41

Endnotes

1IncisiveHealth&CRUK(2014).Savinglives,avertingcosts.2NCRAS(2016).Routestodiagnosis.Availableathttp://www.ncin.org.uk/publications/routes_to_diagnosis3IncisiveHealth&CRUK(2014).Savinglives,avertingcosts.4PHE(2018).CancerServices:two-weekwaitreferrals.https://fingertips.phe.org.uk/profile/cancerservices/data#page/0/gid/1938133085/pat/46/par/E39000037/ati/152/are/E380000165Baili,P.,DiSalvo,F.,Marcos-Gragera,R.,Siesling,S.,Mallone,S.,Santaquilani,M.,...&Zielonke,N.(2015).Ageandcasemix-standardisedsurvivalforallcancerpatientsinEurope1999–2007:resultsofEUROCARE-5,apopulation-basedstudy.Europeanjournalofcancer,51(15),2120-2129.;Allemani,C.,Weir,H.K.,Carreira,H.,Harewood,R.,Spika,D.,Wang,X.S.,...&Marcos-Gragera,R.(2015).Globalsurveillanceofcancersurvival1995–2009:analysisofindividualdatafor25676887patientsfrom279population-basedregistriesin67countries(CONCORD-2).TheLancet,385(9972),977-1010.6Alsoknownasthetwo-weekwaitreferralorsuspectedcancerreferral.7NCRAS(2016).Routestodiagnosis.Availableathttp://www.ncin.org.uk/publications/routes_to_diagnosis8Baughan,P.,Keatings,J.,&O’Neill,B.(2011).Urgentsuspectedcancerreferralsfromgeneralpractice:auditofcompliancewithguidelinesandreferraloutcomes.BrJGenPract,61(592),e700-e706.9PHE(2018).CancerServices:two-weekwaitreferrals.https://fingertips.phe.org.uk/profile/cancerservices/data#page/0/gid/1938133085/pat/46/par/E39000037/ati/152/are/E3800001610NICEguidelines(June2015).NG12:Suspectedcancer:Recognitionandreferral.11Differentsymptomsareassignedapositivepredictivevalue–apercentageriskofcancer.TheGuidelineDevelopmentGroup(GDG)decidedonusinga3%PPVthreshold.Inpractice,thismeansthatapatientwithsymptomsaddingtoacumulative3%riskofcancershouldbereferredviatwo-weekurgentreferral.(basedonNICEsuspectedcancerreferralguidelines(2016)).12Møller,H.,Gildea,C.,Meechan,D.,Rubin,G.,Round,T.,&Vedsted,P.(2015).UseoftheEnglishurgentreferralpathwayforsuspectedcancerandmortalityinpatientswithcancer:cohortstudy.BMJ,351,h5102.13-11%latebreastcancercases,-4%boweland-5%latelungcancercases,BITcalculationsbasedonMaclean,R.,Jeffreys,M.,Ives,A.,Jones,T.,Verne,J.,&Ben-Shlomo,Y.(2015).Primarycarecharacteristicsandstageofcanceratdiagnosisusingdatafromthenationalcancerregistrationservice,qualityoutcomesframeworkandgeneralpracticeinformation.BMCcancer,15(1),500.14IndependentCancerTaskforce(2015).Achievingworld-classcanceroutcomes,AStrategyforEngland2015-2016.http://www.cancerresearchuk.org/sites/default/files/achieving_world-Forinstance,someestimationssuggestedthatthenumberofendoscopiesmayrisebyupto79%by2019/20inCRUK(2016).PracticalperspectivesonimplementingtheNICEGuidelineSuspectedcancer:recognitionandreferral.https://www.cancerresearchuk.org/sites/default/files/nice_implementation_roundtable_report.pdf15Saillour-Glenisson,F.,&Michel,P.(2003).Individualandcollectivefacilitatorsofandbarrierstotheuseofclinicalpracticeguidelinesbyphysicians:aliteraturereview.Revued’epidemiologieetdesantepublique,51(1Pt1),65-80.

42

16Spallek,H.,Song,M.,Polk,D.E.,Bekhuis,T.,Frantsve-Hawley,J.,&Aravamudhan,K.(2010).BarrierstoImplementingEvidence-BasedClinicalGuidelines:ASurveyofEarlyAdopters.JournalofEvidenceBasedDentalPractice,10(4),195-206.17Francke,A.L.,Smit,M.C.,Veer,A.J.,&Mistiaen,P.(2008).Factorsinfluencingtheimplementationofclinicalguidelinesforhealthcareprofessionals:Asystematicmeta-review.BMCMedicalInformaticsandDecisionMaking,8(1).18Ibid.19Round,T.(2015,November/December).Gatekeepingandreferralmanagement:Iscancerdifferent?https://www.bjfm.co.uk/gatekeeping-and-referral-management-is-cancer-different.aspx20Matthews-Kings,A.(2015,October1).GPpracticesoffered‘ethicallyquestionable’incentivestocuturgentcancerreferrals.RetrievedNovember29,2016,fromhttp://www.pulsetoday.co.uk/news/commissioning/commissioning-topics/referrals/gp-practices-offered-payments-to-cut-urgent-cancer-referrals/20030100.fullarticle21Price,C.(2016,February17).NICEtwo-weekcanceradvice’isdelayingdiagnoses’,claimsurgeons.Retrievedfromhttp://www.pulsetoday.co.uk/clinical/cancer/nice-two-week-cancer-advice-is-delaying-diagnoses-claim-surgeons/20031149.fullarticle22Imison,C.,&Naylor,C.(2010).Referralmanagement.Lessonsforsuccess.London:KingsFund.23Grol,R.,&Grimshaw,J.(2003).Frombestevidencetobestpractice:effectiveimplementationofchangeinpatients’care.TheLancet,362(9391),1225-1230.24Jamtvedt,G.,Young,J.M.,Kristoffersen,D.T.,O’Brien,M.A.,&Oxman,A.D.(2006).Auditandfeedback:effectsonprofessionalpracticeandhealthcareoutcomes.CochraneDatabaseSystRev,2(2).25Cialdini,R.B.andTrost,M.R.(1998),‘Socialinfluence:Socialnorms,conformityandcompliance’,inGilbert,D.,Fiske,S.,Lindzey,G.(eds)TheHandbookofSocialPsychology,4thedn.NewYork:OxfordUniversityPress,pp.151-9226Hallsworth,M.,List,J.,Metcalfe,R.,&amp;Vlaev,I.(2014).Thebehavioralistastaxcollector:Usingnaturalfieldexperimentstoenhancetaxcompliance(No.w20007).NationalBureauofEconomicResearch.27TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.18.28Hallsworth,M.,List,J.A.,Metcalfe,R.D.,&Vlaev,I.(2017).Thebehavioralistastaxcollector:Usingnaturalfieldexperimentstoenhancetaxcompliance.JournalofPublicEconomics,148,14-31.29Hallsworth,M.,List,J.A.,Metcalfe,R.D.,&Vlaev,I.(2017).Thebehavioralistastaxcollector:Usingnaturalfieldexperimentstoenhancetaxcompliance.JournalofPublicEconomics,148,14-31.30Hallsworth,M.,Chadborn,T.,Sallis,A.,Sanders,M.,Berry,D.,Greaves,F.,...Davies,S.C.(2016).Provisionofsocialnormfeedbacktohighprescribersofantibioticsingeneralpractice:Apragmaticnationalrandomisedcontrolledtrial.TheLancet,387(10029),1743-1752.31Steele,M.,Bess,D.,Franse,V.,&Graber,S.(1989,June).CostEffectivenessofTwoInterventionsforReducingOutpatientPrescribingCosts.PubMed,23(6),497-500.32Winkens,R.,Pop,P.,Bugter-Maessen,A.,Grol,R.,Kester,A.,Beusmans,G.,&Knottnerus,J.(1995).Randomisedcontrolledtrialofroutineindividualfeedbacktoimproverationalityandreducenumbersoftestrequests.TheLancet,345(8948),498-502.33Haynes,L.C.,Green,D.P.,Gallagher,R.,John,P.,&Torgerson,D.J.(2013).Collectionofdelinquentfines:Anadaptiverandomizedtrialtoassesstheeffectivenessofalternativetextmessages.JournalofPolicyAnalysisandManagement,32(4),718-730.34Mostofian,F.,Ruban,C.,Simunovic,N.,&Bhandari,M.(2015).Changingphysicianbehavior:whatworks.AmJManagCare,21(1),75-84.;Chauhan,B.F.,Jeyaraman,M.,Mann,A.S.,Lys,J.,Skidmore,B.,Sibley,K.M.,...&Zarychanksi,R.(2017).Behaviorchangeinterventionsandpolicies

43

influencingprimaryhealthcareprofessionals’practice—anoverviewofreviews.ImplementationScience,12(1),3.35Grimshaw,J.M.,Eccles,M.P.,Walker,A.E.,&Thomas,R.E.(2002).Changingphysicians’behavior:whatworksandthoughtsongettingmorethingstowork.JournalofContinuingEducationintheHealthProfessions,22(4),237-243.36Grol,R.,&Grimshaw,J.(2003).Frombestevidencetobestpractice:effectiveimplementationofchangeinpatients’care.TheLancet,362(9391),1225-1230.37TheBehaviouralInsightsTeam(2017).UpdateReport2016-17,p.20.38TheBehaviouralInsightsTeam(2017).UpdateReport2016-17,p.19.39Mullainathan,S.,&Shafir,E.(2013).Scarcity:Whyhavingtoolittlemeanssomuch.Macmillan.40TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.38.41Ibid,p.23.42EdwardsP.,RobertsI.,ClarkeM.etal.(2007).Methodstoincreaseresponseratestopostalquestionnaires.CochraneDatabaseSystematicReview,2.43TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.24.44WebbandSheeran(2006).Doeschangingbehavioralintentionsengenderbehaviourchange?Ameta-analysisoftheexperimentalevidence.PsychologicalBulletin,13,249–268.45Cialdini,R.(2007)Influence:Thepsychologyofpersuasion.NewYork:HarperBusiness,RevisedEdition.46SeeindicatordefinitionsonPHEFingertips.47ThesewereBolton,Bury,Heywood,Middleton&Rochdale,Manchester,Oldham,Salford,Stockport,Tameside&Glossop,TraffordandWrightington,Wigan&Leigh.48SeeindicatordefinitionsonPHEFingertips.Availableat:https://fingertips.phe.org.uk/profile/cancerservices/data#page/9/gid/1938133085/pat/152/par/E38000001/ati/7/are/B8362049Wecannotdeterminewhetherthe5missingpracticesdidnotreferanyoneorwearesimplymissingtheirdata.WehaveonemissingfromeachoftheseCCGs:Central,Tameside&Glossop,Bolton,StockportandBury.Wecouldnotobtainthedemographicdatafor1morepractice.50AnalysisofthedatabyBITwouldrequireadditionalresourceabovethatallocatedforthistrial.51PHECancerdata,GreaterManchester’sannualcrudereferralratein2016/2017of3003dividedby12givesus250permonth.52Weestimatethatabout75%oftheeffectwasdrivenbythelowreferrers(i.e.50%ofoursample).53Ourcalculation:244GPpracticesreferring1.5morepersonforonaverage3.5months,equals244*1.5*3.5,so1,281.Withanaverageconversionrate(i.e.theproportionofpatientsreferredwhoaresubsequentlydiagnosedwithcancer)of7%inGreaterManchester(PHECancerData,2016/2017),thismeanthat90cancercaseswouldbediagnosedearly(1,281*0.07).54Ourcalculation:244GPpracticesreferring1.5morepersoneachmonthoverayear(244*1.5*6)equals2,196additionalpatientsreferred.Withanaverageconversionrate(i.e.theproportionofpatientsreferredwhoaresubsequentlydiagnosedwithcancer)of7%inGreaterManchester(PHECancerData,2016/2017),thismeans154cancercaseswouldbediagnosedearly.55https://www.cancerresearchuk.org/sites/default/files/saving_lives_averting_costs.pdfOurcalculations:Costofstage4cancer(averageofcolon,rectal,lungandovarian):£13,123-Costofstage1cancer(averageofabove):£5,276=Savingsof£7,847percancercase.56Ourcalculation:3716GPpracticesbelowtheEnglandaveragereferring1.5morepersoneachmonthoverayear(3716*1.5*6)equals33,444additionalreferrals.Withanaverageconversionrateof7.6%inEngland(PHECancerData,2016/2017),thisgivesus2,542additionalearlycancerdiagnoses.

44

57Meechan,D.,Gildea,C.,Hollingworth,L.,Richards,M.A.,Riley,D.,&Rubin,G.(2012).Variationinuseofthe2-weekreferralpathwayforsuspectedcancer:across-sectionalanalysis.BrJGenPract,62(602),e590-e597.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3426597/58Ourcalculation:154extrapeoplereferredinGreaterManchesterover6monthswithasavingof£7,847percase,minusthecostofintervention(£1,220),equals(154*7,847)-1,220=1,207,218.Thelowreferrersareresponsiblefor75%ofoureffect,hence1,207,218*0.75equals905,413.59TheBehaviouralInsightsTeam.(2014).EAST:Foursimplewaystoapplybehaviouralinsights.60UrgentGPreferralratesforsuspectedcancer.(n.d.).http://www.ncin.org.uk/publications/data_briefings/gp_referral_rates61DepartmentofHealth.2011.ImprovingOutcomes:aStrategyforCancer.62DepartmentofHealth.2010.DeliveringtheCancerReformStrategy,p.8.63Baughan,P.,Keatings,J.,&O’Neill,B.(2011).Urgentsuspectedcancerreferralsfromgeneralpractice:auditofcompliancewithguidelinesandreferraloutcomes.BrJGenPract,61(592),e700-e706.64TheseareinternalcalculationsundertakenbyBITusing2015/16dataontwo-weekwaitreferralratesforsuspectedcancer:https://fingertips.phe.org.uk/profile/cancerservices/data#page/4/gid/1938133085/pat/46/par/E39000037/ati/19/are/E38000016/iid/91882/age/1/sex/465ItshouldbenotedthatthiscalculationdoesnottakeintoaccountdemographiccharacteristicswithinGreaterManchester.66NICEguidelines(June2015).NG12:Suspectedcancer:Recognitionandreferral.67Differentsymptomsareassignedapositivepredictivevalue–apercentageriskofcancer.TheGuidelineDevelopmentGroup(GDG)decidedonusinga3%PPVthreshold.Inpractice,thismeansthatapatientwithsymptomsaddingtoacumulative3%riskofcancershouldbereferredviatwo-weekurgentreferral.(basedonNICEsuspectedcancerreferralguidelines.(2016))68Møller,H.,Gildea,C.,Meechan,D.,Rubin,G.,Round,T.,&Vedsted,P.(2015).UseoftheEnglishurgentreferralpathwayforsuspectedcancerandmortalityinpatientswithcancer:cohortstudy.69NICEsuspectedcancerreferralguidelines.(2016).http://www.cancerresearchuk.org/health-professional/learning-and-development-tools/nice-cancer-referral-guidelines#NICE170Eccles,M.,Grimshaw,J.,Walker,A.,Johnston,M.,&Pitts,N.(2005).Changingthebehaviorofhealthcareprofessionals:Theuseoftheoryinpromotingtheuptakeofresearchfindings.JournalofClinicalEpidemiology,58(2),107-112.71Grol,R.(2001).SuccessesandFailuresintheImplementationofEvidence-BasedGuidelinesforClinicalPractice.MedicalCare,39.72ThevariationinthenumberofreferralsisthreefoldintheUK(NationalAuditOffice(2010).DeliveringtheCancerReformStrategy,p.8.).Inotherwords,thereferralrateisonlyaround1,640per100,000inthelowestreferringCCGcomparedto4,400per100,000inthehighestreferringCCG(Basedonthe2015/2016dataPublicHealthEnglanddatabase(ttps://fingertips.phe.org.uk/)).73Cabana,M.D.,Rand,C.S.,Powe,N.R.,Wu,A.W.,Wilson,M.H.,Abboud,P.A.C.,&Rubin,H.R.(1999).Whydon’tphysiciansfollowclinicalpracticeguidelines?:Aframeworkforimprovement.Jama,282(15),1458-1465.74Saillour-Glenisson,F.,&Michel,P.(2003).Individualandcollectivefacilitatorsofandbarrierstotheuseofclinicalpracticeguidelinesbyphysicians:aliteraturereview.Revued’epidemiologieetdesantepublique,51(1Pt1),65-80.75Grilli,R.,&Lomas,J.(1994).Evaluatingthemessage:therelationshipbetweencompliancerateandthesubjectofapracticeguideline.Medicalcare,202-213.76Cabana,M.D.,Rand,C.S.,Powe,N.R.,Wu,A.W.,Wilson,M.H.,Abboud,P.A.C.,&Rubin,H.R.(1999).Whydon’tphysiciansfollowclinicalpracticeguidelines?:Aframeworkforimprovement.Jama,282(15),1458-1465.77Davis,D.A.,&Taylor-Vaisey,A.(1997).Translatingguidelinesintopractice:asystematicreviewoftheoreticconcepts,practicalexperienceandresearchevidenceintheadoptionofclinicalpracticeguidelines.CanadianMedicalAssociationJournal,157(4),408-416.

45

78Saillour-Glenisson,F.,&Michel,P.(2003).Individualandcollectivefacilitatorsofandbarrierstotheuseofclinicalpracticeguidelinesbyphysicians:aliteraturereview.Revued’epidemiologieetdesantepublique,51(1Pt1),65-80.79Spallek,H.,Song,M.,Polk,D.E.,Bekhuis,T.,Frantsve-Hawley,J.,&Aravamudhan,K.(2010).BarrierstoImplementingEvidence-BasedClinicalGuidelines:ASurveyofEarlyAdopters.JournalofEvidenceBasedDentalPractice,10(4),195-206.80Schers,H.,Braspenning,J.,Drijver,R.,Wensing,M.,&Grol,R.(2000).Lowbackpainingeneralpractice:reportedmanagementandreasonsfornotadheringtotheguidelinesinTheNetherlands.BrJGenPract,50(457),640-644.81Kao,R.T.(2006).Thechallengesoftransferringevidence-baseddentistryintopractice.JournalofEvidenceBasedDentalPractice,6(1),125-128.82Wensing,M.,Bosch,M.,&Grol,R.(2010).Developingandselectinginterventionsfortranslatingknowledgetoaction.CanadianMedicalAssociationJournal,182(2),E85-E88.83Rochette,A.,Korner-Bitensky,N.,&Thomas,A.(2009).Changingclinicians’habits:Isthisthehiddenchallengetoincreasingbestpractices?.Disabilityandrehabilitation,31(21),1790-1794.84Eccles,M.,Grimshaw,J.,Walker,A.,Johnston,M.,&Pitts,N.(2005).Changingthebehaviorofhealthcareprofessionals:Theuseoftheoryinpromotingtheuptakeofresearchfindings.JournalofClinicalEpidemiology,58(2),107-112.85Francke,A.L.,Smit,M.C.,Veer,A.J.,&Mistiaen,P.(2008).Factorsinfluencingtheimplementationofclinicalguidelinesforhealthcareprofessionals:Asystematicmeta-review.BMCMedicalInformaticsandDecisionMaking,8(1).86Maue,S.K.,Segal,R.,Kimberlin,C.L.,&Lipowski,E.E.(2004).Predictingphysicianguidelinecompliance:anassessmentofmotivatorsandperceivedbarriers.AmJManagCare,10(6),383-391.87Mottur-Pilson,C.,Snow,V.,&Bartlett,K.(2000).Physicianexplanationsforfailingtocomplywith"bestpractices".Effectiveclinicalpractice:ECP,4(5),207-213.88Cabana,M.D.,Rand,C.S.,Powe,N.R.,Wu,A.W.,Wilson,M.H.,Abboud,P.A.C.,&Rubin,H.R.(1999).Whydon’tphysiciansfollowclinicalpracticeguidelines?:Aframeworkforimprovement.Jama,282(15),1458-1465.89Saillour-Glenisson,F.,&Michel,P.(2003).Individualandcollectivefacilitatorsofandbarrierstotheuseofclinicalpracticeguidelinesbyphysicians:aliteraturereview.Revued’epidemiologieetdesantepublique,51(1Pt1),65-80.90Francke,A.L.,Smit,M.C.,Veer,A.J.,&Mistiaen,P.(2008).Factorsinfluencingtheimplementationofclinicalguidelinesforhealthcareprofessionals:Asystematicmeta-review.BMCMedicalInformaticsandDecisionMaking,8(1).91Grol,R.,&Grimshaw,J.(2003).Frombestevidencetobestpractice:effectiveimplementationofchangeinpatients’care.Thelancet,362(9391),1225-1230.92Cabana,M.D.,Rand,C.S.,Powe,N.R.,Wu,A.W.,Wilson,M.H.,Abboud,P.A.C.,&Rubin,H.R.(1999).Whydon’tphysiciansfollowclinicalpracticeguidelines?:Aframeworkforimprovement.Jama,282(15),1458-1465.93Francke,A.L.,Smit,M.C.,deVeer,A.J.,&Mistiaen,P.(2008).Factorsinfluencingtheimplementationofclinicalguidelinesforhealthcareprofessionals:asystematicmeta-review.BMCmedicalinformaticsanddecisionmaking,8(1),1.94Saillour-Glenisson,F.,&Michel,P.(2003).Individualandcollectivefacilitatorsofandbarrierstotheuseofclinicalpracticeguidelinesbyphysicians:aliteraturereview.Revued’epidemiologieetdesantepublique,51(1Pt1),65-80.95Davis,D.A.,&Taylor-Vaisey,A.(1997).Translatingguidelinesintopractice:asystematicreviewoftheoreticconcepts,practicalexperienceandresearchevidenceintheadoptionofclinicalpracticeguidelines.CanadianMedicalAssociationJournal,157(4),408-416.96Price,C.(2014,December18).UrgentGPcancerreferralsbouncedbackashospitalstrytomanagedemand.RetrievedNovember29,2016,fromhttp://www.pulsetoday.co.uk/clinical/cancer/urgent-gp-cancer-referrals-bounced-back-as-hospitals-try-to-manage-demand/20008775.fullarticle

46

97Round,T.(2015,November/December).Gatekeepingandreferralmanagement:Iscancerdifferent?https://www.bjfm.co.uk/gatekeeping-and-referral-management-is-cancer-different.aspx98Matthews-Kings,A.(2015,October1).GPpracticesoffered‘ethicallyquestionable’incentivestocuturgentcancerreferrals.RetrievedNovember29,2016,fromhttp://www.pulsetoday.co.uk/news/commissioning/commissioning-topics/referrals/gp-practices-offered-payments-to-cut-urgent-cancer-referrals/20030100.fullarticle99Price,C.(2016,February17).NICEtwo-weekcanceradvice’isdelayingdiagnoses’,claimsurgeons.Retrievedfromhttp://www.pulsetoday.co.uk/clinical/cancer/nice-two-week-cancer-advice-is-delaying-diagnoses-claim-surgeons/20031149.fullarticle100TheBehaviouralInsightsTeam.(2014).EAST:Foursimplewaystoapplybehaviouralinsights.101Halford,G.S.,Baker,R.,McCredden,J.E.,&Bain,J.D.(2005).Howmanyvariablescanhumansprocess?.Psychologicalscience,16(1),70-76.102Miller,G.A.(1956).Themagicalnumberseven,plusorminustwo:Somelimitsonourcapacityforprocessinginformation.Psychologicalreview,63(2),81.103Rice,T.(2013).Thebehavioraleconomicsofhealthandhealthcare.Annualreviewofpublichealth,34,431-447.104Francke,A.L.,Smit,M.C.,deVeer,A.J.,&Mistiaen,P.(2008).Factorsinfluencingtheimplementationofclinicalguidelinesforhealthcareprofessionals:asystematicmeta-review.BMCmedicalinformaticsanddecisionmaking,8(1),1.105Grol,R.,Dalhuijsen,J.,Thomas,S.,Rutten,G.,&Mokkink,H.(1998).Attributesofclinicalguidelinesthatinfluenceuseofguidelinesingeneralpractice:observationalstudy.Bmj,317(7162),858-861.106WrightJ,HarrisonS,McGeorgeM,PattersonC,RussellI,RussellD,SmallN,TaylorM,WalshM,WarrenE,YoungJ(2006).‘Improvingthemanagementandreferralofpatientswithtransientischaemicattacks:achangestrategyforahealthcommunity’.QualityandSafetyinHealthCare,vol15,no1,pp9–12.107AkbariA,MayhewA,Al-AlawiMA,GrimshawJ,WinkensR,GlidewellE,PritchardC,ThomasR,FraserC(2008).‘Interventionstoimproveoutpatientreferralsfromprimarycaretosecondarycare(CochraneReview)’.CochraneDatabaseofSystematicReviews,issue4,articleCD005471.108TheseareinternalcalculationsundertakenbyBITusing2015/16dataontwo-weekwaitreferralratesforsuspectedcancer:https://fingertips.phe.org.uk/profile/cancerservices/data#page/4/gid/1938133085/pat/46/par/E39000037/ati/19/are/E38000016/iid/91882/age/1/sex/4109TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights.110TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.12.111Thaler,R.H.,&Sunstein,C.(2008).Nudge:Improvingdecisionsabouthealth,wealth,andhappiness.NewHaven,CT:YaleUniversityPress.112Iyengar,S.,andKamenicaE.(2010).ChoiceProliferation,simplicityseeking,andassetallocation.JournalofPublicEconomics,94(7-8),530-539.113King,D.,Jabbar,A.,Charani,E.,Bicknell,C.,Wu,Z.,Miller,G.,...&Darzi,A.(2014).Redesigningthe‘choicearchitecture’ofhospitalprescriptioncharts:amixedmethodsstudyincorporatinginsitusimulationtesting.BMJopen,4(12),e005473.114Shalev,V.,Chodick,G.,&Heymann,A.D.(2009).Formatchangeofalaboratorytestorderformaffectsphysicianbehavior.Internationaljournalofmedicalinformatics,78(10),639-644.115Gigerenzer,G.,&Gaissmaier,W.(2011).Heuristicdecisionmaking.Annualreviewofpsychology,62,451-482.116Winters,B.D.,Gurses,A.P.,Lehmann,H.,Sexton,J.B.,Rampersad,C.J.,&Pronovost,P.J.(2009).Clinicalreview:checklists-translatingevidenceintopractice.CriticalCare,13(6),1.117Lingard,L.,Regehr,G.,Orser,B.,Reznick,R.,Baker,G.R.,Doran,D.,...&Whyte,S.(2008).Evaluationofapreoperativechecklistandteambriefingamongsurgeons,nurses,andanesthesiologiststoreducefailuresincommunication.Archivesofsurgery,143(1),12-17.118Verdaasdonk,E.G.G.,Stassen,L.P.S.,Hoffmann,W.F.,VanderElst,M.,&Dankelman,J.(2008).Canastructuredchecklistpreventproblemswithlaparoscopicequipment?.Surgicalendoscopy,22(10),2238-2243.

47

119Byrnes,M.C.,Schuerer,D.J.,Schallom,M.E.,Sona,C.S.,Mazuski,J.E.,Taylor,B.E.,...&Bailey,R.A.(2009).Implementationofamandatorychecklistofprotocolsandobjectivesimprovescompliancewithawiderangeofevidence-basedintensivecareunitpractices.Criticalcaremedicine,37(10),2775-2781.120Pronovost,P.,Needham,D.,Berenholtz,S.,Sinopoli,D.,Chu,H.,Cosgrove,S.,...&Bander,J.(2006).Aninterventiontodecreasecatheter-relatedbloodstreaminfectionsintheICU.NewEnglandJournalofMedicine,355(26),2725-2732.121Pronovost,P.,Berenholtz,S.,Dorman,T.,Lipsett,P.A.,Simmonds,T.,&Haraden,C.(2003).ImprovingcommunicationintheICUusingdailygoals.Journalofcriticalcare,18(2),71-75.122Torr,J.,Iacono,T.,Graham,M.J.,&Galea,J.(2008).Checklistsforgeneralpractitionerdiagnosisofdepressioninadultswithintellectualdisability.JournalofIntellectualDisabilityResearch,52(11),930-941.123Imison,C.,&Naylor,C.(2010).Referralmanagement.Lessonsforsuccess.London:KingsFund.124http://www.cancerresearchuk.org/health-professional/learning-and-development-tools/clinical-decision-support-tools/qcancer125https://d2jsz3umxxetg1.cloudfront.net/126TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.19.127IrishRevenue(2013).“SurveyofSmallandMediumSizedBusinessCustomers”http://www.revenue.ie/en/about/publications/business-survey-2013.pdf128Sweeney,M.,&Phillips,O.(2016,May03).Howcanaletterincreasesewerbillpayments?http://www.behaviouralinsights.co.uk/north-america/how-can-a-letter-increase-sewer-bill-payments/129TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.38.130TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.23.131EdwardsP.,RobertsI.,ClarkeM.etal.(2007).Methodstoincreaseresponseratestopostalquestionnaires.CochraneDatabaseSystematicReview,2.132TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.24.133TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.19.134Kahneman,D.&Tversky,A.(1992)."Advancesinprospecttheory:Cumulativerepresentationofuncertainty".JournalofRiskandUncertainty.5(4):297–323.135Kahneman,D.&Tversky,A.(1984)."Choices,Values,andFrames".AmericanPsychologist.39(4):341–350.136McNeil,B.J.,Pauker,S.G.,SoxJr,H.C.,&Tversky,A.(1982).Ontheelicitationofpreferencesforalternativetherapies.NewEnglandjournalofmedicine,306(21),1259-1262.137Rothman,A.J.,&Salovey,P.(1997).Shapingperceptionstomotivatehealthybehavior:Theroleofmessageframing.PsychologicalBulletin,121(1),3-19.138Robberson,M.R.,&Rogers,R.W.(1988).BeyondFearAppeals:NegativeandPositivePersuasiveAppealstoHealthandSelf-Esteem.JApplSocialPyscholJournalofAppliedSocialPsychology,18(3),277-287.139Schneider,T.R.,Salovey,P.,Pallonen,U.,Mundorf,N.,Smith,N.F.,&Steward,W.T.(2001).Visualandauditorymessageframingeffectsontobaccosmoking.JournalofAppliedSocialPsychology,31,667-682.140Detweiler,J.B.,Bedell,B.T.,Salovey,P.,Pronin,E.,&Rothman,A.J.(1999).Messageframingandsunscreenuse:Gain-framedmessagesmotivatebeach-goers.HealthPsychology,18(2),189-196.141Levin,IrwinP.,SandraK.Schnittjer,andShannonL.Thee(1988),“InformationFramingEffectsinSocialandPersonalDecisions,”JournalofExperimentalSocialPsychology,24(November),520–529.142Banks,S.M.,Salovey,P.,Greener,S.,Rothman,A.J.,Moyer,A.,Beauvais,J.,etal.(1995).Theeffectsofmessageframingonmammographyutilization.HealthPsychology,14,178-184.

48

143Rivers,S.E.,Salovey,P.,Pizarro,D.A.,Pizarro,J.,&Schneider,T.R.(2005).Messageframingandpaptestutilizationamongwomenattendingacommunityhealthclinic.JournalofHealthPsychology,10(1),65-77.144Garcia-Retamero,R.,&Cokely,E.T.(2011).Effectivecommunicationofriskstoyoungadults:usingmessageframingandvisualaidstoincreasecondomuseandSTDscreening.JournalofExperimentalPsychology:Applied,17(3),270.145IndependentCancerTaskforce(2015).Achievingworld-classcancerouctomes,AStrategyforEngland2015-2016.http://www.cancerresearchuk.org/sites/default/files/achieving_world-class_cancer_outcomes_-_a_strategy_for_england_2015-2020.pdf146Meechan,D.,Gildea,C.,Hollingworth,L.,Richards,M.A.,Riley,D.,&Rubin,G.(2012).Variationinuseofthe2-weekreferralpathwayforsuspectedcancer:across-sectionalanalysis.BrJGenPract,62(602),e590-e597.147Møller,H.,Gildea,C.,Meechan,D.,Rubin,G.,Round,T.,&Vedsted,P.(2015).UseoftheEnglishurgentreferralpathwayforsuspectedcancerandmortalityinpatientswithcancer:cohortstudy.148Jacobs,K.,Marcon,G.,&Witt,D.(2004).Costandperformanceinformationfordoctors:Aninternationalcomparison.ManagementAccountingResearch,15(3),337-354.149Fogarty,A.W.,Sturrock,N.,Premji,K.,&Prinsloo,P.(2013).HospitalClinicians’ResponsivenesstoAssayCostFeedback:AProspectiveBlindedControlledInterventionStudy.JAMAInternalMedicineJAMAInternMed,173(17),1654.150Hallsworth,M.,Berry,D.,Sanders,M.,Sallis,A.,King,D.,Vlaev,I.,&Darzi,A.(2015).Correction:StatingAppointmentCostsinSMSRemindersReducesMissedHospitalAppointments:FindingsfromTwoRandomisedControlledTrials.PLOSONE,10(10).151Imison,C.,&Naylor,C.(2010).Referralmanagement.Lessonsforsuccess.London:KingsFund.152Grol,R.,&Grimshaw,J.(2003).Frombestevidencetobestpractice:effectiveimplementationofchangeinpatients’care.Thelancet,362(9391),1225-1230.153Jamtvedt,G.,Young,J.M.,Kristoffersen,D.T.,O’Brien,M.A.,&Oxman,A.D.(2006).Auditandfeedback:effectsonprofessionalpracticeandhealthcareoutcomes.CochraneDatabaseSystRev,2(2).154TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights.155Hallsworth,M.,Chadborn,T.,Sallis,A.,Sanders,M.,Berry,D.,Greaves,F.,...Davies,S.C.(2016).Provisionofsocialnormfeedbacktohighprescribersofantibioticsingeneralpractice:Apragmaticnationalrandomisedcontrolledtrial.TheLancet,387(10029),1743-1752.156Steele,M.,Bess,D.,Franse,V.,&Graber,S.(1989,June).CostEffectivenessofTwoInterventionsforReducingOutpatientPrescribingCosts.PubMed,23(6),497-500.157Winkens,R.,Pop,P.,Bugter-Maessen,A.,Grol,R.,Kester,A.,Beusmans,G.,&Knottnerus,J.(1995).Randomisedcontrolledtrialofroutineindividualfeedbacktoimproverationalityandreducenumbersoftestrequests.TheLancet,345(8948),498-502.158Evans,E.,Aiking,H.,&Edwards,A.(2011).Reducingvariationingeneralpractitionerreferralratesthroughclinicalengagementandpeerreviewofreferrals:aserviceimprovementproject.Qualityinprimarycare,19(4),263-272.159Evans,E.(2009).TheTorfaenreferralevaluationproject.Qualityinprimarycare,17(6),423-429.160Hibbard,J.H.,Stockard,J.,&Tusler,M.(2003).DoesPublicizingHospitalPerformanceStimulateQualityImprovementEfforts?HealthAffairs,22(2),84-94.161Ibid.162Mannion,R.,Davies,H.,&Marshall,M.(2004).Culturesforperformanceinhealthcare.McGraw-HillEducation(UK).163Besley,T.,Bevan,G.,&Burchardi,K.B.(2009).Naming&shaming:TheimpactsofdifferentregimesonhospitalwaitingtimesinEnglandandWales.London:CentreforEconomicPolicyResearch.164Bevan,G.,&Wilson,D.(2013).Does‘namingandshaming’workforschoolsandhospitals?LessonsfromnaturalexperimentsfollowingdevolutioninEnglandandWales.PublicMoney&Management,33(4),245-252.165Oliver,A.(2013).BehaviouralPublicPolicy.CambridgeUniversityPress,p.54.

49

166Oliver,A.(2013).BehaviouralPublicPolicy.CambridgeUniversityPress,p.54.167Boyce,C.J.,Brown,G.D.,&Moore,S.C.(2010).Moneyandhappinessrankofincome,notincome,affectslifesatisfaction.PsychologicalScience,21(4),471-475.168Powdthavee,N.(2009).Howimportantisranktoindividualperceptionofeconomicstanding?Awithin-communityanalysis.TheJournalofEconomicInequality,7(3),225-248.169Gilovich,T.,Medvec,V.H.,&Savitsky,K.(2000).Thespotlighteffectinsocialjudgment:Anegocentricbiasinestimatesofthesalienceofone’sownactionsandappearance.JournalofPersonalityandSocialPsychology,78(02),211–222170Mazar,N.,&Ariely,D.(2006).DishonestyinEverydayLifeandItsPolicyImplications.JournalofPublicPolicy&Marketing,25(1),117-126.171CancerServices:Two-WeekWaitReferrals.(n.d.).RetrievedAugust09,2016,fromhttp://fingertips.phe.org.uk/profile/cancerservices/data#page/3/gid/1938132830/pat/19/par/E38000123/ati/7/are/P84623/iid/91344/age/1/sex/4172TheBehaviouralInsightsTeam.2014.EAST:Foursimplewaystoapplybehaviouralinsights,p.37.173Haynes,L.,Green,D.P.,Gallagher,R.,John.,O.&Torgerson,D.J.(2013).Collectionofdelinquentfines:Anadaptiverandomizedtrialtoaccesstheeffectivenessofalternativetextmessages.174BehaviouralInsightsTeam(2013).“ApplyingBehaviouralInsightstoCharitableGiving”CabinetOffice.175Hoff,G.,&Bretthauer,M.(2008).Appointmentstimedinproximitytoannualmilestonesandcompliancewithscreening:randomisedcontrolledtrial.176Grimshaw,J.M.,Eccles,M.P.,Walker,A.E.,&Thomas,R.E.(2002).Changingphysicians’behavior:whatworksandthoughtsongettingmorethingstowork.JournalofContinuingEducationintheHealthProfessions,22(4),237-243.177Grol,R.,&Grimshaw,J.(2003).Frombestevidencetobestpractice:effectiveimplementationofchangeinpatients’care.TheLancet,362(9391),1225-1230.178Grady,K.E.,Lemkau,J.P.,Lee,N.R.,&Caddell,C.(1997).Enhancingmammographyreferralinprimarycare.Preventivemedicine,26(6),791-800.179Dexheimer,J.W.,Talbot,T.R.,Sanders,D.L.,Rosenbloom,S.T.,&Aronsky,D.(2008).Promptingcliniciansaboutpreventivecaremeasures:asystematicreviewofrandomizedcontrolledtrials.JournaloftheAmericanMedicalInformaticsAssociation,15(3),311-320.180Schriefer,S.P.,Landis,S.E.,Turbow,D.J.,&Patch,S.C.(2009).Effectofacomputerizedbodymassindexpromptondiagnosisandtreatmentofadultobesity.FamMed,41(7),502-7.181Nilasena,D.S.,&Lincoln,M.J.(1995).Acomputer-generatedremindersystemimprovesphysiciancompliancewithdiabetespreventivecareguidelines.InProceedingsoftheAnnualSymposiumonComputerApplicationinMedicalCare(p.640).AmericanMedicalInformaticsAssociation.182Hunt,D.L.,Haynes,R.B.,Hanna,S.E.,&Smith,K.(1998).Effectsofcomputer-basedclinicaldecisionsupportsystemsonphysicianperformanceandpatientoutcomes:asystematicreview.Jama,280(15),1339-1346.183Shojania,K.G.,Jennings,A.,Mayhew,A.,Ramsay,C.R.,Eccles,M.P.,&Grimshaw,J.(2009).Theeffectsofon-screen,pointofcarecomputerremindersonprocessesandoutcomesofcare.TheCochraneLibrary.184Grimshaw,J.M.,&Russel,I.T.(1994).Implementingclinicalpracticeguidelines:canguidelinesbeusedtoimproveclinicalpractice.EffectiveHealthCare,8,1-12.185Steele,A.W.,Eisert,S.,Witter,J.,Lyons,P.,Jones,M.A.,Gabow,P.,&Ortiz,E.(2005).Theeffectofautomatedalertsonproviderorderingbehaviorinanoutpatientsetting.PLoSMed,2(9),e255.186Arditi,C.,Rege-Walther,M.,Wyatt,J.C.,Durieux,P.,&Burnand,B.(2011).Computer-generatedremindersdeliveredonpapertohealthcareprofessionals;effectsonprofessionalpracticeandhealthcareoutcomes.TheCochranedatabaseofsystematicreviews,12,CD001175-CD001175.187Gollwitzer,P.M.(1993).Goalachievement:Theroleofintentions.Europeanreviewofsocialpsychology,4(1),141-185.

50

188Milkman,K.L.,Beshears,J.,Choi,J.J.,Laibson,D.&Madrian,B.C.(2011).Usingimplementationintentionspromptstoenhanceinfluenzavaccinationrates.ProceedingsoftheNationalAcademyofSciences.108(26),Sciences.108(26),10415–10420.189Milkman,K.L.,Beshears,J.,Choi,J.J.,Laibson,D.&Madrian,B.C.(2012).Followingthroughongoodintentions:ThepowerofPlanningPrompts.WorkingPaperNo.17995.190Verplanken,B.,&Faes,S.(1999).Goodintentions,badhabits,andeffectsofformingimplementationintentionsonhealthyeating.EuropeanJournalofSocialPsychologyEur.J.Soc.Psychol.,29(5-6),591-604.