Aug 1, 2007Aug 1, 2007 11

Conducting Cost-Effectiveness Conducting Cost-Effectiveness Analyses of Behavioral Analyses of Behavioral

InterventionsInterventions

Todd H. Wagner, Ph.D.Todd H. Wagner, Ph.D.

Mary K. Goldstein, M.D. Mary K. Goldstein, M.D. 

Aug 1, 2007Aug 1, 2007 22

AcknowledgementsAcknowledgements

Partial funding was through a grant from the Partial funding was through a grant from the National Cancer Institute (2PO1 CA 55112-National Cancer Institute (2PO1 CA 55112-05A1). 05A1).

Dr. Goldstein’s contribution was informed in part Dr. Goldstein’s contribution was informed in part by work conducted with a grant from the by work conducted with a grant from the National Institute on Aging (R01 AG15110).National Institute on Aging (R01 AG15110).

David Meltzer, Jodi Prochaska, Lisa Faulkner, David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants and Stanford University seminar participants provided helpful comments.provided helpful comments.

Aug 1, 2007Aug 1, 2007 33

Outline of the TalkOutline of the Talk

Background on cost-effectiveness analysis Background on cost-effectiveness analysis (CEA)(CEA)

A bias in CEAs for behavioral A bias in CEAs for behavioral interventionsinterventions

How to fix the problemHow to fix the problem ExampleExample Study design considerationStudy design consideration

Aug 1, 2007Aug 1, 2007 44

Economic EvaluationsEconomic Evaluations

Most trials don’t include economic Most trials don’t include economic analysisanalysis

Economic analysis in only 0.2% of 50,000 Economic analysis in only 0.2% of 50,000 trials conducted between 1966-1988trials conducted between 1966-1988– Adams et al. Medical Care 30(3):231-43 Adams et al. Medical Care 30(3):231-43

Economic analysis in randomized clinical Economic analysis in randomized clinical trials is increasingtrials is increasing

Aug 1, 2007Aug 1, 2007 55

Dollars and SenseDollars and Sense

Economic studies inform decisionsEconomic studies inform decisions– formularyformulary– adoption of new technologyadoption of new technology– scope of benefitsscope of benefits– strategies for management of carestrategies for management of care– organization of health care organization of health care

Aug 1, 2007Aug 1, 2007 66

Cost Effectiveness RatioCost Effectiveness Ratio

1 0

1 0

C C

E EICER

Where C1 is the average cost of the intervention groupC0 is the average cost of the control groupE1 is the average effectiveness of the intervention groupE0 is the average effectiveness of the control group

Aug 1, 2007Aug 1, 2007 77

StandardizationStandardization

USPTF guidelines (Gold et al., 1996; USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles)summarized in three JAMA articles)

Drummond et al. (1997)Drummond et al. (1997) Created, along with journal editors, Created, along with journal editors,

standards for bothstandards for both– MethodsMethods– ReportingReporting

Aug 1, 2007Aug 1, 2007 88

Whose Costs?Whose Costs?

Guidelines recommend a societal Guidelines recommend a societal perspective for costsperspective for costs

IncludeInclude– ProviderProvider– PayerPayer– PatientPatient

Aug 1, 2007Aug 1, 2007 99

EffectivenessEffectiveness

Guidelines recommend QALYsGuidelines recommend QALYs

Values both quality and quantity of lifeValues both quality and quantity of life

Each year of life is multiplied by a Each year of life is multiplied by a weighting factor (utility)weighting factor (utility)

Utilities measure the preference of Utilities measure the preference of different health states on a 0-1 scaledifferent health states on a 0-1 scale

Aug 1, 2007Aug 1, 2007 1010

Behavioral InterventionsBehavioral Interventions

CEA framework holds, but caveats…CEA framework holds, but caveats…

– Behavior change is a “slow” processBehavior change is a “slow” process

– Treat many to prevent a fewTreat many to prevent a few

– Use of intermediate outcomes (proxies)Use of intermediate outcomes (proxies)

Aug 1, 2007Aug 1, 2007 1111

Intermediate OutcomesIntermediate Outcomes Outcome is clinically relevant and predicts Outcome is clinically relevant and predicts

mortality or morbiditymortality or morbidity– Receipt of a mammogramReceipt of a mammogram– Substance use abstinenceSubstance use abstinence– Change in dietary fiberChange in dietary fiber

QALYs would require huge and/or very QALYs would require huge and/or very long studieslong studies

Aug 1, 2007Aug 1, 2007 1212

CEA with an Intermediate CEA with an Intermediate OutcomeOutcome

Sufficient for publicationSufficient for publication Hard to interpret ICERHard to interpret ICER

– Can’t easily compare two CEAs with different Can’t easily compare two CEAs with different intermediate outcomesintermediate outcomes

– Can’t compare CEA to other CEA from Can’t compare CEA to other CEA from another clinical areaanother clinical area

Sometimes only feasible approachSometimes only feasible approach

Aug 1, 2007Aug 1, 2007 1313

CEA with QALYsCEA with QALYs

Measure QALYs orMeasure QALYs or Translate intermediate outcome to QALYsTranslate intermediate outcome to QALYs

– Either build a model de novo or use an Either build a model de novo or use an existing modelexisting model

– Requires a lot of resourcesRequires a lot of resources Most useful, but most challengingMost useful, but most challenging

Aug 1, 2007Aug 1, 2007 1414

CEA and Behavior ChangeCEA and Behavior Change

CEA analysts treat behavior change as a CEA analysts treat behavior change as a dichotomous outcomedichotomous outcome

Partial behavior change is not the same as Partial behavior change is not the same as no behavior changeno behavior change

““Getting the person to recognize that they Getting the person to recognize that they have a problem is half the battle.”have a problem is half the battle.”

Aug 1, 2007Aug 1, 2007 1515

What’s Missing? What’s Missing?

Partial behavior change is missing from Partial behavior change is missing from current modelscurrent models– People who progressed in their process of People who progressed in their process of

changing their behavior but did not changing their behavior but did not successfully change their behavior at the end successfully change their behavior at the end of the study (Stages of Change) of the study (Stages of Change)

Aug 1, 2007Aug 1, 2007 1616

Partial Behavior ChangePartial Behavior Change

All behavioral interventions yield some All behavioral interventions yield some partial behavior changepartial behavior change

Amount depends on Amount depends on – the duration of the studythe duration of the study– baseline stage of changebaseline stage of change

Aug 1, 2007Aug 1, 2007 1717

Partial Behavior ChangePartial Behavior Change

Should we and can we value partial Should we and can we value partial behavior change in a CEA?behavior change in a CEA?

Aug 1, 2007Aug 1, 2007 1818

Should We?Should We?

NONO Intentions are not the Intentions are not the

same as observed same as observed behavior changebehavior change

Stage of change is Stage of change is flawedflawed

Too difficultToo difficult

YESYES Behavior change Behavior change

takes timetakes time Unobservable Unobservable

differences existdifferences exist Future benefits are Future benefits are

importantimportant Otherwise favors Otherwise favors

med/surg txmed/surg tx

Aug 1, 2007Aug 1, 2007 1919

Can We?Can We? Behaviors are like “value chains”Behaviors are like “value chains”

– A series of linked processesA series of linked processes– Interventions may be designed to improve a link Interventions may be designed to improve a link

(stage-matched design)(stage-matched design) Matching chemotherapy protocol to cancer stageMatching chemotherapy protocol to cancer stage

– Interventions may have differential effects on Interventions may have differential effects on different linksdifferent links

Stages of Change (TTM) Model by Prochaska Stages of Change (TTM) Model by Prochaska and DiClimenteand DiClimente

Aug 1, 2007Aug 1, 2007 2020

Stages of ChangeStages of Change People progress through successive stages until People progress through successive stages until

they change their behaviorthey change their behavior– PrecontemplationPrecontemplation: no self-recognition of a problematic behavior.: no self-recognition of a problematic behavior.

– ContemplationContemplation: self-recognition of a problem without action.: self-recognition of a problem without action.

– PreparationPreparation: Planning to change behavior soon.: Planning to change behavior soon.

– ActionAction: In the process of change.: In the process of change.

– Maintenance: Maintenance: adherence to the new behavior over time.adherence to the new behavior over time.

Art of behavioral interventions is to achieve Art of behavioral interventions is to achieve action, and hopefully maintenanceaction, and hopefully maintenance

Aug 1, 2007Aug 1, 2007 2121

Stages of Change and CEAStages of Change and CEA

Stages of change is critical for Stages of change is critical for interpreting CEA resultsinterpreting CEA results– Stage of change may be associated with Stage of change may be associated with

receptivity or motivationreceptivity or motivation– Intervention effects can vary by stage of Intervention effects can vary by stage of

changechange– Incremental cost-effectiveness ratio can vary Incremental cost-effectiveness ratio can vary

by stage of changeby stage of change

Aug 1, 2007Aug 1, 2007 2222

Integrating Stages of Change in a Integrating Stages of Change in a CEACEA

Aug 1, 2007Aug 1, 2007 2323

What’s Needed?What’s Needed?

Data on stages of changeData on stages of change

Probability of moving from partial to Probability of moving from partial to successful behavior changesuccessful behavior change– Note: these probabilities are not observed in Note: these probabilities are not observed in

the intervention groupthe intervention group

Aug 1, 2007Aug 1, 2007 2424

Two Step ProcessTwo Step Process Step 1Step 1

– Calculate ICERs by baseline stage of changeCalculate ICERs by baseline stage of change– This alone can provide much more useful and This alone can provide much more useful and

interpretable information for stage-matched interpretable information for stage-matched interventionsinterventions

Step 2Step 2– Estimate probability of people moving into successive Estimate probability of people moving into successive

stages of changestages of change– For example, % moved from precontemplation to For example, % moved from precontemplation to

contemplation and preparationcontemplation and preparation

Aug 1, 2007Aug 1, 2007 2525

Source of the ProbabilitiesSource of the Probabilities

Probabilities from the literature orProbabilities from the literature or The control group (if possible)The control group (if possible)

– Possible when control group gets usual carePossible when control group gets usual care– If 10% of precontemplators in the control If 10% of precontemplators in the control

group changed behavior, this is the probability group changed behavior, this is the probability for the modelfor the model

Aug 1, 2007Aug 1, 2007 2626

Hypothetical ExampleHypothetical Example

Aug 1, 2007Aug 1, 2007 2727

Randomized Controlled TrialRandomized Controlled Trial

2700 participants enrolled in 3 arms2700 participants enrolled in 3 arms– Post card reminder (n=900)Post card reminder (n=900)– Reminder phone call (n=900)Reminder phone call (n=900)– Personal motivational phone call (n=900)Personal motivational phone call (n=900)

Sample from managed care planSample from managed care plan Managed care organization wants to know Managed care organization wants to know

which reminder to usewhich reminder to use

Aug 1, 2007Aug 1, 2007 2828

Cost and EffectivenessCost and EffectivenessAvg Unit

Cost EffectivenessPostcard $3.50 23.6%Reminder call $19.00 34.2%Motivational call $24.00 33.0%

Incremental cost

Incremental Effectiveness ICER

Reminder call vs postcard $15.50 10.6% $147Motivational call vs postcard $20.50 9.3% $220Reminder call vs motivational call -$5.00 1.2% -$409

Aug 1, 2007Aug 1, 2007 2929

Report to MCOReport to MCO

MCO should choose between postcard or MCO should choose between postcard or reminder callreminder call

Aug 1, 2007Aug 1, 2007 3030

Transition ProbabilitiesTransition ProbabilitiesBegan in precontemplationFollow-up Stage PC RC MC

Precontemplation 63.3% 51.7% 46.3%Contemplation 25.0% 29.0% 16.7%Preparation 7.0% 13.3% 21.3%Action 4.7% 6.0% 15.7%

Began in contemplationFollow-up Stage

Contemplation 63.3% 42.7% 45.3%Preparation 7.0% 13.3% 21.3%Action 29.7% 44.0% 33.3%

Began in preparationFollow-up Stage

Preparation 63.3% 47.3% 50.0%Action 36.7% 52.7% 50.0%

Total Action at Follow-up 23.6% 34.2% 33.0%

Aug 1, 2007Aug 1, 2007 3131

Including Partial Behavior Including Partial Behavior ChangeChange

Note motivational call is now most Note motivational call is now most effective overalleffective overall

PC RC MCTotal Action at Follow-up 23.6% 34.2% 33.0%

Estimated action including partial behavior change 35.9% 45.6% 46.7%

Aug 1, 2007Aug 1, 2007 3232

Reminder Call vs postcard ICERBaseline Stage of Change

Precontemplation $384Contemplation $99Preparation $161

Motivational Call vs postcardBaseline Stage of Change

Precontemplation $127Contemplation $246Preparation $256

Motivational Call vs reminder callBaseline Stage of Change

Precontemplation $123Contemplation -$68Preparation -$312

ICERsICERs

No single strategy No single strategy is always is always preferred.preferred.

Motivational call Motivational call is dominated by is dominated by reminder call for reminder call for contemplators and contemplators and those in actionthose in action

Aug 1, 2007Aug 1, 2007 3333

Origin of These IdeasOrigin of These Ideas

Aug 1, 2007Aug 1, 2007 3434

Patient RemindersPatient Reminders

Mailed reminder vs no reminder for Mailed reminder vs no reminder for mammographymammography

A meta analysis of 16 studies A meta analysis of 16 studies – US studies Pooled OR=1.48 (participants US studies Pooled OR=1.48 (participants

come from provider files)come from provider files)

– Aust / NZ studies Pooled OR=5.57 Aust / NZ studies Pooled OR=5.57 (participants from voter lists)(participants from voter lists)

Wagner TH. The effectiveness of mailed patient reminders on mammography screening: a meta-analysis. Am J Prev Med. 1998;14(1):64-70.

Aug 1, 2007Aug 1, 2007 3535

CEA and Stage of ChangeCEA and Stage of Change

Fishman P. et al. Cost-effectiveness of Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. strategies to enhance mammography use. Eff Clin Pract. Eff Clin Pract. 20002000

Compared three alternative methods for Compared three alternative methods for increasing mammography screeningincreasing mammography screening– Reminder postcardReminder postcard– Reminder callReminder call– Motivational callMotivational call

Aug 1, 2007Aug 1, 2007 3636

Reminder postcard

Motivational call

Reminder call

Sample size 590 590 585

Efficacy 36.1% 50.3% 52.6%Women screened 213 297 308Women not screened 377 293 277

Total costs $3,551.80 $15,334.10 $12,413.70Incremental CER dominated $92.12

Fishman’s (2000) resultsFishman’s (2000) results

ICER varied by prior mammography status ICER varied by prior mammography status (i.e., maintenance)(i.e., maintenance)

Aug 1, 2007Aug 1, 2007 3737

Feeling Lucky?Feeling Lucky?

The implications of a CEA should not vary The implications of a CEA should not vary by who enters the trialby who enters the trial

Two randomized trials with same Two randomized trials with same interventionintervention– RCT 1: all participants are in preparationRCT 1: all participants are in preparation– RCT 2: all participants are in RCT 2: all participants are in

precontemplationprecontemplation

Aug 1, 2007Aug 1, 2007 3838

Clinical Trial DesignClinical Trial Design

If the effect of the intervention might vary If the effect of the intervention might vary by stage of changeby stage of change– Qx: Enroll people from all stages?Qx: Enroll people from all stages?– Enroll sufficient numbers in each stageEnroll sufficient numbers in each stage– Protect randomizationProtect randomization

Aug 1, 2007Aug 1, 2007 3939

ConclusionsConclusions

Subgroup analysis by stage of change may Subgroup analysis by stage of change may be critical for interpreting the CEAbe critical for interpreting the CEA

Partial behavior change is important and Partial behavior change is important and can affect interpretationcan affect interpretation

These methods are appropriate for stage-These methods are appropriate for stage-matched studiesmatched studies

Need to consider study design implicationsNeed to consider study design implications