Estimating Outcomes in Decision Analysis Brian Harris MPP Candidate Goldman School of Public Policy...

Estimating Estimating Outcomes in Outcomes in

Decision AnalysisDecision AnalysisBrian HarrisBrian Harris

MPP CandidateMPP Candidate

Goldman School of Public PolicyGoldman School of Public Policy

University of California, BerkeleyUniversity of California, Berkeley

Review—Last LectureReview—Last Lecture

• Formulate an explicit questionFormulate an explicit question• Make a decision treeMake a decision tree• Estimate probabilitiesEstimate probabilities• Estimate utilitiesEstimate utilities• Compute expected utility of Compute expected utility of each brancheach branch• Perform sensitivity analysis Perform sensitivity analysis

Preview—Where We Are Preview—Where We Are GoingGoing

Recall Ms. Brooks and her Recall Ms. Brooks and her aneurysm.aneurysm.

We want to: We want to: • Determine her utility: clip, Determine her utility: clip, and not clipand not clip• Compare incremental utility Compare incremental utility and costand cost• Compare cost-per-unit of Compare cost-per-unit of utility across private and utility across private and public uses of funds.public uses of funds.

Overview – Today’s Overview – Today’s LectureLecture

• OutcomesOutcomes• Measuring UtilitiesMeasuring Utilities• Quality Adjusted Life Years Quality Adjusted Life Years (QALYs)(QALYs)• Discounting Discounting

Quantifying Health Quantifying Health OutcomesOutcomes

• Mortality Mortality • Life YearsLife Years

number of expected years of life number of expected years of life • Significant MorbiditySignificant Morbidity

Paralysis, loss of sightParalysis, loss of sight• Quality Adjusted Life YearsQuality Adjusted Life Years

Expected life years adjusted for the valuation Expected life years adjusted for the valuation of the possible states in each yearof the possible states in each year

• Financial Valuation of these OutcomesFinancial Valuation of these OutcomesCosts to patient, payor, or societyCosts to patient, payor, or societyWillingness to pay to avoid outcomes, obtain Willingness to pay to avoid outcomes, obtain treatment treatment

Health Outcomes – Health Outcomes – MortalityMortality

• MortalityMortalityDeath from disease/accident/procedureDeath from disease/accident/proceduree.g. If Ms. Brooks undergoes surgery, e.g. If Ms. Brooks undergoes surgery, one of the possible outcomes is one of the possible outcomes is mortalitymortality

• Life Years Life Years Calculate an expected value of life Calculate an expected value of life years using a probabilistically weighted years using a probabilistically weighted average of expected life average of expected life e.g. If Ms. Brooks does not undergo e.g. If Ms. Brooks does not undergo surgery, her life expectancy is less than surgery, her life expectancy is less than if she did not have aneurysm, these if she did not have aneurysm, these outcomes are measured in expected life outcomes are measured in expected life yearsyears

Health Outcomes – Health Outcomes – MorbidityMorbidity

• MorbidityMorbiditySome health state that is less than Some health state that is less than perfectperfecte.g. disability from stroke, chronic e.g. disability from stroke, chronic painpain

• Comparison of morbidities Comparison of morbidities Difficult – apples and oranges Difficult – apples and oranges problem problem e.g. which is worse:e.g. which is worse:Blind v. DeafBlind v. DeafDeaf v. ParaplegiaDeaf v. ParaplegiaParaplegia v. BlindParaplegia v. Blind

Outcomes - UtilityOutcomes - Utility

• Utility is the currency of Utility is the currency of outcomesoutcomes• Scaled from 0 to 1Scaled from 0 to 1• Commonly Death = 0Commonly Death = 0• Perfect Health = 1Perfect Health = 1• Sounds good, but how can this Sounds good, but how can this be measured?be measured?

Utility MeasurementUtility Measurement

• Utilities are commonly Utilities are commonly estimated using comparisons to estimated using comparisons to the 0 and 1 anchorsthe 0 and 1 anchors• Methods are:Methods are:• Visual Analog ScaleVisual Analog Scale• Standard GambleStandard Gamble• Time Trade-offTime Trade-off


Clinical Scenario:Clinical Scenario:• Patient in the hospital has infection of Patient in the hospital has infection of the leg the leg • Two options: BKA v. Medical Two options: BKA v. Medical ManagementManagement• BKA – below the knee amputation (1% BKA – below the knee amputation (1% mortality risk)mortality risk)• Medical – 20% chance of infection Medical – 20% chance of infection worseningworsening

If worse, AKA – above the knee If worse, AKA – above the knee amputation (10%amputation (10%

mortality risk) mortality risk) • How can we value these outcomes?How can we value these outcomes?


Visual Analog ScaleVisual Analog Scale

Linear Scale from 0 to 1Linear Scale from 0 to 1

(death 0-----------------------------1.0 cure)(death 0-----------------------------1.0 cure)Where is AKA?Where is AKA?

(death 0-----------------------------1.0 cure)(death 0-----------------------------1.0 cure)Where is BKA?Where is BKA?


Visual Analog ScaleVisual Analog Scale

• Advantages: quantitative, easy to Advantages: quantitative, easy to understand, visual understand, visual

• Disadvantages: may bias values to the Disadvantages: may bias values to the middle, seems middle, seems

disconnected from medical realitydisconnected from medical reality

Utility Utility Measurement--Measurement--Standard GambleStandard Gamble

Method relies on patients choosing Method relies on patients choosing between:between: 1) a certain outcome1) a certain outcome 2) a gamble between a better outcome and a 2) a gamble between a better outcome and a worse outcomeworse outcome

How it works:How it works: Choice 1Choice 1: You live with a : You live with a BKABKA Choice 2Choice 2: The gamble – you might have a : The gamble – you might have a curecure; you ; you

might might diedie..

Goal of method is to find the break-even point. Goal of method is to find the break-even point. What probability of death would you accept to avoid What probability of death would you accept to avoid

living with the BKA? living with the BKA?

Utility Measurement – Utility Measurement – Standard GambleStandard Gamble

Choose BKA?

Yes

No

BKA (intermediate outcome)

Perfect health

Death

Live?

p %

(100-p) %


• Standard gamble measurement involves questioning Standard gamble measurement involves questioning patients to determine the p at which the two outcomes patients to determine the p at which the two outcomes are equivalent are equivalent

• Using expected utilities, the value of p [p = P(cure)] Using expected utilities, the value of p [p = P(cure)] gives the utility:gives the utility: Utility (BKA) x Prob (BKA) = Utility (BKA) x Prob (BKA) = Utility(cure) x (p) + Utility(death) x (1-p)Utility(cure) x (p) + Utility(death) x (1-p) [Note that here, P(BKA) = 1.][Note that here, P(BKA) = 1.]

You can demonstrate that the utility of BKA = p:You can demonstrate that the utility of BKA = p:

Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-p)] p)]

= [1.0 * p + 0 * (1-p)] = p= [1.0 * p + 0 * (1-p)] = p


• In utility estimation utilizing the standard gamble In utility estimation utilizing the standard gamble measurement, using the simple comparison described, measurement, using the simple comparison described, the value of p, the probability of perfect health needed the value of p, the probability of perfect health needed to make the patient indifferent between the two to make the patient indifferent between the two choices is the estimated utility of the outcome.choices is the estimated utility of the outcome.

• Advantages:Advantages: the incorporation of risk into the the incorporation of risk into the model, comparison or choice between different model, comparison or choice between different outcomes.outcomes.

•Disadvantages: Disadvantages: possible nonrealistic choices possible nonrealistic choices patients may be asked to make, the difficulty of patients may be asked to make, the difficulty of understanding the question (especially for non-understanding the question (especially for non-gamblers), and the difficulty for some in understanding gamblers), and the difficulty for some in understanding a math equation.a math equation.

Utility Measurement – Time Utility Measurement – Time Trade-offTrade-off

• Time Trade-offTime Trade-off involves patients choosing involves patients choosing between: between: quality of life vs. length of time alivequality of life vs. length of time alive

We want to determine when patients are equivocal We want to determine when patients are equivocal between choice:between choice: Time A * Utility A = Time B * Utility BTime A * Utility A = Time B * Utility B e.g. -- If you have a life expectancy of 30 years with a e.g. -- If you have a life expectancy of 30 years with a BKA, how much BKA, how much time would you give up to live in your current time would you give up to live in your current state?state?

Would you give up 5 years? 3 years? 1 year?Would you give up 5 years? 3 years? 1 year? 30 years * Utility (BKA) = (30-x) years * 1.030 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing to give up 3 years, that means If you’re willing to give up 3 years, that means the utility of BKA is: the utility of BKA is: = [(30-3)*1/ 30] = 27/30 = 0.9= [(30-3)*1/ 30] = 27/30 = 0.9

Utility Measurement – Time Utility Measurement – Time Trade-offTrade-off

• Time trade off can be used to measure the Time trade off can be used to measure the utility of an outcome in the fashion described. utility of an outcome in the fashion described. Like standard gamble, patients need to Like standard gamble, patients need to understand subtlety of questions being asked.understand subtlety of questions being asked.

• Advantages:Advantages: the simplicity of the choice the simplicity of the choice between different outcomes, consideration of between different outcomes, consideration of long-term outcomes.long-term outcomes.

• Disadvantages: Disadvantages: fails to incorporate risk, fails to incorporate risk, lack of clarity of when time traded occurs, lack of clarity of when time traded occurs, different valuation of time during life, and the different valuation of time during life, and the theoretical lack of realism of the choice.theoretical lack of realism of the choice.

Utility Measurement – Utility Measurement – Health State Excluding Clinical Health State Excluding Clinical

ProblemProblem• 0 to 1 utility scaling is simple and useful for 0 to 1 utility scaling is simple and useful for determining the utility of different outcomes determining the utility of different outcomes for a single patient.for a single patient.

• However, comparisons across patients or However, comparisons across patients or across programs raises the issue of ‘Is a utility across programs raises the issue of ‘Is a utility of 1 really available?’of 1 really available?’

• Consider: (a) polio vaccinations for children, Consider: (a) polio vaccinations for children, and (b) hip replacement surgery for 85-year and (b) hip replacement surgery for 85-year old patients.old patients.

• Typically (on average) the health state of a Typically (on average) the health state of a polio-free 5-year old will be very close to 1, polio-free 5-year old will be very close to 1, whereas the health state of an 85-year old whereas the health state of an 85-year old even with a perfectly functioning hip will even with a perfectly functioning hip will typically be less than one.typically be less than one.

Utility Measurement – ChainingUtility Measurement – Chaining

• The utility of the health state of the 85-year The utility of the health state of the 85-year old can be determined by “chaining”.old can be determined by “chaining”.

• Consider first the clinical situation under Consider first the clinical situation under review: what is the utility, on a 0 to 1 scale, of review: what is the utility, on a 0 to 1 scale, of a hip with reduced function and some pain? a hip with reduced function and some pain? Let us say utility = .8.Let us say utility = .8.

• Consider next the utility of an 85-year old Consider next the utility of an 85-year old with no hip problems, but some other reduced with no hip problems, but some other reduced function or chronic pain. Here, perhaps function or chronic pain. Here, perhaps overall utility = .9.overall utility = .9.

• Total utility for the bad-hip case = .8 * .9 Total utility for the bad-hip case = .8 * .9 = .72= .72 Here, the perfect-hip case utility = 1 * .9 Here, the perfect-hip case utility = 1 * .9 = .9= .9

Utility Measurement – Utility Measurement – Additional InformationAdditional Information

• Multi-Attribute Health Status Multi-Attribute Health Status Classification SystemClassification System

• Developed by Health Utilities, Developed by Health Utilities, Inc.Inc.

•Available at: Available at: http://www.healthutilities.com/overhttp://www.healthutilities.com/overview.htmview.htm

Utilities in Decision Utilities in Decision AnalysisAnalysis

• Now that we have methods to Now that we have methods to estimate utilities, these can be estimate utilities, these can be used in the DAused in the DA• However, our outcomes often However, our outcomes often include both mortality and include both mortality and morbiditymorbidity• Want a way to add in life Want a way to add in life expectancyexpectancy• Quality Adjusted Life-Years Quality Adjusted Life-Years (QALYs)(QALYs)

QALYsQALYs

• QALYs are generally considered QALYs are generally considered the standard unit of comparison the standard unit of comparison for outcomes for outcomes

• QALYs = time (years) x quality QALYs = time (years) x quality (utils)(utils)

• e.g. 30 years after BKA, e.g. 30 years after BKA, util(BKA) = 0.9util(BKA) = 0.9

= 30 x 0.9 = 27 QALYs= 30 x 0.9 = 27 QALYs

QALYsNo aneurysm rupture0.9825

No surgery34.86 Die

Aneurysm rupture 0.450.0175 Survive

0.55

No aneurysm ruptureDifference 1

_ QALYs -2.85 Survive surgery0.902 Die

Aneurysm rupture 0.45Clipping 0 Survive

32.01 0.55Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023Surg morb (disability) 0.075

0.0

Ms. Brooks

17.5

35.0Normal survival

Disability, shorter survival

5.8

Immediate death

Normal survival 35.0

Normal survival

Normal survival

Early death

Early death

35.0

17.5

35.0

QALYsQALYs

QALYsQALYs

Aneurysm ExampleAneurysm Example

• We said life expectancy is reduced by We said life expectancy is reduced by 2/3, so instead of 35, it is = 35 * .333 2/3, so instead of 35, it is = 35 * .333 = 11.67= 11.67

• Here, we have assigned a utility of .5 Here, we have assigned a utility of .5 to surgery-induced disability, so QALYs to surgery-induced disability, so QALYs = =

years * utils = 11.67 * .5 = 5.8years * utils = 11.67 * .5 = 5.8

Outcomes - DiscountingOutcomes - Discounting

• However, are all years However, are all years considered equal?considered equal?

• Consider: Consider: Favorite MealFavorite Meal

Extreme PainExtreme Pain

Lifetime IncomeLifetime Income

Outcomes - DiscountingOutcomes - Discounting• Generally, present > futureGenerally, present > future• One common way to value the One common way to value the different times is discounting different times is discounting • Essentially this year is worth Essentially this year is worth δδ more more than next yearthan next year• δδ is commonly set at 0.03 or 3% is commonly set at 0.03 or 3%• In order to compare values of all In order to compare values of all future times, a calculation, net present future times, a calculation, net present value, is often usedvalue, is often used• NPV = 1 / (1 + NPV = 1 / (1 + δδ))t t Where t is number Where t is number of years in the futureof years in the future

Outcomes - DiscountingOutcomes - Discounting• Aneurysm ExampleAneurysm Example• If utility is 0.5 and life expectancy is 3 If utility is 0.5 and life expectancy is 3 yearsyears• NPV would be: NPV would be: Utility / (1 + Utility / (1 + δδ))t t

• However, when is year 1? Often, since However, when is year 1? Often, since events in year one occur on average events in year one occur on average half way through, we use 0.5 for year 1half way through, we use 0.5 for year 1• NPV = 0.5 / (1.03)NPV = 0.5 / (1.03)0.50.5 + 0.5 / (1.03) + 0.5 / (1.03)1.51.5 + + 0.5 / (1.03)0.5 / (1.03)2.52.5

• NPV = 0.5 * {(1.03)NPV = 0.5 * {(1.03)-0.5 -0.5 + (1.03) + (1.03) -1.5-1.5 + + (1.03) (1.03) -2.5-2.5}}

QALYsdisc

No aneurysm rupture0.9825

No surgery21.37 Die

Aneurysm rupture 0.45

0.0175 Survive0.55

No aneurysm ruptureDifference 1

Δ QALYs -1.63 Survive surgery0.902 Die

Aneurysm rupture 0.45

Clipping 0 Survive19.74 0.55

Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075

Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023

Surg morb (disability) 0.075

0.0

Ms. Brooks

13.3

21.4Normal survival,

worry

Disability, shorter survival

4.8

Immediate death

Normal survival,worry

21.4

Normal survival

Normal survival

Early death,worry

Early death

21.5

13.4

21.5



• Another issue is partial yearsAnother issue is partial years• Can use similar adjustment:Can use similar adjustment:• Exponent is half way through Exponent is half way through time periodtime period• Numerator is multiplied by Numerator is multiplied by fraction of yearfraction of year• e.g. disability for 1.6 years, e.g. disability for 1.6 years, utility of 0.5utility of 0.5• NPV = NPV = Utility / (1 + Utility / (1 + δδ))t t

• NPV = 0.5 / (1.03)NPV = 0.5 / (1.03)0.50.5 + 0.5*0.6 / + 0.5*0.6 / (1.03)(1.03)1.31.3

Discounting – Special Discounting – Special TopicTopic

• Issues with exponential discounting:Issues with exponential discounting:• Relatively easy to manipulateRelatively easy to manipulate• However, assumes same difference However, assumes same difference between any two time periods is the between any two time periods is the same value difference same value difference • Consider: Consider:

today vs. tomorrow and today vs. tomorrow and 10 yrs vs. 10 yrs + 1 day 10 yrs vs. 10 yrs + 1 day

• Essentially, “today” versus all other Essentially, “today” versus all other time periods is valued higher for many time periods is valued higher for many outcomesoutcomes

Discounting – Special Discounting – Special TopicTopic

• Importance of discount rate chosen:Importance of discount rate chosen:• Imagine a benefit of $1,000,000 to be Imagine a benefit of $1,000,000 to be received in 30 yearsreceived in 30 years• Present value of this benefit at Present value of this benefit at different discount rates:different discount rates:

1% -- $741,9231% -- $741,923

3% -- $411,9873% -- $411,987

8% -- $99,3778% -- $99,377

17% -- $9,004 17% -- $9,004

Overall ReviewOverall Review• OutcomesOutcomes

MortalityMortalityMorbidityMorbidity

• Measuring UtilitiesMeasuring UtilitiesVisual AnalogVisual AnalogStandard GambleStandard GambleTime Trade-offTime Trade-off

• Quality Adjusted Life Years (QALYs)Quality Adjusted Life Years (QALYs) QALYs = time (years) x quality QALYs = time (years) x quality (utils)(utils)• Discounting Discounting

NPV = NPV = Utility / (1 + Utility / (1 + δδ))t t

Estimating Outcomes in Decision Analysis Brian Harris MPP Candidate Goldman School of Public Policy...

Documents

Transcript of Estimating Outcomes in Decision Analysis Brian Harris MPP Candidate Goldman School of Public Policy...