Technology Assessment, Outcomes Research and Economic Analyses
Transcript of Technology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and
Economic Analyses
Deborah A. Baumgarten, M.D., M.P.H.Emory University HospitalDepartment of Radiology
Rationale• Cost of medical care increasing faster than
cost of living• 5.6% of GDP in 1965; 17.6% 2012; 19.8% 2020• $27.5 B 1960; $2.71 T 2011; $4.64 T 2020 • $8649 per person in 2011; $13,709 2020
• US spends more than 33 others in Organization for Economic Co-operation and Development• But fewer MDs, hospital beds per capita
OECD spending on health care
OECD Health Data 2012
Health care expenditure as % GDP
OECD Health Data 2012
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
0
2
4
6
8
10
12
14
16
18 US (17.1%)FR (11.6%)SWE (11.5%)GER (11.2%)NETH (11.1%)SWIZ (11.1%)DEN (11.1%)NZ (11.0%)CAN (10.7%)JAP (10.2%)NOR (9.4%)AUS (9.4%)*UK (8.8%)
Health Care Spending as Percentage of GDP, 1980–2013
Percent
OECD Health Data 2015
Rationale• Workers now paying at least $1,094 more in
premiums annually for coverage than in 2000• Still waiting to see what ACA does…
• Despite expenditures, can we document significant improvement in health status?• Life expectancy• Perinatal mortality
Life expectancy
Infant mortality rate
Our poor health• Life expectancy 78.8 vs. 81.2 (2013)• Infant mortality 6.1/1000 vs. 3.5 (2011)• 68% adults over 65 have at least 2 chronic
conditions (33% UK, 56% Canada; 2014)• Obesity rate 35.3% (3.7% Japan, 30.6% NZ; 2014)• Smokers 13.7 (12.8 Australia, 24.1 France; 2013)
US does more of…United States Rank compared
with OECD countries
OECD median
MRI units 35.5 per 1,000,000 2nd (Japan 1st) 11.4 per 1,000,000
MRI exams 106.9 per 1000 50.6 per 1000
CT units 43.5 per 1,000,000 3rd (Japan 1st, Australia 2nd)
17.6 per 1,000,000
CT exams 240 per 1000 136 per 1000
Tonsillectomy 254.4 per 100,000 1st 130 per 100,000
Coronary bypass 79.0 per 100,000 3rd 47.3 per 100,000
Knee replacement 226 per 100,000 1st 121.6 per 100,000
Caesarian section 32.9 per 1000 live births
6th 26.1 per 1000 live births
OECD Health Data 2012 and 2015
Why?• Current payment structure means MDs get
paid more to do more interventions• ACA has not had effect yet…
• Fear of litigation• Patients ask for more—and usually do not
pay out of pocket (insurance covers)
Why?• MDs in US paid more but seen less
• 4 visits/year (OECD average 6.5 visits/year)• US uses more pharmaceuticals
• 2.2 drugs per adult (1.2 Netherlands)• Cost more (US=100; Australia=49; Germany=95)
• Procedures cost more • Bypass $75,345 vs. 42,130 in Australia• Appendectomy $13,910 vs 4,995 Netherlands
Technology Assessment
Bit of background• Technology assessment hierarchy• Based on efficacy (benefit under ideal
circumstances)• Differs from effectiveness (benefit in real
world)• Differs from efficiency (sufficient value?
adds economic component)
Fryback D, Thornbury J. The efficacy of diagnostic imaging. Med Dec Making 1991;11:88-94
Technology assessment hierarchy• Technical efficacy• Diagnostic accuracy• Diagnostic impact• Therapeutic impact• Patient outcomes• Societal outcomes
Fryback D, Thornbury J. The efficacy of diagnostic imaging. Med Dec Making 1991;11:88-94
If technology fails at lower level,
no point going further
Technical efficacy• Ability of technology to produce image• Measured by image characteristics
• Signal to noise, resolution of line pairs, etc.• Developmental: prototype and phantom,
safety• Exploratory: possible use, wide range of
patients, situations, conditions
Technical efficacy• Ideas of what falls here?
Diagnostic accuracy• Distinguish between disease, non-disease,
or normal, abnormal• Establish diagnosis if symptoms• Screen in asymptomatic patient
• Quantify severity or extent of disease• Prognostic information (staging)• Monitor therapy
• Need gold standard; measure of truth
Diagnostic accuracy• Sensitivity, specificity, positive and negative
predictive values • 2 x 2 table to display comparison to gold
standard or reference• Area under ROC curve• Most radiology research falls into this
category
Diagnostic accuracy• Ideas?
Diagnostic impact
• Which imaging tests have greatest impact on clinical decisions?
• Before test need to know• Differential diagnosis• Degree of certainty of diagnoses• Probability of each consideration
• Reassess after test performed• Best done prospectively
Diagnostic impact• Ideas?
Therapeutic impact• Which imaging tests have greatest impact on
therapeutic decisions?• Percent procedures avoided due to imaging• Percent imaging changed plan
• Retrospectively inferred from records• Prospectively assessed using clinicians’
stated plans
Therapeutic impact• Ideas?
Patient outcomes• Traditionally, morbidity or mortality• Change in quality adjusted life years (QALY)• Can be difficult to prove link as imaging
study several steps removed from patient outcome
Societal outcomes• Cost analyses from societal perspective• Usually reported as cost per QALY• Must be below some threshold to be
accepted (? $50,000/QALY)• Example, head CT to screen for brain mets
in lung cancer patients or only if symptomatic?• $70,000/QALY deemed too high to adopt
Outcomes research
Outcomes research—goals• Use imaging wisely
• Avoid over-diagnosis and over-treatment• Risks of treatment in early or mild disease
may outweigh benefits• Data that will permit maintenance of quality
at reduced cost• Increasingly important• Radiology needs to prove worth
Randomized controlled trial• Gold standard of outcomes research• Patients assigned to different groups based
on imaging (or no imaging) to be performed • Measure appropriate outcomes• Confounders and biases accounted for in
design
RCT—limitations• Cost• Size of population
• Power calculation• Estimate expected differences
• Time to complete• Obsolete by time outcomes assessed?• Technology as moving target
• Results may only apply to narrow spectrum of patients; generalizeability
RCT—unnecessary when?• Anecdotal evidence of benefit dramatic;
observational studies suffice• Example: head CT in trauma
• New technology:• Has same or fewer side effects• Has same or better accuracy• Is less expensive
RCT—necessary when?• Really don’t know answer• New technology:
• More expensive• May result in more morbidity• Very controversial
ACRIN—NLST• National Lung cancer Screening Trial• 25,000 patients randomized; low dose CT vs. CXR• Baseline, years 1 and 2• Q6 month health status and medical intervention
questionnaire• Very detailed description of what constitutes
positive screen and what to do• Outcomes
• Primary: Lung cancer specific mortality• Secondary: huge list
ACRIN—NLST• Low dose CT more sensitive than
radiography• Two annual incidence screenings with low-
dose CT resulted in • Decrease number of advanced-stage
cancers• Increase number of early-stage lung
cancers
NEJM 2013;369:920-921
Outcomes research—getting started• Choose technology to be evaluated• Define specific clinical indication• Define target population• Choose alternative techniques for
comparison (may include no imaging)• Define outcomes affected by technology
Example to work through• CT for appendicitis: improve outcomes?• Technology:
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication:
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication: RLQ pain, fever, etc.• Population:
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication: RLQ pain, fever, etc.• Population: patients presenting to ER• Alternative:
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication: RLQ pain, fever, etc.• Population: patients presenting to ER• Alternative: best clinical judgment plus lab• Outcomes:
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication: RLQ pain, fever, etc.• Population: patients presenting to ER• Alternative: best clinical judgment plus lab• Outcomes: morbidity, mortality, perforation rate,
length of hospital stay, surgery rate, % normal appendices removed, cost to hospital, etc.
Example to work through• CT for appendicitis: improve outcomes?• Technology: MDCT (?oral, ?IV, ?rectal)• Indication: RLQ pain, fever, etc.• Population: patients presenting to ER• Alternative: best clinical judgment plus lab• Outcomes: morbidity, mortality, perforation rate,
length of hospital stay, surgery rate, % normal appendices removed, cost to hospital, etc.
• Is this feasible?
What if RCT not feasible?• Modeling: Use of decision analysis software• Need information from other studies
• Accuracy, sensitivity, specificity• Course of disease, natural history• Effectiveness of therapy• Patient utilities and costs
• Meta-analyses for combining results of studies
Modeling• Assumptions about test parameters
(sensitivity, specificity) based on literature • Then vary assumptions
• Assumptions about frequency of outcomes based on personal experience or literature (mortality and morbidity rates, etc.)• Allows variation of these
Modeling—literature exampleCervical spine screening with CT vs. plain films
Blackmore CC et al. Radiology 1999;212:117-125
CT in appendicitis example• Retrospective study of 651 patients treated before
use of CT for appendicitis (1992-1995)• 52% went to surgery; 24% for observation then
surgery; 24% observation to discharge• Initial surgical group, 81% appendicitis• Observation to surgery group, 80% appendicitis• Observation to D/C group, assumed no
appendicitis (no returns to their hospital)
Rhea JT et al. AJR 1997;169:113-118
CT in appendicitis example• Apply sensitivity and specificity of CT from
literature to 100 theoretical patients• Use morbidity, mortality, perforation, correct
diagnosis rates from study of 651 patients• Model impact of CT on outcomes
• Decrease in nontherapeutic surgery (13 patients), observation days (1 day less on average), perforation rate, etc.
• Alternative diagnoses also confirmed
Rhea JT et al. AJR 1997;169:113-118
Intermediate outcomes
• Choose levels lower in hierarchy• Diagnostic impact
• Change in diagnostic possibilities and confidence
• Before and after test ask referring MD:• Differential diagnosis• % certainty for each diagnosis• Degree of satisfaction with test
Intermediate outcomes• Seattle Low Back Pain Imaging Project (SLIP)• RCT of plain films vs. MR as initial imaging• Diagnostic impact
• 15% new diagnosis with plain film• 30% for MRI (spinal stenosis, HNP, etc)
• Change in probability for most likely diagnosis for HNP (p<.002)
Jarvik JG et al. Radiology 1997;204:447-454
Intermediate outcomes• Therapeutic impact
• Before test, ask MD for treatment plan• Observe if plan changes after test
• If temporally related, infer causal relationship between test and plan change
• SLIP: 38% avoided additional test if plain film first, 64% if MRI first (p<.001)
Intermediate outcomes• Advantages
• Cheaper and easier than full RCT (RCT may show intermediate outcomes if assessed as in SLIP)
• No one denied potentially good test• Works well for “add-on tests”
• Tests that would not necessarily replace existing work-up
Intermediate outcomes• Disadvantages
• Stated plans do not necessarily equal clinical action• If treatment invasive, difficult, expensive
or risky, easier to hypothetically choose• RCT still better for “replacement test”
Patient outcomes• Reduced mortality• Reduced morbidity
• Decrease complications from other tests• Avoid invasive procedures• Faster return to normal (or pretest) state
Patient outcomes• Quality of life: ability to undertake
rewarding and enjoyable activities• How person feels, functions
• Quality adjusted life years (QALYs)• Patient satisfaction, reassurance• Physiologic marker improvement• Improved function
Functional status• General and specific measures • SF-36 (Short Form 36)
• 36 questions, 8 domains• General health, pain, physical functioning,
social functioning, mental health, etc.• Roland score (back pain specific index)• Sciatic frequency index
SLIP follow-up• RCT ultimately enrolled 380 patients• Roland score at 12 months did not differ between 2
groups (8.75 plain film vs. 9.34 MRI, p = .53)• p >.75 for all domains of SF-36• Patient reassurance measure
• 1 month p=.09• 3 month p=.08• 12 month p=.002 (58% plain film vs. 74% MRI)• Increased reassurance with MRI over time
• 10 in MRI vs. 4 in plain film had surgery (p =.09)
Jarvik JG et al. JAMA 2003;289:2810-2818
Outcomes research—limitations• Tenuous link between test and patient outcome
• Test usually steps removed from outcome• Best situation: proven link between therapy
and outcome link diagnosis to therapy to outcome
• Example: detection of cerebral aneurysms with MRA treatment decreases morbidity and mortality link MRA to reduced morbidity/mortality
Outcomes research—limitations• If no proven impact on outcome, may be:
• Diagnostic test (inaccurate, insensitive)• Referring clinician (inappropriate use of
info; info does not reach)• Therapy (ineffective or unavailable)• Patient (lack of compliance)• System (lack of patient access)
Alternative outcome• Therapeutic value of diagnostic test• Diagnostic test can have direct impact on
quality of life• Reassurance important to patients and
referring clinicians• If consider only positive tests fail to consider
that negative test has value
Alternative outcome• Example: nonspecific chest pain• 176 patients randomized to receive no
studies or EKG and CPK level• Test group had less short term disability
(20% versus 46%; P=.001)• Use of diagnostic test independent predictor
of recovery in logistic regression• Test group felt care “better than usual”
Sox HC, et al. Ann Intern Med 1981;95:680-5
Economic Analyses
Economic evaluations• Must establish viewpoint of analysis• Society, insurer, provider, patient• All have different inputs and outcomes;
different costs to consider• Example: patient may value increased
quality of life at any cost while HMO may value decreasing overhead costs
Costs
• Value of resources • Charges as estimate• Medicare/Medicaid reimbursement as
estimate• Micro-costing techniques—all resources
identified and quantified• Consider cost of missed abnormalities and
superfluous work-up of false positives
Costs—examples• Medical
• Diagnostic test and interpretation• Nursing care
• Nonmedical• Time (travel, tests and treatment)• Travel• Support with daily activities• Lost productivity
Types of economic analyses• Cost identification or cost minimization
• Compares alternative health care strategies assumed equally effective
Cost identification—example• CT vs. US plus plain film after ESWL• 25 patients underwent all 3• Assumed equal accuracy• Exams timed; direct technical costs calculated
using procedural-based accounting system• CT 15.3 minutes; US + plain film 37.2 minutes• CT $36.86; US + plain film $57.60• Sensitivity analysis on time; within reasonable
range CT always cost less
Remer EM et al. Radiology 1997;204:33-37
Types of economic analyses• Cost benefit
• Costs and benefits assigned dollar values• Comparison of different technologies• Limited use in medicine due to difficulty
assigning monetary values to health states• Cost effectiveness leaves outcomes in
natural, objective units• # lives saved, # cancers detected, etc.
CEA—example• What gain from sixth stool guaiac?• Progression from fifth to sixth stool guaiac
costs $47 million/cancer detected• Can use intermediate outcomes (length of
stay, readmission rates, number of unnecessary surgeries prevented)
Neuhauser D, Lewicki AM. N Engl J Med 1975; 293
Cost utility analysis• Similar to CEA but uses subjective outcomes that
explicitly incorporate patient preferences• QALY: basic outcome measure• Year of life with significant morbidity less desirable
than year with excellent health• Full health equals 1• Death equals 0• Morbidity somewhere between 0 and 1
• Preferred method for economic evaluations
Utility values• Side effects from interferon—.93• Mild angina—.90• Prophylactic mastectomy, oophorectomy—.86• Moderate angina—.70• Hospitalization for tuberculosis—.60• Severe angina—.50• Moderate to severe stroke—.39• Recurrent stroke—.12
Components of economic analysis• Most with modeling software• Reference case (defines typical patient)• Strategies (set of alternative care pathways)• Time horizon (period to consider risks and
benefits)• Perspective (viewpoint of decision maker)• Effectiveness measure (outcome)• Decision tree allows varying components
Decision tree
Singer ME and Applegate KE. Radiology 2001;219:611-620
Components of economic analysis• Probabilities (all events in model have one)
• Estimated from historical data; literature, pilot study or expert opinion
• Utilities (assign utility to every health condition in model)
• Costs (identify all relevant)• Assumptions (due to limitations imposed by
available data; must explicitly state)
Economic analysis—example• Whole body CT screening• Base-case: cohort of 500,000 50 year old men• Sex, age varied in analysis• Benefits of screening assumed due to earlier
detection of disease, improvement in survival• 8 conditions included in model: ovarian, pancreatic,
lung, liver, kidney and colon cancer; aortic aneurysm, coronary artery disease
• Costs of screening, follow-up tests, patient care
Beinfeld, M. T. et al. Radiology 2005;234:415-422
Beinfeld, M. T. et al. Radiology 2005;234:415-422
Schematic of decision-analytic model with life expectancy and cost outcomes used in the cost effectiveness analysis
Economic analysis—example• Compared to routine care
• 6 days of life gained at $2513/patient• Incremental cost: $151,000/life-year gained• 90.8% had finding; only 2% had disease• 32.3% of total cost = work-up of FPs
• With favorable assumptions, one time screening not cost effective; would add burden to healthcare system
Beinfeld, M. T. et al. Radiology 2005;234:415-422
Economic analysis—limitations• Not all agree with strategies chosen
• Was comparison to standard of care?• Were all reasonable options considered?• Was reference case typical of patient population?
• Fears of bias if study funded by drug company or equipment manufacturer
• If too early, data scant; if late, decisions made and model obsolete• Noncontrast CT for renal colic: accepted standard
without study of cost
Conclusions
• Need to be familiar with concepts• Way to inform health care policy makers, guide
research• Goal to improve quality in rational manner, decrease
waste and unnecessary expenditure• Enhance Radiology community’s ability to participate
in health care decision-making locally and nationally• Preserve our specialty!
Thanks!
US does more of…
OECD Health Data 2015