THE USE OF SYSTEMATIC REVIEWS IN EVIDENCE-BASED DECISION MODELLING
description
Transcript of THE USE OF SYSTEMATIC REVIEWS IN EVIDENCE-BASED DECISION MODELLING
THE USE OF SYSTEMATIC REVIEWS IN EVIDENCE-BASED
DECISION MODELLING
Nicola Cooper, Alex Sutton,
Keith Abrams, Paul LambertDepartment of Epidemiology & Public Health,
University of Leicester.
• Increasingly decision models are being developed to inform complex clinical/economic decisions
• Parameters can include: –clinical effectiveness, –costs, –disease progression rates, and –utilities
• Evidence based - use systematic methods for evidence synthesis to estimate model parameters with appropriate levels of uncertainty
BACKGROUND
SOURCES OF UNCERTAINTY IN
DECISION MODELS
• Statistical error
• Systematic error
• Evidence relating to parameters indirectly
• Data quality, publication bias, etc.
Stable
Progressive
Death
MARKOV MODEL – TAXANE vs. STANDARD(2nd line treatment of advanced breast cancer)
Response
Cycle length 3 weeks
QR , CR QS , CS
QP , CP
QD = 0
Quality of Life (Q)
Cost (C)
PSR
PSP
PPD
PRP
PR PS
PP
Probability (P)
1. Meta-analyse available evidence to obtain a distribution for each model parameter using random effect models
2. Transform the pooled results, if necessary, and input into the model directly as a distribution and evaluate the model
3. All analyses (decision model and subsidiary analyses) implemented in one cohesive statistical model/program
4. Implemented in a fully Bayesian way using Markov Chain Monte Carlo simulation within WinBUGS software
5. All prior distributions intended to be ‘vague’. Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables
GENERAL APPROACH
Stable
Progressive
Death
MARKOV MODEL – TAXANE vs. STANDARD
(2nd line treatment of advanced breast cancer)
Response
Cycle length 3 weeks
QR , CR QS , CS
QP , CP
QD = 0
Quality of Life (Q)
Cost (C)
PSR
PSP
PPD
PRP
PRPS
PP
Probability (P)
1) M-A of RCTs: Annual ln(odds) of responding
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
MODEL PARAMETER ESTIMATION e.g. PSR, TAX – The probability of moving from stable to
response in a 3 week period
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
2) Pooled ln(odds) distribution
3) Transformation of ln(odds)distrn to transition probability
)3/52/(1
/1
]42.01[1
)],(1[1
jjo ttP
4) Apply to model
Respond
Stable
Progressive
Death
-0.3 (-0.9 to 0.3) PSR
THE REMAINING PARAMETERS
• The Transition Probabilities need estimating for each intervention being compared
• Costs and Utilities can be extracted from the literature and synthesised using a similar approach within the same framework
META-ANALYSES OF LITERATURE
(where required) No. of
studies Time in weeks
(95% Credible Interval) Progression-free time 3 25 (15 to 24)
Time to response from stable 1 12 (6 to 18) Time to progressive from response 1 35 (29 to 41)
Overall survival time 3 53 (35 to 74) Probabilities
Response rate 4 0.43 (0.29 to 0.58) % moving directly to progressive at stage 2. 1 0.13 (0.08 to 0.18)
% with infections / febrile neutropenia 3 0.18 (0.04 to 0.56) % hospitalised with infection / febrile neutropenia 1 0.08 (0.05 to 0.11)
% dying from infections / febrile neutropenia 1 0.01 (0.00 to 0.02) % discontinue treatment due to adverse event 3 0.16 (0.03 to 0.49)
% with Neutropenia grades 3 & 4 2 0.94 (0.82 to 0.98) % with Anaemia grades 3 & 4 2 0.03 (0.00 to 0.28) % with Diarrhoea grades 3 & 4 3 0.09 (0.06 to 0.14) % with Stomatis grades 3 & 4 3 0.08 (0.04 to 0.14) % with vomiting grades 3 & 4 2 0.03 (0.00 to 0.12)
% with fluid retention grades 3 & 4 3 0.05 (0.02 to 0.12) % with cardiac toxicity grades 3 & 4 1 0.00 (0.00 to 0.02)
TRANSITION PROBABILITIES FOR MODEL
(Derived from M-As)
Transition Probabilities (95% Credible Interval)
Infection/FN 0.09 (0.02 to 0.32)
Hospitalised due to infection/FN 0.04 (0.03 to 0.05)
Dying from infection/FN after hospitalisation 0.00 (0.00 to 0.01)
Discontinuation due to major adverse events 0.04 (0.04 to 0.16)
Adverse events – Neutropenia 0.50 (0.34 to 0.63)
Adverse events – Anaemia 0.01 (0.00 to 0.07)
Adverse events – Diarrhoea 0.02 (0.01 to 0.37)
Adverse events – Stomatis 0.02 (0.01 to 0.04)
Adverse events – Vomiting 0.01 (0.00 to 0.03)
Adverse events – Fluid retention 0.01 (0.00 to 0.03)
Adverse events – Cardiac toxicity 0.00 (0.00 to 0.01)
Transition directly to ‘progressive’ state 0.12 (0.08 to 0.18)
Transition ‘stable’ to ‘stable’ 0.65 (0.44 to 0.75)
Transition ‘stable’ to ‘response’ 0.16 (0.11 to 0.28)
Transition ‘stable’ to ‘progressive’ 0.18 (0.11 to 0.37)
Transition ‘response’ to ‘response’ 0.94 (0.93 to 0.95)
Transition ‘response’ to ‘progressive’ 0.06 (0.05 to 0.07)
Transition ‘progressive’ to ‘progressive’ 0.93 (0.79 to 0.96)
Transition ‘progressive’ to ‘death’ 0.07 (0.04 to 0.21)
EVALUATION OF THE MODEL
• A cohort of 1,000 persons is run through the model over 35 3-weekly cycles (until the majority of people are dead) for each treatment option
• Costs and utilities are calculated at the end of each cycle and the average cost and utilities for an individual across all 35 cycles for each treatment option are calculated
• This process is repeated 4,000 times (each time different values from each parameter distribution are sampled)
Bayesian (MCMC) Simulations
-£4,000
-£2,000
£0
£2,000
£4,000
£6,000
£8,000
£10,000
-0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50
Incremental utility
Inc
rem
en
tal
co
st
Standard dominates
Taxane more effective but more costly
Taxane less costly but less
effective
Taxanedominates
COST-EFFECTIVENESS PLANE
• Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin
• However, there is a risk of a fatal hemorrhage as a result of taking warfarin
• For whom do the benefits outweigh the risks?
CLINICAL NET BENEFIT
- Warfarin for non-rheumatic atrial fibrillation
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
Multivariate riskequations
Meta analysisof RCTs
Meta analysis of RCTs obs studies QoL study
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
Multivariate riskequations
Meta analysisof RCTs
Meta analysis of RCTs obs studies QoL study
0.002 0.004 0.006 0.008 0.010 0.012 0.014
050
100
150
200
250
300
risk of bleed per year
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
810
-1.5 -1.0 -0.5 0.0 0.5 1.0
02
46
reduction in relative risk
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.4
Outcome ratio
Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents)
% of
cohort
T hrombo -
embolism
rate (%
per year
(95% CI))
Mean
(s.e.)
Median
(95%
CrI)
Probability of
Benefit > 0
Simulated PDF
12
17.6 (10.5
to 29.9)
- 0.0004
(0.15)
0.06
( - 0.29 to
0.20)
54.2 %
-0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4
01
23
45
6
2 or 3 Clinical factors
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
Multivariate riskequations
Meta analysisof RCTs
Meta analysis of RCTs obs studies QoL study
0.002 0.004 0.006 0.008 0.010 0.012 0.014
050
100
150
200
250
300
risk of bleed per year
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
810
-1.5 -1.0 -0.5 0.0 0.5 1.0
02
46
reduction in relative risk
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.4
Outcome ratio
ADVANTAGES OF APPROACH Synthesis of evidence, transformation of variables &
evaluation of a complex Markov model carried out in a unified framework
Facilitates sensitivity analysis
Provides a framework to incorporate prior beliefs of experts
Allows for correlation induced where studies included in the estimation of more than one parameter
Uncertainty in all model parameters automatically taken into account
Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propagated
FURTHER ISSUES1. Handling indirect comparisons correctly
• E.g. Want to compare A vs. C but evidence only available on A vs. B & B vs. C etc.
• Avoid breaking randomisation
2. Necessary complexity of model?• When to use the different approaches outlined
above?
3. Incorporation of Expected Value of (Perfect/Sample) Information
4. Incorporation of all uncertainties
1. Handling indirect comparisons correctly• E.g. Want to compare A vs. C but evidence only
available on A vs. B & B vs. C etc.
• Avoid breaking randomisation
2. Necessary complexity of model?• When to use the different approaches outlined
above?
3. Incorporation of Expected Value of (Perfect/Sample) Information
4. Incorporation of all uncertainties