What role should probabilistic sensitivity analysis play in SMC decision making?
-
Upload
bethany-russo -
Category
Documents
-
view
43 -
download
0
description
Transcript of What role should probabilistic sensitivity analysis play in SMC decision making?
What role shouldprobabilistic sensitivity analysisplay in SMC decision making?
Andrew Briggs, DPhilUniversity of Oxford
What probabilistic modelling offers
• Generating the appropriate (expected) cost-effectiveness
• Reflects combined implications of parameter uncertainty on the outcome(s) of interest (cost-efectiveness)
• Can make probability statements about cost-effectiveness results – error probability under decision maker’s control
• Offers a means of to calculate the value of collecting additional information
Role of probabilistic sensitivity analysisOverview
• Data sources for parameter fitting• Distributions for common model parameters• Correlating parameters• Presenting simulation results• Using PSA for decision making• Continuing role of traditional sensitivity analysis• Micro simulation models
• Primary data– Can ‘fit’ parameters using standard statistical methods– Provides standard estimates of variance and correlation
• Secondary data– With appropriate information reported can still fit parameters– Meta analysis may be possible
• Expert opinion– Usefulness of Delphi limited (focus on consensus!)– Variability across estimates– Individual estimates of dispersion
Data sources for parameter estimation
• Probabilities are constrained on the interval zero-one• Probabilities must sum to one• Probabilities often estimated from proportions
– Data informing estimation are binomially distributed– Use Beta distribution
• May estimate probabilities from rates– E.G. from hazard rates in survival analysis– Use (multivariate) normal on log scale– Must make transformation from rates to probabilities
Distributions for common parametersProbability parameters
• Costs are a mixture of resource counts and unit costs• Could model counts individually as Poisson with
Gamma distributed mean (parameter)• Costs are constrained to be zero or positive• Can use Gamma distribution if cannot rely on the
Central Limit Theorem (if skewed)• Popular alternative is log-normal, particular when using
regression models on log cost
Distributions for common parameters Cost parameters
• Utilities are somewhat unusual with one representing perfect health and zero representing death
• Can have states worse than death so constraints are negative infinity up to one
• If far from zero, pragmatic approach is to fit beta distribution
• If it is important to represent negative utilities consider the transformation X = 1- U (utility decrement) and fit Gamma or log normal distribution to X
Distributions for common parameters Utility parameters
• Relative risks are ratios!• Can log transform to make additive• Variances and confidence intervals are estimated on the
log-scale then exponentiated• Suggests the log-normal distribution
Distributions for common parameters Relative risk parameters
Relative risk from published meta-analysisExample
• Suppose a published meta analysis quotes a relative risk of 0.86 with 95%CI(0.71 to 1.05)
• Log transform these to give -0.15 (-0.35 to 0.05) on log scale
• Calculate the SE on log scale:(0.05 - -0.35)/(1.96*2) = 0.1
• Generate a normally distributed random variable with mean –0.15 and SE 0.10
• Exponentiate the resulting variable
Correlating parameters
• PSA has sometimes been criticised for treating parameters as independent
• In principle can correlate parameters if we have information on covariance structure– e.g. covariance matrix in regression
• Cholesky decomposition used for correlated normal distributions
• Correlations among other distributional forms not straightforward
Variability and nonlinearity
Even if we are interested only in expected values we need to consider uncertainty when nonlinearities are involved:
E[ g(x) ] g( E[x])
• Uncertainty needed to calculate expectation of nonlinear parameters
• Uncertainty needed to calculate expectation of nonlinear models
Point estimates and variability
0.6 0.7 0.8 0.9 1 1.1 1.2
Relative risk
RR: 0.86 (95% CI: 0.71-1.05)
Standard point estimateExpected value
A model of Total Hip ReplacementExample: interpreting simulation results
cPrimary
RRR
cSuccess
uSuccessP
mr[age]
1- (omrRTHR + mr[age])
omrPTHR
1- (omrPTHR)
RR[age,sex,time]omrRTHR + mr[age]
Primary THR
Revision THR
Successful Primary
Deathmr[age]
uRevision
cRevision cSuccess
uSuccessR
Successful Revision
Example on the CE plane Spectron versus Charnley Hip prosthesis
-£1,000
-£800
-£600
-£400
-£200
£-
£200
£400
-0.05 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40
QALYs gained
Ad
dit
ion
al
Co
st
Corresponding CEAC Spectron versus Charnley Hip prosthesis
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
£- £5,000 £10,000 £15,000 £20,000
Value of ceiling ratio
Pro
ba
bil
ity
co
st-
eff
ec
tiv
e
Multiple acceptability curves Why and how?
• Two reasons for employing multiple acceptability curves– Heterogeneity between patient groups– Multiple treatment options
• Correspond to two situations in CEA– Independent programmes– Mutually exclusive options
• Lead to two very different representations!
Multiple CEACs: handling heterogeneity Spectron versus Charnley (Males)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
£- £5,000 £10,000 £15,000 £20,000
Value of ceiling ratio
Pro
ba
bili
ty c
os
t-e
ffe
cti
ve
Age 90
Age 80
Age 70
Age 60
Ages 40 & 50
Multiple CEACs: handling heterogeneity Spectron versus Charnley (Females)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
£- £5,000 £10,000 £15,000 £20,000
Value of ceiling ratio
Pro
ba
bili
ty c
os
t-e
ffe
cti
ve
Age 90
Age 80
Age 70
Age 60
Age 40
Age 50
Example: GERD management Baseline results
D
C
A
E
B
F
600
700
800
900
1000
1100
1200
38.00 39.00 40.00 41.00 42.00 43.00 44.00 45.00 46.00 47.00 48.00
Weeks free of GERD
Str
ateg
y co
st
A: Intermittent PPIB: Maintenance PPIC: Maintenance H2RAD: Step-down maintenance PAE: Step-down maintenance H2RAF: Step-down maintenance PPI
$10/GFW
$264
/GF
W
$36/GFW
D
C
A
E
B
F
600
700
800
900
1000
1100
1200
38.00 39.00 40.00 41.00 42.00 43.00 44.00 45.00 46.00 47.00 48.00
Weeks free of GERD
Str
ateg
y co
st
A: Intermittent PPIB: Maintenance PPIC: Maintenance H2RAD: Step-down maintenance PAE: Step-down maintenance H2RAF: Step-down maintenance PPI
$10/GFW
$264
/GF
W
$36/GFW
Example: GERD management Uncertainty on the CE plane
F
B
E
A
C
D
$600
$700
$800
$900
$1,000
$1,100
$1,200
38.00 39.00 40.00 41.00 42.00 43.00 44.00 45.00 46.00 47.00 48.00
Weeks free of GERD
Str
ateg
y co
st
Example: GERD management Multiple CEACs
A
BCE
F0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 10 100 1000
Ceiling Ratio (Rc)
Pro
bab
ilit
y co
st-e
ffec
tive
Using probabilistic analysis for making decisions?
Link with standard statistical methods
1. Use standard inference (link with frequentist methods) 2. Use cost-effectiveness acceptability curves to allow
decision maker to select own ‘threshold’ error probability (more Bayesian)
3. Use PSA to establish the value of collecting additional information to inform decision (fully Bayesian decision theoretic approach)
Cost of uncertainty (value of information)
-£30,000 -£20,000 -£10,000 £0 £10,000 £20,000 £30,000
Net monetary benefit
£0
£500,000,000
£1,000,000,000
£1,500,000,000
£2,000,000,000
£2,500,000,000
£3,000,000,000
£3,500,000,000
Op
po
rtu
nit
y l
os
s
Net benefit probability density
Implementation loss function
Non-implementation loss function
Micro-simulation models and PSA
• Microsimulation is an ‘individual’ (rather than ‘cohort’) method of model evaluation
• Typically used to capture patient histories• Calculation requires large number of individual
simulations• PSA would require a second ‘layer’ of simulations
(increases computational time)• Think carefully about whether a micro simulation is
necessary• If it is, buy a fast machine, or use an approximate
solution
Is there any role for standard sensitivity analysis?
• Probabilistic sensitivity analysis is important for capturing parameter uncertainty
• Other forms of uncertainty relate to– Methodology– Structural uncertainty– Data sources– Heterogeneity
• Standard sensitivity analysis retains an important role (in conjuction with PSA)
Critiquing a probabilistic CE model
• Are all parameters included in PSA?• Were standard distributions specified?
– No triangular/uniform distributions• Was the appropriate expected value calculated?• Was standard sensitivity analysis employed to handle
non sampling uncertainty?• Was heterogeneity handled separately?• Was the effect of individual parameters explored?
Summary: the role of PSA
PSA has important role to play• Calculating the correct expected value• Calculating combined effect of uncertainty in all
parameters• Opening the debate about appropriate error probability• Required to calculate the value of information• Continuing role for standard sensitivity analysis