Leeds Institute of Health Sciences Getting better evidence Stephen Morley.
-
Upload
reese-southworth -
Category
Documents
-
view
215 -
download
1
Transcript of Leeds Institute of Health Sciences Getting better evidence Stephen Morley.
Leeds Institute of Health Sciences
Getting better evidence
Stephen Morley
Why bother with trials … ?
From: Moore & McQuay ‘Bandolier’s little book of making sense of the evidence’ 2006
Knowledge
Wisdom
Sys Reviews& Meta-anal
Evidence in clinicalpractice
DistillationIntegration
Quality
ExperienceValues
Conditions
InformationSingle RCTs
Trials
Good Poor
ReviewsGood Ideal May
helpPoor Can
repeatWill
mislead
From: Moore & McQuay ‘Bandolier’s little book of making sense of the evidence’
Issues
• Trial quality– Design, size matters– Quality and effect size
• Outcomes– Variety, validity and ‘clinical’ relevance– Efficacy and effectiveness
• Treatment content and coherence– Is there a model?– Mediation?
Trial quality
Cumulative trials over years
Hoffman et al 2007
Morley et al 1999
Words of caution …
What to count?Quality …Content …
CBT on the label ‘may not be CBT in the tin’
What is CBT in this context ?
Tools for assessing quality
Ideal May help
Can repeat
Will mislea
dFilter out poor quality trials by setting cut-offs
Investigate influences of feature on
conclusions – Meta-regression
WHY?
Trials
+ -
+
MA
-
Quality scales: The Jadad Scale
1. Is the trial randomised (1 point)+1 point if method described and appropriate
2. Is the trial double blind (1 point)+1 point if method given and appropriate
3. Is there a description of withdrawals and drop outs (1 point)
Suggested cut-off = 3
It’s simple Quick Captures major biases Can be reliable with basic training
But
Criterion 2 eliminates all complex interventions
Doesn’t capture important features of psychological trials
Quality scale for psychological trials
Yates et al, Pain 2005: 117; 314-325
Identify and recruitDelphi panel
Panel generates and agreesItems: 3 rounds
Expert panel writes QS
Reliability and validity studies using novice and expert raters
Datafrom 31
published trials
Final QS
Quality scale for psychological treatment trials
32 parts
Is there a good description of the sample in the trial?
Sample characteristics 0 1
Group equivalence 0 1
44 parts
Have adequate steps been taken to minimise biases?
Randomisation 0 1 2
Allocation Bias 0 1
Measurement Bias 0 1
Treatment expectations 0 1
Reliability
ICC absolute agreement
Full scale > 0.9
Treatment quality > 0.9
Design quality = 0.85
Kappa for items
range 0.0 to 0.74
Agreement coefficient for items
>80%
Strengths and weakness of psychological trials
Yates et al, Pain 2005: 117; 314-325
Design
Treatment
Quality over time – the good news
Morley, Eccleston & Williams, unpublished
Effect size and quality
Yates et al, Pain 2005: 117; 314-325
TotalQSβ = -.35 , p =.057
Treatment QS ns
Design QS, β = -.4 , p <.05
Size matters
Data from Hoffman et
al 2007
Outcomes
Outcomes
underlying scale
x►
y►
z►►
x►
dysfunctional or clinical sample
functional or normal sample
a bc
►
CSC criteria
Morley in McQuay et al ‘Systematic reviews in pain research’ 2008, IASP press
Turning continuous outcomes into dichotomous ones
0
5
10
15
20
25
30
35
40
0 5 10 15 20 25 30 35 40
Pre-treatment
Po
st-
treatm
en
t
Deteriorated from pre-treatment good functioning
Reliably deteriorated
Reliable improvement but not clinically significant
Reliable and clinically significant improvement
No reliable change
Reliable improvement but not clinically significant
Reliably deteriorated
Heterogeneity of outcomes in trials
0
5
10
15
20
25
30Pain experience
Mood/ affect
Social role
Cognitive
Behavioural activity
Biological
Health care use
Miscellaneous
Number oftrials usingthe measure
Mean numberof measuresper trial
Data from Morley et al, Pain 1999: 80; 1-13
IMMPACT core outcomes
1. Pain
2. Physical Functioning (interference/disability)
3. Emotional functioning
4. Global improvement
5. Symptoms/adverse effects
6. CONSORT data
Dworkin et al, Pain 2005: 113; 9-19
Stakeholders and outcomes:who wants what change?
0
5
10
15
20
25
30Pain experience
Mood/ affect
Social role
Cognitive
Behavioural activity
Biological
Health care use
Miscellaneous
Health care provider
Researcher
The patient
Employers
What outcome do you want?
Sleep
Weakness
Fatigue
Emotional well-being
Enjoyment of life
Doing tasks
IMMPACT, 2008 Pain:137; 276-285
How much change do you want?
Mdn % change desired
ES(d) %meeting RCI
%meeting CSC
Severity 60 1.45 61.2 20.9
Impact 75 1.70 73.2 57.7
Interfere 66 1.82 75.0 63.2
Activity 44 1.38 36.8 16.2
Thorne & Morley in preparation
How much change do you want?
Thorne & Morley in preparation
Interference Pain severity
The evidence cycle
Efficacy studiesRandomisedControlled
Trials
Evidence-based practiceas policy
Practitioners
Practice-based evidence
Effectiveness studiesRoutineClinical
Treatment
Practitioners
Practice based evidence
Morley, Williams & Hussain, Pain 2008; 137: 670-680
Outcome categories - efficacy
8 58
3 5 4
20 6 20
Crude NNT values
Morley, Williams & Hussain, Pain 2008; 137: 670-680
Benchmarking from RCT data
From Minami et al J Consult Clin Psychol 2007;75: 232-43
Effectiveness + benchmark
WLC group
Tx
Group
RCT is: Williams et al. Pain 1996;66(1):13-22.
Treatment
Quality controlling treatment
• Manuals – protocols?• Training for therapist and teams?• Supervision?
• Patient monitoring systems?
Is there a model: what’s in the tin?
Unpublished data from Morley et al, Pain 1999: 80 1-13
CBT treatment components across trials
What’s the model?
• Generic– Principles of engagement and delivery
• Collaborative, information provision / education – Changing key ‘cognitive appraisals’ through behavioural
experimentation– Techniques: principled application or self service store?
• In PMP/CPM programmes embedded within pharmacotherapy, functional restoration, medical management– How coherent and integrated are they?
• Developing more specific models? A debate (JV)
Thanks to …
• Chris Eccleston• Amanda Williams• Henry McQuay• Andrew Moore
• Wendy Callaghan
• Johan Vlaeyen• Lance McCracken
• Shona Yates• Sumerra Hussain• Fiona Thorne
Seen in Leeds ….
PAINis just
weakness
leaving
the
BODY