How qualitative research contributes to evaluation Professor Alicia O’Cathain ScHARR University of...
-
Upload
darlene-stewart -
Category
Documents
-
view
217 -
download
0
Transcript of How qualitative research contributes to evaluation Professor Alicia O’Cathain ScHARR University of...
How qualitative research contributes to evaluation
Professor Alicia O’CathainScHARR University of Sheffield
22 June 2015
What is qualitative research?
• Normal in evaluation• Understanding not
measuring • Set of methods– Focus groups– Semi-structured or in
depth interviews– Non-participant
observation– Diaries
What is evaluation?
• Researcher led evaluation
• Policy evaluation
Researcher-led
• MRC Framework developing an evaluating complex interventions
• ACTIF programme – 5 years– RCT– Qualitative at each phase
O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A
systematic mapping review. BMJ Open 2013;3:e002889
Intervention Trial design and conduct Outcomes
Measures Health conditions
Intervention n=254Develop
n=48
Describe it n=10
Understand how it works
n=23
Value and benefits
n=42Acceptability in principle
n=32
Feasibility and acceptability
n=83
Fidelity, reach and dose n=12
Implementation in real world n=4
Trial design and conduct n=54
Recruitment
Diversity
Participation in trials
Acceptability in principle
Acceptability in practice
Ethics/informed consent
Adapting to local circumstances
Impact on staff, researchers, patients
Potential value Potential valueBias Avoidance of measurement biasEfficiency Faster recruitment Saves money Ethics Trials sensitive to human beings
Improved informed consent
Implementation Facilitates replicability of intervention in the real world
Facilitates transferability of findings in the real world
Interpretation Explains trial findingsRelevance Ensures interventions meet the needs of health
professionals and patients
Success Makes a trial successful, feasible, viableValidity Improves internal validity Improves external validity
Maximising value…
1. Do it early– 28% pre-trial• Intervention development 100%• Acceptability of intervention in principle 25%• Acceptability of intervention in practice 24%• Recruitment 18%• Breadth of outcomes 0%
…otherwise its about future trials
2. Publish learning for specific trial or future trials3. Think beyond interviews: non-participant observation4. Try iterative or dynamic or participatory approaches at feasibility phase5. Not just complex interventions
• 38% of 104 data extracted were drugs or devices
6. Think about the range of work
Problems with quantitative only
• Null RCTs….explain findings (context, mechanisms of action, implementation)
• Failed trials….prevent this at pilot stage
• It works but what is ‘it’?
…..qualitative can fix
Policy evaluation
• Learning from early adopters (feasibility)• Stakeholder reception (acceptability)• Service delivered (implementation, workforce)
Useful but challenges remain
– Fast evaluation– When to evaluate– Moving target– Replacement of difficult-to-measure outcomes
with understanding of processes
Conclusions
• Useful contribution no matter what type of evaluation – essential due to complexity
• Can help to fix problems faced in researcher-led evaluation
• Challenges in policy evaluation need reflection