www.3ieimpact.orgPhilip Davies
What Is Evidence, and How Can It Improve Decision Making?
Philip DaviesInternational Initiative for Impact Evaluation [3ie]
BCURE Evidence-Informed Decision-Making Capacity Building Workshop1st and 2nd June 2015Pretoria, South Africa
www.3ieimpact.orgPhilip Davies
Group Exercise
What is Evidence?
www.3ieimpact.orgPhilip Davies
• Evidence is information or data that supports, or rejects, a conclusion
• Evidence has standards of validity and reporting
• Evidence is almost always probabilistic
• Often disagreement on what counts as ‘evidence’
• Evidence is rarely self-evident
• Evidence can be about what is generalisable – and what is particular
• Hence evidence in both quantitative and qualitative
• Not all research is of equal value/sufficient quality
• Single studies can misrepresent the balance of evidence
• Hence, the need for systematic reviews/synthesis of evidence
Some Features of Evidence
www.3ieimpact.orgPhilip Davies
Different Notions of Evidence
Policy Makers’ Evidence
• Colloquial (Narrative)
• Anything that seems reasonable
• Policy relevant
• Timely
• Clear Message
Researchers’ Evidence
• ‘Scientific’ (Generalisable)
• Proven empirically
• Theoretically driven
• As long as it takes
• Caveats and qualificationsSource: J. Lomas et al, 2005
Knowledge Translation and
Transfer
www.3ieimpact.orgPhilip Davies
UK Policymakers’ Views of Evidence
• Focus on the ‘end product’, rather than how the information was either collected or analysed
• Use of ‘anecdotal’ evidence (“tells a story”)• Drawing on such things as ‘real life stories’,
‘fingers in the wind’, ‘local’ and ‘bottom-up’ evidence
But:
• “If we try and move anywhere without having the scientific basis to do so we get fleeced in the House”
• And: DfID Evidence into Action Team
• And: BCURE Programme + DPME (South Africa)
www.3ieimpact.orgPhilip Davies
Evidence
Experience & Expertise
Judgement
Resources
Values, Beliefs and
Ideology
Habits & Bureaucratic
Culture
Lobbyists & Pressure Groups
Pragmatics & Contingencies
Factors Other Than Evidence
www.3ieimpact.orgPhilip Davies
Challenge for this Workshop
How can we integrate the use of evidence with these other influencing factors?
www.3ieimpact.orgPhilip Davies
• Help people make better decisions and achieve better outcomes
• Provide a better understanding of the nature, size and dynamics of the issue at hand
• Compare policy options objectively and rigorously
• Clarify how a policy is supposed to work (ToC)
• Calculate empirically the size of likely impacts
• Provide more precise estimates of risk and bias
• Calculate empirically the diversity/variance of outcomes and impacts
• Establish a cumulative evidence base for decision making.
How Can Evidence Influence Policy?
www.3ieimpact.orgPhilip Davies
Group Exercise:
What is Evidence versus Opinion?
www.3ieimpact.orgPhilip Davies
Evidence versus Opinion
Evidence:
• Data that meet the standards of internal validity, adequacy of reporting, and external validity
• Key qualities: independence, objective, verifiable
Opinion
• Statements and claims that do not meet the standards of evidence
• Key qualities: positional, subjective, partial (selective), hard to verify
www.3ieimpact.orgPhilip Davies
Hoodies
Maths Teaching
Welfare-to-Work
Opinion-Based Policy
Increasing Pressure/Time
Evidence-Based Policy
Evidence-Based vs Opinion-Based Policy
www.3ieimpact.orgPhilip Davies
Thank youPhilip Davies
Email: [email protected]+44 (0)207 958 8350
Visit www.3ieimpact.org
Top Related