Www.3ieimpact.org Philip Davies What Is Evidence, and How Can It Improve Decision Making? Philip...

www.3ieimpact.orgPhilip Davies

What Is Evidence, and How Can It Improve Decision Making?

Philip DaviesInternational Initiative for Impact Evaluation [3ie]

BCURE Evidence-Informed Decision-Making Capacity Building Workshop1st and 2nd June 2015Pretoria, South Africa


Group Exercise

What is Evidence?


• Evidence is information or data that supports, or rejects, a conclusion

• Evidence has standards of validity and reporting

• Evidence is almost always probabilistic

• Often disagreement on what counts as ‘evidence’

• Evidence is rarely self-evident

• Evidence can be about what is generalisable – and what is particular

• Hence evidence in both quantitative and qualitative

• Not all research is of equal value/sufficient quality

• Single studies can misrepresent the balance of evidence

• Hence, the need for systematic reviews/synthesis of evidence

Some Features of Evidence


Different Notions of Evidence

Policy Makers’ Evidence

• Colloquial (Narrative)

• Anything that seems reasonable

• Policy relevant

• Timely

• Clear Message

Researchers’ Evidence

• ‘Scientific’ (Generalisable)

• Proven empirically

• Theoretically driven

• As long as it takes

• Caveats and qualificationsSource: J. Lomas et al, 2005

Knowledge Translation and

Transfer


UK Policymakers’ Views of Evidence

• Focus on the ‘end product’, rather than how the information was either collected or analysed

• Use of ‘anecdotal’ evidence (“tells a story”)• Drawing on such things as ‘real life stories’,

‘fingers in the wind’, ‘local’ and ‘bottom-up’ evidence

But:

• “If we try and move anywhere without having the scientific basis to do so we get fleeced in the House”

• And: DfID Evidence into Action Team

• And: BCURE Programme + DPME (South Africa)


Evidence

Experience & Expertise

Judgement

Resources

Values, Beliefs and

Ideology

Habits & Bureaucratic

Culture

Lobbyists & Pressure Groups

Pragmatics & Contingencies

Factors Other Than Evidence


Challenge for this Workshop

How can we integrate the use of evidence with these other influencing factors?


• Help people make better decisions and achieve better outcomes

• Provide a better understanding of the nature, size and dynamics of the issue at hand

• Compare policy options objectively and rigorously

• Clarify how a policy is supposed to work (ToC)

• Calculate empirically the size of likely impacts

• Provide more precise estimates of risk and bias

• Calculate empirically the diversity/variance of outcomes and impacts

• Establish a cumulative evidence base for decision making.

How Can Evidence Influence Policy?


Group Exercise:

What is Evidence versus Opinion?


Evidence versus Opinion

Evidence:

• Data that meet the standards of internal validity, adequacy of reporting, and external validity

• Key qualities: independence, objective, verifiable

Opinion

• Statements and claims that do not meet the standards of evidence

• Key qualities: positional, subjective, partial (selective), hard to verify


Hoodies

Maths Teaching

Welfare-to-Work

Opinion-Based Policy

Increasing Pressure/Time

Evidence-Based Policy

Evidence-Based vs Opinion-Based Policy


Thank youPhilip Davies

Email: [email protected]+44 (0)207 958 8350

Visit www.3ieimpact.org

mailto:[email protected]

Www.3ieimpact.org Philip Davies What Is Evidence, and How Can It Improve Decision Making? Philip...

Documents

Transcript of Www.3ieimpact.org Philip Davies What Is Evidence, and How Can It Improve Decision Making? Philip...