Getting Public Health Research into Policy: Science, Art or Dance

www.hertsdirect.org

Research to Policy:Science, Art or Dance?

Prof Jim McManus, OCDS, CPsychol, CSci, FFPH, FRSB, FRSPH, AFBPsS

Director of Public Health, Hertfordshire County [email protected]

July 14th 2016, UK Public Health Research Centres of Excellence Conference

www.hertfordshire.gov.uk

What I want to cover

• A slightly tongue-in-cheek journey• Poking fun at a relationship which doesn’t quite work,

because I care so much that it should• Seen policy making in various places over 25 years• Practical and real world view of research interface with policy• Things that work and things that don’t work• What not to do• Failings of Policy Makers


Longer Version

• A range of case studies including some from criminology

• Some further tools for some of the steps


A Thesis...• Public health is

inherently political• Because it affects

populations• It inevitably impacts on

or is impacted on by Policy

• Need to think much more about these issues and claims


An average week for a chief officer/senior civil servant

• 200 - 500 or more emails a day

• Pile of FOIs, Surveys, Reports, MP letters, Briefings

• Politicians confused about media reporting

• Policymakers dissing experts and trying to assert integrity of expertise

• NGOs, Advocates, Government all want a piece

• Lots of people “with solutions” wanting time

• 63 meetings, many of them triple booked

• Series of conflicting deadlines• Reducing budgets• Things come up unpredictably

which need “solving”• Wanting to do the right thing for

the population• Trying to get inside the

Politicians and Partners heads


Strategic Challenges1. Delivering core public health must dos for less2. Growing demand and need 3. The challenges of preventing cost and adverse outcome

(Falls going up in some areas)4. Levering whole system redesign and prevention using

everyone’s resources to save money and improve outcomes

5. Doing stuff which works6. Massive and confusing literature on implementing research


The 3 or 4E’s Duty or Value for Money Duty

• Economy -value for money and delivering the required service on budget, on time and within other resource constraints

• Efficiency- getting an acceptable return on the money and resources invested in a service.

• Effectiveness - extent to which the organisation delivers what it is intended to deliver.

• Equity – not always in there even with legal frameworks


What Policymakers think of researchers

Q. How many scientists does it take to reach a firm conclusion?

A.There are too many parameters to be definitive here


What Politicians Think...• What is needed?• What is right?• What is doable?• What is politically and financially doable?• Who’s in my way? Who’s with me?• What do my voters want?• What do experts think?• What do I now think?............Conclusion


The “knowing” and “interpretive” world of policymakers

• The legacy (good and bad) of New Public Management...measure everything, quantify

• Money, existing policy and law, influence• The desire to do the right thing or the desire to please

political masters. Why are they in the job?• Who’s got a view and how to I balance them?• There’s no money• What is the space for the possible when you take all this

into account


The Epistemology of the Policy ProcessWhat they teach you at government school – the science

Source http://www.thisnation.com/textbook/processes-policyprocess.html

http://www.thisnation.com/textbook/processes-policyprocess.html

http://www.thisnation.com/textbook/processes-policyprocess.html

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=imgres&cd=&cad=rja&uact=8&ved=0ahUKEwjKhPjWkt_NAhVLBcAKHcMtCAAQjRwIBw&url=http%3A%2F%2Fwww.thisnation.com%2Ftextbook%2Fprocesses-policyprocess.html&psig=AFQjCNEb_DgaJB-dXKHbxxGz9CwvXpNVVQ&ust=1467905076184853


The Policy ProcessWhat usually tends to happen – the art of influence

1001 issues and influencers, many competing

Not everyone happy

PolicyProposalrefinery

RefinementCompromise

PartialImplementation orEffectiveness

Repeat loop eternally

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=imgres&cd=&cad=rja&uact=8&ved=0ahUKEwi3xsyxk9_NAhVhDMAKHS37CB0QjRwIBw&url=http%3A%2F%2Fwww.openforum.com%2Farticles%2F10-tips-to-eliminate-chaos-in-your-daily-routine%2F&psig=AFQjCNEFf54OAuHHuWXcJYfLEJFbf9TQuQ&ust=1467905266091083


More like a dance than art or science

• Requires instinct and “feel” for where you are in the dance and why you’re doing it

• Requires good evidence and knowing when to use it

• Requires relationship “heart speaks to heart”• Requires emotional engagement• Partnership with the other dancer(s)• Adapting steps to where you are


The Policy Process Dance: Art & Science in relationshipThe Policy Networks and Influences Approach

Source: http://www.lancaster.ac.uk/fass/projects/hvp/newsletters/5findings.htm

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=imgres&cd=&cad=rja&uact=8&ved=0ahUKEwjlm4TYkt_NAhWkA8AKHWmKDNwQjRwIBw&url=http%3A%2F%2Fwww.lancaster.ac.uk%2Ffass%2Fprojects%2Fhvp%2Fnewsletters%2F5findings.htm&psig=AFQjCNGbc7cuEGs42yhPXO8NJ-Slyy9r2g&ust=1467905078429906


The Dance Floor - Policy Process in 5 DimensionsA model for dynamic tension

PolicyProcessQuadrilateral

“Evidence”The

Desirable,The

Achievable

Motivations

Influences

Multiple layers:1. Advocates2. Politician3.Officers 4. Scientists5. UsersAcross Time too

www.hertsdirect.org

Onto the dance steps....If that’s how it happens, what works for the Policy/Research relationship?

I have a personal proposal...


Περιχώρησις - Research, Policy and Gregory Nazianzen

• Idea of περιχώρησις perikhōrēsis• Dancing round. Mutual Intimacy. • Interprenetration of understanding • Dynamic hermeneutic unity

but not identity• Psychology, Emotion, Mutuality• Perikhoresis as a model for research

into policy


What to do – dance etiquette

• Build a Relationship!• Research on who you need to talk to• Focus on what you want to say and why they

should listen• Hone• The 60 Second Pitch to the right person made

personally• And the usual newsletters etc


Dance Steps – 7/8 C’s of Consulting1st for me – What’s your Capital to offer?

1.Client – who, what do they want?2.Clarify – what is the dance?3.Create – work up the doable4.Change - what is to be done5.Confirm – the what and the doing6.Continue – keep doing it7.Close – thanks for the dance, happy?

Read one review of this here http://www.sweeneycomms.com/The-Seven-Cs-of-Consulting-g.asp

http://www.sweeneycomms.com/The-Seven-Cs-of-Consulting-g.asp

http://www.sweeneycomms.com/The-Seven-Cs-of-Consulting-g.asp

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiJiaKmx-PNAhUrDsAKHeqdBfcQjRwIBw&url=http%3A%2F%2Fwww.pearsoned.co.uk%2Fbookshop%2Fdetail.asp%3Fitem%3D100000000356905&psig=AFQjCNGFYOxGAubzZyb3w8HGOKZNmnjLmA&ust=1468056627103560


What not to do..wrongfooting• Send a bulletin and expect anyone to read it• Write in scientific terms for policymakers• Set out what you’ve done without setting any

context• Send a 99 page report with eight appendices

and no executive summary• Send anything before you’ve worked out who to

talk to


Research on Diet

Is your research clear and conclusive enough to give definitive guidance to people?

If yes – be clear and don’t be vague

If no – then shut up and continue researching until it is clear enoughA moral responsibility on scientists NOT to overstate their case and generalizability


Family Nurse Partnership – applying the evaluation to public health decision making

• Evaluation report says doesn’t work not cost effective

• FNP says it does other valuable things

• Where do these contradictory views fit in policymaker decisions?

• Issues raised:– Appraisal of research– By products of

interventions and roundedness of evaluations

– Applicability


Research on effects of e-cigarettes

• Most of it unappliable because of design flaws or even repeating existing badly done research

• In vitro studies which don’t reflect real life• Speculation and going beyond the research

doesn’t help


You could interchange these headings..Policy is...• Messy and Spiral• Contested Space• “Oh God, someone’s had an

idea”• “We must do something”• “Any evidence”• “I want answers”• “What will it do”• Compromise, Influence and

Change

Research is..

• A trade off between what you want to know/do and resources to do it

• A methodological trade off• Hermeneutically situated• Contested


Knowing your partner...feasibility

Option A Option B Option C

Effectiveness Very Positive Positive No impact

Flexibility Very Positive Positive Positive

Sustainability Positive Positive Negative

Political Feasibility High Medium Low

Administrative Feasibility

High Medium Low

Time Short Medium Long

Cost High Medium Low

Suggested criteria example


Communication - Writing style as an exampleAcademic• Heavily referenced• Third person • Passive voice• Tentative conclusions

Local Govt• Active voice• Summaries• Recommendations• Costings• Policy options• Impact measurements• Feasibility• risk


Two partners in the same dance – i) the research itself and ii) the organisation and its agility


Soft skills for the dance....making the 8 Cs live• Contested space – competing demands• Learning and mutual understanding • influence, nuance• Understand what constitutes evidence• Understand each side’s concerns • Understand the pressure of being a policymaker and the

often limited analytical tools and apparatus available • Craft a working relationship in that space


Formation for the Dance not Training for the Science

• Policy makers are formed – psychological, political, emotional, conceptual, skills and tools

• Researchers need to be formed not trained in how to work with one another

• A conscious relationship and working with one another

• Too little interpenetration of the two• Policy Networks


Thank you

• [email protected] • Public Health Pages• www.hertsdirect.org/healthinherts • JSNA• http://jsna.hertslis.org/

mailto:[email protected]

www.hertsdirect.org

Slides for a longer session


Tools for the road....• Contested space – competing demands• Learning and mutual understanding • Soft skills – influence, nuance• Understand what constitutes evidence• Understand each side’s concerns • Understand the pressure of being a policymaker and the

often limited analytical tools and apparatus available • Craft a working relationship in that space


Some problems

• Massive literature on applying research in practice

• Lots of problems in the literature – Sometimes conceives receivers of research

as in deficit of knowledge mode– Focuses often on one side – the research

quality or the organisation – not synergistic enough


Thesis

• Applying research needs a partnership between researchers and policymakers from the inception if possible

• Two sides of a coin : the quality of the research and the agility and leadership of the organisation

• Develop a roadmap jointly


Context

• Cuts to public services• Increasing demand on NHS services• Financial unsustainability of current system• We need research which can help us keep

people independent and healthy, and reduce need for public services


Nationally funded research

• http://www.nihr.ac.uk/funding/public-health-research-programme.htm

• http://www.nets.nihr.ac.uk/programmes/phr• http://www.nihr.ac.uk/funding/school-for-public-

health-research.htm• Great these are funded but we need more

emphasis on translatable research and interventions, and evaluating existing ones


Case Studies from Criminology• Very policy directed field• Political and worldview issues in researchers and

politicians• Negotiating a sometimes difficult space• Government commissions a lot of research to

inform policy• Some ongoing controversies• Analogies for health work...


Framing the debate can put your partner off…

Knowing your partner – Kenneth Clarke or Michael Gove would like this. How would it go down with Michael Howard or Theresa May?

Hulley et al, Brit J Crim, 2016


Why are you writing? To Change Policy or just lay bare a problem?Feeling good about what you’ve written but what have you changed?

Raymen, Brit J Crim, 2016


Actors in the System• Actors in

the system can derail the implementation, even when trying to do “the right thing”


Study of Public Health Policy Making

Wilmot et al, Journal of Public Health (2016)


For real world relevance: research cannot be static when life is dynamic

Limitation?Or actually a strength that it addresses dynamic real world decision making?

Wilmot et al, Journal of Public Health (2016)


A good example of policy revelant work

• Frames a policy problem and how to respond to it relatively simply

Walker et al, JPH, 2016


Research on Diet

Is your research clear and conclusive enough to give definitive guidance to people?

If yes – be clear and don’t be vague

If no – then shut up and continue researching until it is clear enoughA moral responsibility on scientists NOT to overstate their case and generalizability


The Problem

• Outcomes Frameworks• Connecting evidence with outcomes• Connecting evidence with interventions• Interests of policymakers and interests of

researchers


So how do you go about applying research into public health practice? • Lots of literature on this• Some very complex and complicated models• A Proposal from me

1. Treat it like a change management process2. Think about two major sides to a coin 3. Project and process manage it4. Fidelity and sensitivity test


Change Management

• Lessons from ESRC evaluations of large scale IT programmes

• Lessons from NHS and Social Care research on applying research


Levels and Phasing

• Which level is the intervention?– Societal– Environmental?– Biological?– Intrapersonal?

• Which phase of time?– 1 year, 2, 3?


Return on Investment as part of the argument

• Variable response from commissioners to this• Too much use of poorly developed ROI models in

evidence• ROI is not always the answer and sometimes doesn’t

even help• Most ROI models aren’t real world enough• Depends on how robust it is• Who cashes the returned cheque? If agency A invests

but Agency Y gets the return, where is the incentive?


Return on Investment Tools

• Helps if the evidence is also very clear • Helps if there is a timescale to the return and

clarity on who gets it• The policy/commissioning/research partnership

is crucial to making ROI useful


What is evidence-based public health? Many definitions

◦ From Evidence-Based Medicine to Evidence-Based Public Health Evidence-based public health is defined as the development, implementation, and

evaluation of effective programs and policies in public health through application of principles of scientific reasoning, including systematic uses of data and information systems, and appropriate use of behavioral science theory and program planning models. (Brownson, Ross C., Elizabeth A. Baker, Terry L. Leet, and Kathleen N. Gillespie, Editors. Evidence-Based Public Health. New York: Oxford University Press, 2003.)

◦ E-Roadmap to Public Health Practice Concepts (New Hampshire Institute for Health Policy and Practice) Developing, implementing, and evaluating public health programs or public health

policies (in public health terms an "intervention") that have 1) data demonstrating their effectiveness and 2) a grounding in a health behavior theory or ecological model of health.


What is evidence-based public health?

• Many definitions– “The process of integrating science-based

interventions with community preferences to improve the health of populations.” (Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: An evolving concept. Am J Prev Med. 2004 Dec;27(5):417-21.)


Steps for EBPH

• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is

already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan

O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.


More on the 2 sides of the dance: Partner 1: the Research itself• Internal and external validity• Applicability• Clarity of methods and outcomes• Translatability• Intervention readiness• Engagement with policymakers


Dance partner 2: the organization and its agility


Example: Behavioural Sciences• Review of Research• Synthesis into policy paper• Applying resource• Articulating work and programme plan• Delivery and championing• Integration into strategy

– This is very high level and much more involved in this


Develop an initial statement of the issue

Use PICO (from EBM) to begin define question◦ P: Population◦ I: Intervention◦ C: Comparison◦ O: Outcome

Use a logic model to begin your strategic planning◦ Inputs, activities, outputs, results (short/long term)◦ W.K. Kellogg Foundation Logic Model Development Guide

http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf◦ Logic model tutorials (CDC)

http://apps.nccd.cdc.gov/dashoet/logic_model_1/menu.html http://apps.nccd.cdc.gov/dashoet/logic_model_2/index.html


Steps for EBPH

• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is

already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan

O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.


Mapping the Policy Process

• General Context issues – domestic and international.• Specific Policy Issues (i.e. the policy cycle)• Who are the Stakeholders? (Stakeholder analysis)

– Arena: government, parliament, civil society, judiciary, private sector.

– Level: local, national, international • Process matrix + political influence ratings• What is their Interest and Influence?

[Sources: M. Grindle / J. Court ]


Ontario Drug Programme (Khan et al,2014)http://dx.doi.org/10.1016/j.healthpol.2014.06.007

• Policymakers have cited barriers to using evidence, including lack of research relevance and timeliness.

• .Although reports often demonstrate an increase in research relevance, rarely do they provide concrete methods of enhancing research timeliness

• . Additionally, the impact of researcher–policymaker collaborations is not well-discussed.


Be clear about how feasible it is to do this

Option A Option B Option C

Effectiveness Very Positive Positive No impact

Flexibility Very Positive Positive Positive

Sustainability Positive Positive Negative

Political Feasibility High Medium Low

Administrative Feasibility

High Medium Low

Time Short Medium Long

Cost High Medium Low

Suggested criteria example


The problems• Early and avoidable disability and disease• Early avoidable death• Smoking prevalence uneven• Obesity rising in adults, flattening in children in most places,

rising in some• Alcohol related disease on the increase• Preventable cost to public health• Multiple needs, worklessnessThe Tartan Rug and local profiles give much more info on this

[email protected]


Thank you

• [email protected] • Public Health Pages• www.hertsdirect.org/healthinherts • JSNA• http://jsna.hertslis.org/

Getting Public Health Research into Policy: Science, Art or Dance

Health & Medicine

Transcript of Getting Public Health Research into Policy: Science, Art or Dance