The value of engagement
-
Upload
wellcometrust -
Category
Health & Medicine
-
view
2.738 -
download
0
description
Transcript of The value of engagement
![Page 2: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/2.jpg)
1. Buzz: “What do you think would constitute good evidence of the value of public engagement?”
2. Presentation: Some approaches to measuring, learning and sharing knowledge.
3. Groups: What do we need to learn and share about engagement, and how we’d like to do it?
Introduction
![Page 3: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/3.jpg)
Why worry about it• Because WT is.• Because all other research donors
are.• Because we all think it’s important, but
don’t really know what works:– How (exactly) does research influence
policy?– Should we invest energy engaging with
legislators or bureaucrats or the media or the public, or schoolchildren?
– Are we even achieving what we’re aiming to achieve?
![Page 4: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/4.jpg)
EvidenceDiscuss with your neighbours:
“What do you think would be convincing evidence of the value of public engagement?”
Write down whatever you come up with.
![Page 5: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/5.jpg)
The DELIVERI ProgrammeDeveloping, testing and promoting new forms of animal health services in Indonesia:
•Pilot projects with farmers & field staff
•Training and capacity development for all•Institutional development•Quality management•Communication & advocacy
“The DELIVERI programme has developed some useful models of institutional change in the context of decentralisation, making a government service more responsive to the needs of local people”1
1 DFID Country Strategy Paper for Indonesia Sept. 2000
![Page 6: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/6.jpg)
ODI and RAPID• UK’s leading development
Think Tank.• RAPID: Promoting greater
use of research-based evidence in development policy & practice– Research / Advice / Information
and Capacity Development– Working with all stakeholders– Case studies, frameworks,
toolkits– evidence-based policy in
development network (ebpdn)
www.odi.org.uk / www.odi.org.uk/rapid
![Page 7: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/7.jpg)
Forms of “engagement”With policy-makers, practitioners and communities to:•Identify problems & issues for research
•Develop research projects and methodologies
•Undertake the research
•Feedback, discuss and validate the results of research
•Formulate solutions – policies and programmes
•Implement the solutions – training and capacity development
•Evaluate their effectiveness
![Page 8: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/8.jpg)
Forms of “engagement”...and we do it through:•(Literature reviews)•Telephone calls, e-mail, (and video conferences)•Face to face 1:1 meetings & field trips•Meetings, workshops & seminars•Collaborative work / projects•Sharing draft outputs for comment•Web 2 – blogs, wikis, discussion groups•Print & web publications•The media
![Page 9: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/9.jpg)
Monitoring and Evaluation
Agenda Setting
DecisionMaking
Policy Implementation
Policy Formulation
Policy processes are complex
Civil Society
DonorsCabinet
Parliament
Ministries
Private Sector
![Page 10: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/10.jpg)
Chronic Poverty in Uganda
Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf)
![Page 11: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/11.jpg)
Factors influencing uptake
The political context – political and economic structures and processes, culture, institutional pressures, incremental vs radical change.
The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of the message, how it is packaged.
External Influences Socio-economic and cultural influences, donor policies etc
The links between policyand research communities – networks, relationships, power, competing discourses, trust, knowledge etc.
www.odi.org.uk/RAPID/Tools/Toolkits/RAPID_Framework.html
![Page 12: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/12.jpg)
A Practical Framework
External Influences political context
evidencelinks
Politics and Policymaking
Media, Advocacy, Networking
Research, learning & thinking
Scientific information exchange & validation
Policy analysis, & research
Campaigning, Lobbying
www.odi.org.uk/RAPID/Tools/Toolkits/Policy_Impact/Framework_qus.html
![Page 13: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/13.jpg)
Health Care in Tanzania
“The results of household disease surveys informed processes of health service reform which contributed to a 43 and 46 per cent reduction in infant mortality between 2000 and 2003 in two districts in rural Tanzania.”
TEHIP Project, Tanzania: www.idrc.ca/tehip
![Page 14: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/14.jpg)
What should you measure?It depends what you’re trying to do….
“If you don't know where you are going, any road will get you there”
![Page 15: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/15.jpg)
Should be:
Specific
Measurable
Achievable
Realistic
Time-bound
(Objective)
Whatever you measure
![Page 16: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/16.jpg)
...and many projects fail when the inputs cease...
Change takes a long time
Inputs Activities Outputs Outcomes Impact
Other Actors
Project Effort
Behaviour Change
![Page 17: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/17.jpg)
Focusing on change
OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo
www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html
www.odi.org.uk/RAPID/Tools/Toolkits/KM/Outcome_mapping.html
![Page 18: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/18.jpg)
Emphasis on “learning”
“…every time we do something again, we should do it better than the last time…”
Goals ResultsActivities
Learnduring
Learnafter
Learnbefore
External networks; Colleagues; Information assets; Own knowledge
www.odi.org.uk/RAPID/Tools/Toolkits/KM/Index.html
![Page 19: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/19.jpg)
Starts with the attitude that someone has probably already done what I am about to do.
I wonder who?”
Learning before: Peer Assist
www.odi.org.uk/RAPID/Tools/Toolkits/KM/Peer_assists.html
![Page 20: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/20.jpg)
• What was the situation?
• What was the challenge?
• What was done?
• What was the result?
• What lessons can be drawn?
Learning During: Stories
www.odi.org.uk/RAPID/Tools/Toolkits/KM/Stories.htmlwww.mande.co.uk/docs/MSCGuide.pdf
Most significant change1.Best stories at each level
2.Synthesis
Stories of change
![Page 21: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/21.jpg)
Horizontal evaluation• Peer review
– Choose the moment– Choose your peers– Limited criteria– e.g. ODI Peer Review
Diagram 8. The Appreciative Inquiry ‘5-D’ model
• Appreciative enquiry– Self-evaluation– CGIAR/CIAT– Workshop
![Page 22: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/22.jpg)
www.odi.org.uk/RAPID/Tools/Toolkits/KM/AAR.html
An after action review asks 4 simple questions:
15 minute team debrief, conducted in a “rank-free” environment.
Learning after: AAR
• What was supposed to happen?
• What actually happened?• Why was there a
difference?• What can we learn from it?
![Page 23: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/23.jpg)
Case & Episode Studies• Classical case studies: how did
evidence shape policy decisions?– e.g. IFPRI & IDRC– Overestimate the role of research
www.idrc.ca/en/ev-26606-201-1-DO_TOPIC.html
www.ifpri.org/impact/impact.htm
www.odi.org.uk/RAPID/Publications/BRP_ITDG.html
www.odi.org.uk/RAPID/Projects/PPA0104/Index.html
www.gdnet.org/middle.php?oid=175
• Episode studies: retrospective tracking back from policy change– e.g. PRSPs, SL, AHC– Underestimate the role of research
![Page 24: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/24.jpg)
RAPID Outcome Mapping
www.odi.org.uk/RAPID/Publications/RAPID_WP_266.html
![Page 25: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/25.jpg)
Social Network Analysis
www.odi.org.uk/RAPID/Tools/Toolkits/KM/Social_network_analysis.html
![Page 26: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/26.jpg)
Other approaches: Public• Citations, webstats, media
logs etc• Surveys
– Quantitative– Qualitative
• Distribution lists and attendance records
• Meeting evaluations• Logs:
– The expected– The unexpected– How you have changed
Evaluation: Practical Guidelines, Research Councils UK. 2002
www.rcuk.ac.uk/cmsweb/downloads/rcuk/publications/evaluationguide.pdf
![Page 27: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/27.jpg)
Other approaches: Policy1. Strategy and direction: Logframes;
Social Network Analysis; Impact Pathways; Modular Matrices
2. Management: ‘Fit for Purpose’ Reviews; ‘Lighter Touch’ Quality Audits; Horizontal Evaluation; Appreciative Inquiry
3. Outputs: Evaluating academic articles and research reports; Evaluating policy and briefing papers; Evaluating websites; Evaluating networks; After Action Reviews
4. Uptake: Impact Logs; New Areas for Citation Analysis; User Surveys
5. Outcomes and impacts: Outcome Mapping; RAPID Outcome Assessment; Most Significant Change; Innovation Histories; Episode Studies
www.odi.org.uk/RAPID/Publications/RAPID_WP_281.html
![Page 28: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/28.jpg)
Learning from each other• Face-to-face meetings
• Establishing a network
• Websites
• Collaborative work
• Publications
• D-groups
So...you already are a sort of network
![Page 29: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/29.jpg)
Network Functions
Facilitators / learners
Community builders
Investor /providers
Convenors
Filters
Amplifiers
Support Agency
![Page 30: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/30.jpg)
Keys to Success1. Clear governance. 2. Strength in numbers.3. Representativeness.4. Quality of evidence.5. Packaging of evidence6. Persistence.7. Membership of key individuals.8. Making use of informal links. 9. Complementing official structures. 10.Good use of ICTs.
www.odi.org.uk/RAPID/Publications/RAPID_WP_276.html
Or contact Enrique Mendizabal – [email protected]
![Page 31: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/31.jpg)
Community of practice
“a group of individuals participating in communal activity, and experiencing/continuously creating their shared identity through engaging in and contributing to the practices of their communities”
Wenger, Etienne (1998), Communities of Practice: Learning, Meaning, and Identity, Cambridge: Cambridge University Press,
ISBN 978-0-521-66363-2
![Page 32: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/32.jpg)
evidence based policy in development networkTo promote greater use of research-based evidence in development policy & practice•Research•Consultations with CSOs and other stakeholders•Capacity-development•Collaborative action-research•Joint projects•Mutual learning
www.ebpdn.org
![Page 33: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/33.jpg)
To promote greater use of research-based evidence in development policy & practice•Research•Consultations with CSOs and other stakeholders•Capacity-development•Collaborative action-research•Joint projects•Mutual learning
www.ebpdn.org• Meet annually
Interactive map of members
evidence based policy in development network
![Page 34: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/34.jpg)
Further information• MandE News by Rick Davies:
www.mande.co.uk
• Psci-com (practical guides section) by the Wellcome Trust: http://www.intute.ac.uk/healthandlifesciences/pscicom/
• Wellcome Trust Researcher Support Links: www.wellcome.ac.uk/Professional-resources/Researcher-support/WTD026043.htm
• RAPID Website: www.odi.org.uk/rapid
• John Young: [email protected]
![Page 35: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/35.jpg)
Group work – 4 Questions:1. What sort of evidence do you need in
your own project(s) to make sure you are on track (and how to collect it)?
2. What sort of evidence would you like to have about other public engagement projects?
3. How would you like to get that evidence (and share your own)?
4. Who should do what (you, other projects, 3rd party, Wellcome)?
![Page 36: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/36.jpg)
Process• 4 groups, each + facilitator & rapporteur
• Split into 2 sub-groups:1. Own evidence
2. Evidence from others
• Whole group3. How to get it
4. Who should do what
• Max. 3 responses to each question, highlighting the most important.
• Report this back to plenary
• “Discussion” on D-Groups
![Page 37: The value of engagement](https://reader035.fdocuments.us/reader035/viewer/2022062513/55577c9ad8b42ad4278b477f/html5/thumbnails/37.jpg)
Focus and location• Policy – in the Auditorium
Fac: Michelle JimenezRap: Greer Van Zyl
• Community – in the Boardroom– Fac: Bella Starling– Rap: Monica Bonaccorso
• Media – in the canteen– Fac: Craig Brierly– Rap: Katrina Nevin Ridley
• Creative/Other – on the Terrace– Fac: Laura Harper– Rap: Marina Joubert