Methods Enabling Constituency Voice in Impact Evaluation for Improving Development
description
Transcript of Methods Enabling Constituency Voice in Impact Evaluation for Improving Development
MethodsEnabling Constituency Voice
in Impact Evaluation for Improving Development
Andre Proctor, Keystone
March 2009
Rethinking IMPACT
IPAL innovations
What is Constituency Voice Bringing the voices of constituents,
especially those meant to benefit, meaningfully into decision-making.
All constituents play a role in How we define successHow we work towards it (strategy)How we learn (measure impacts and reflect
on their meaning)How we communicate it (reporting and
sharing)
CV addresses the 6 purposes of Impact Planning, Assessment and Learning (1) Improve projects
ensures usefulness for constituents Demonstrate impact
from different perspectives on key enablers of change, insight in causal mechanism, triangulation
Inform strategy real time feedback
CV addresses the 6 purposes of Impact Planning, Assessment and Learning (2) Sustain credibility and legitimacy
Confidence and trust among constituents Build capacity
empowers constituents Educate society
reporting re-imagined as a learning activity both for constituents and wider society
Comparative Constituency Feedback
An alternative approach to measuring performance and driving improvements: “Ask them!” Ask recipients how useful they find an
organisation’s work.
Related to ‘customer satisfaction’. Empowers primary constituents.
An emerging fieldMacro level: Humanitarian Response Index
www.daraint.org All OECD government funding for humanitarian response
Center for Effective Philanthropy
www.effectivephilanthropy.org Grantee perceptions of US foundations
Micro level: Measuring Empowerment
quality.bond.org.uk Social movement in Bangladesh Listen First
www.listenfirst.org Concern Worldwide pilot
Comparative feedback
Comparative constituency feedback Structured feedback from primary constituents
on their experience of receiving funding / services / assistance.
Generates quantitative data, for a group of similar actors. Performance can be aggregated and benchmarked.
Provides data from the bottom upwards for learning and improvement.
Case study
East-African grantmakers Cohort of 10 grantmakers. Keystone surveyed all of their grantees.
50% response rate, 336 completed surveysSix key areas:
Nature of funding Application processMonitoring, reporting
& evaluation
Relationship Non-financial supportGrantmaker’s knowledge &
influence
Ethical FrameworkArea Purpose
Primary constituents are involved in designing survey
Constituency voiceRelevance
Independently administered Objectivity
Informed consent of respondents
No one is forced to participate
Anonymous responses & data kept confidential
No discrimination
Report back to constituents Shared learning and open dialogue
Summary findings (extract)
“XXX’s grantees are very satisfied with XXX’s application processes, rating it 15.3 out of 20. … XXX keeps grantees better informed of progress and provides them with clearer information than most other grantmakers.”
“XXX’s grantees rated it 6.4 out of 20 for non-financial support (e.g. capacity building or advice). All grantmakers struggle in this area. Around 40% of XXX’s grantees do not receive assistance in each area; and in some areas over 20% of grantees receive assistance that they do not find helpful.”
Follow up questions… “Could XXX consider:
more site visits, and think about what would be most valuable to do on a site visit?
reviewing its communications – especially web site and its annual report as communication tools?
improving the way that it listens and responds to feedback from Grantees (responding to the feedback from this survey could be a good start).
developing its staff understanding of developmental issues and create opportunities for learning from constituents?
others …”
Conclusion Robust data on grantmaker’s performance – not
just focusing on grantees. Comparison drives insight and learning;
approach fits with other monitoring activities. Drives real change (e.g. CEP). Adaptable to many different situations; the field
is emerging. Puts the rhetoric of empowerment and
accountability into practice. The future … ??
Discussion questions CV How does Constituency Voice contribute
to Impact Evaluation for Improving Development?
How useful could Comparative Constituency Feedback be in what you do?