1
A review of Evaluation and Impact Assessment A review of Evaluation and Impact Assessment by DAC Heads of Information in 2006 by DAC Heads of Information in 2006
Ida Mc Donnell
Annual Meeting of DAC HoI
Canberra May 2006
2
3
Structure
Challenges and Next Steps
I
II
III
Objectives of the Review
Survey Results
4
Challenges and Next Steps
I
II
III
Objectives of the Review
Survey Results
5
Objectives Objectives
• Overview of current practice in evaluating communications, awareness raising, development education
• Identify common lessons and challenges
6
Challenges and Next Steps
I
II
III
Objectives of the Review
Survey Results
7
Why evaluate?Why evaluate?
Source: Questionnaire administered to DAC Heads of Information October 2005
Purpose of Evaluations for Aid Agencies and NGOs Funded by Agencies (n=19)
1716
11 1110
1314 14
11
0
02468
1012141618
Lessons forfuture
Monitoring Accountability Impact Other
nu
mb
er o
f re
spo
nse
Aid agency
NGO
8
Common criteria for evaluating an activity Common criteria for evaluating an activity (Govt & NGO) in 11 DAC Members(Govt & NGO) in 11 DAC Members
New/pilot activityCost/size/importance
Depends on activity
Ongoing,L-T activity
Belgium * * *
Canada * *
Denmark * (NGOs only) *
Finland * * (NGOs) *
Germany * (> than euro 7K for NGOs)
*
Luxembourg *
Netherlands * (>euro 20K) *
Norway *
Sweden *
Switzerland * *
UK *
Source: Questionnaire administered to DAC Heads of Information October 2005
9
Tools & IndicatorsTools & Indicators
16
15
13
12
11
11
11
7
3
2
2
0 2 4 6 8 10 12 14 16
Polls
Visits to website
Number participants
Media coverage
Focus group
Interview target group
Newspaper refs
Request information
Other
Invitations to show activity
Questions in Parliament
Figure 2 Tools and Indicators for M & E in DAC Members
no. of response
Source: Questionnaire administered to DAC Heads of Information October 2005
10
Challenges and Next Steps
I
II
III
Objectives of the Review
Survey Results
11
Top challenges Govt Evaluation of CommunicationTop challenges Govt Evaluation of Communication
• How to measure impact• Attribution• Resource issues - cost and time• Indicators, incl. process, short-term, long-term and
comparable between different activities.• Culture of evaluation• Status of communications; not treated seriously/seen as
part of development cooperation so less status…
12
Top challenges Evaluation by NGO of Top challenges Evaluation by NGO of CommunicationCommunication
• Similar to Govt. (cost, time, impact, culture, comparability…)
With a few specificities to NGOs: • lack of interest in evaluating long-term reach, especially
when funded for short-term activities• diversity of NGO activities and targets • difficulty in having standard indicators • no established methodologies• few trainers• no expertise in evaluation• trust to report negative conclusions from evaluation
13
HOI Network can helpHOI Network can help
• Sixteen responses
• 12 think HOI can exchange best/good practice, experiences, and information on evaluation and impact assessment.
• Nine think HOI can identify relevant methodologies, common indicators, and best practice guidelines on evaluating effective communication and development education.
• Other: testing/experimenting with new methods; solving the issue of attribution; organise workshop on the art and theory of evaluation; better coordination; share good external evaluators.
14
Options for Next Steps…Options for Next Steps…
• Organise workshop in early 2007 with expert evaluators, including members of the DAC Evaluation Network
• HOI network work on/agree on common guidelines/best practice on evaluation?
• Investigate the possibility of experimenting with methodologies and create a knowledge-base on which activities are more effective in changing behaviour than others?
• Other?
15
Thank you!Thank you!
www.oecd.org/dev/opinion
Top Related