Measuring Impact in Implementation Monday, April 21, 2014 @ 2.00pm
description
Transcript of Measuring Impact in Implementation Monday, April 21, 2014 @ 2.00pm
Measuring Impact in ImplementationMonday, April 21, 2014 @ 2.00pm
Jeffrey Swedberg (Senior M&E Associate, QED Group)
Purpose
“The success or otherwise of the implementation of security and human rights programs can often be difficult to measure. Regardless, interested parties demand that monitoring and evaluation provide demonstrable results of the effectiveness of implementation projects. This Roundtable will examine how the effectiveness of implementation can be measured.”
Overview
• USG Peace and Security Indicators – Program Level (Where is the impact?)
• Peace and Security Index – Country Level (Where is the attribution?)
• Need to focus on regions, or sub-regions of need• Two evaluations conducted for USAID in
unstable regions of Africa (2010-2013) provide some lessons for future exploration
USG Peace & Security Indicator List (Examples)
Program Area: Counter-Terrorism• ELEMENT: PS 1.1 – DENY TERRORIST SPONSORSHIP,
SUPPORT AND SANCTUARY• INDICATOR TITLE: NUMBER OF PEOPLE TRAINED ON
COUNTERTERRORISM FINANCE BY USG PROGRAMS
• ELEMENT: PS 1.2 – DE-LEGITIMIZE TERRORIST IDEOLOGY• INDICATOR TITLE: NUMBER OF PUBLIC INFORMATION
CAMPAIGNS COMPLETED BY USG PROGRAMS
An Index for Peace and Security Program Elements as Defined by U.S. State Department
Index Elements: State Department’s annual Country Reports on Terrorism…
THAILAND There was no direct evidence that international terrorist groups were directly involved in attacks within Thailand, nor was there any evidence of operational linkages between the southern Thai insurgent groups and international terrorist networks. ….
Countering Radicalization and Violent Extremism: A range of Thai government agencies, including the Ministries of Interior and of Social Development and Human Security, and the Thai military and police academies continued to organize outreach programs to ethnic Malay-Muslims to counter radicalization and violent extremism. A small group of international non-governmental organizations also reached out to communities in the southern provinces to provide services and to identify the underlying causes of the area's violence…..
State Department’s Annual Country Reports on Terrorism (excerpt)
Index Elements: Failed States Index
Failed States Index
Index Elements: World Bank Institute
Index Elements: Human Rights
Index Elements: Homicide Rate
International Crime - Trafficking
Peace and Security Issues Are Often Very
Localized…within a Country…
…or a larger region
Proposed Indicators from Two Evaluations
Data Collected in Five Countries (2010-12)
Based on USAID Analysis of Drivers of Violent Extremism
• Perception that the international system is unjust; a mechanism through which Muslims are oppressed and their culture de-valued
• Perceptions of social exclusion and marginality
• Real or perceived societal discrimination
• Frustrated expectations and relative deprivation
• Unmet social and economic needs
• Local conflicts
Output Indicators – From “F” Common Indicator List
Indicator Source Definition
Number of public information campaigns completed by USG programs
Implementing Partners
Public information campaigns include radio, public service announcements, print media, internet postings, etc. that provide information de-legitimizing terrorist ideology
Number of community development projects
Implementing Partners Measures the local government’s ability to execute/implement community-initiated development projects in what are considered to be ungoverned spaces.
Number of individuals from at-risk groups that have been reached Implementing Partners Number of people from ‘at-risk’ groups
who have participated in USAID-sponsored civic and/or social activities specifically conducted to benefit members of such groups Activities can include but are not limited to, conferences, dialogues, training courses, workshops, and/or seminars;
Acceptance of violence in name of religion
Indicator Source Definition
Perception of terrorism as permissible under Islamic Law.
Survey Question. Answer (on a 1 to 5 Scale) to question: Are Al Qaeda’s violent activities permitted under Islamic law?
Community AlienationIndicator Source DefinitionPerception of individual power within community
Survey Question. Answer (on a 1 to 5 Scale) to question:? How much can an ordinary person do to improve community problems?
Satisfaction level with community services
Survey Question. Answer (on a 1 to 5 Scale) to question: What is your level of satisfaction with your access to services and resources in your community?
Community Participation Indicator Source Definition
Participation in Community Decision Making
Survey Question.
Answer (on a 1 to 5 Scale) to question: Do you participate in decision making in your community?
Level of engagement with local government?
Survey Question.
Answer (on a 1 to 5 Scale) to question: Have you contacted local government?
Results: Mogadishu
1.01.52.02.53.03.54.04.55.0Attendance of a community meeting
Raised an issue with authorities
Participation in decision-making
Satisfaction with local governmentdecision making
How much can an ordinary person do?o?
Do youth associations make apositive contribution?
View of youth by leadersSupported by youth organizations?
Preparedness to enter the job market
Optimism about a better future
Importance of training vs familyconnections
Justification of using violence in thename of Islam
Violent activities permitted underIslamic law?
Full beneficiaries (100) Partial beneficiaries (80) Comparison group (100)
What Does this Tell Us?
• Indicators based on targeted surveying can assess attitudes at sub-national level - information not always available at country level.
• Provide information beyond output level• Can address attribution through use of
counterfactuals• Use of standard survey questions allows
comparability