Quarterly Performance Report - pdf.usaid.govpdf.usaid.gov/pdf_docs/PA00N37P.pdf · Prepared by...

April 15, 2013

This publication was produced for review by the United States Agency for International

Development. It was prepared by DevTech Systems, Inc. under Contract Number AID-520-C-

13-00001.

Quarterly Performance Report

January 1, 2013 – March 31, 2013

i

Monitoring & Evaluation Program

Quarterly Performance Report: January 1, 2013 – March 31, 2013

Submitted to:

Josefina Martinez, Contracting Officer’s Representative

USAID/Guatemala

Submitted by:

DevTech Systems, Inc.

Contract AID-520-C-13-00001

DISCLAIMER:

This report is made possible by the support of the American People through the United State Agency for

International Development (USAID). The contents of this report are the sole responsibility of DevTech and do not

necessarily reflect the views of USAID or the United States Government

USAID/Guatemala Monitoring & Evaluation Program

Draft Work Plan: January 2013-September 2013

AID-520-C-13-00001

Prepared by DevTech Systems, Inc., IARNA, ASIES and Geotecnologica.

ii

TABLE OF CONTENTS

ACRONYMS ........................................................................................................................ III

EXECUTIVE SUMMARY ...................................................................................................... 1

I. INTRODUCTION ............................................................................................................... 4

II. MONITORING AND EVALUATION PROGRAM RESULTS ............................................. 5

Result 1: Statistical Tools for Baseline, Mid- and Final-term Data Collection Developed ................. 5

Result 2: Local Capacity in Monitoring and Evaluation, including Data Collection of National and Sub-National Indicators, increased ..................................................................................................... 6

Result 3: Clearinghouse, GIS, Environmental Compliance and other M&E Services Developed and Delivered ............................................................................................................................................. 7

Result 4: Mission Program and Project Evaluations Conducted and Results Reported ................... 9

Result 5: Monitoring and Evaluation Services for other USAID/Guatemala Offices, Conducted Upon Request............................................................................................................................................. 10

III. SUMMARY AND CONCLUSION ................................................................................... 11



AID-520-C-13-00001

Prepared by DevTech Systems, Inc., IARNA, ASIES and Geotecnológica.

iii

ACRONYMS

APSAN Support for the Food Security Policy, European Union

ASIES Asociación de Investigación y Estudios Sociales

CBA Cost Benefit Analysis

CDCS Country Development Cooperation Strategy

CEA Cost-Effectiveness Analysis

CH Clearinghouse

CONASAN National Food Security Council

CONAP National Council of Protected Areas

COP Chief of Party

COR Contracting Officer’s Representative

DGO Democracy & Governance Office

DO Development Objective

DQA Data Quality Assessements

EGO Economic Growth Office

EMP Environmental Mitigation Plan

ENA Agriculture Census

ENCOVI Household Expenditure Survey

ENSMI Maternal and Infant Health Survey

ENRD National Strategy for the Reduction of Deforestation

EU European Union

FTF Feed the Future

FY Fiscal Year

GCC Global Climate Change

GIS Geographic Information System

GOG Government of Guatemala

HEO Health & Education Office

IARNA Instituto de Agricultura, Recursos Naturales y Ambiente, Landivar University

INE National Statistical Institute

INGUAT National Tourism Institute

IP Implementing Partner

IR Intermediate Result

MAGA Ministry of Agriculture

MARN Ministry of Environment

M&E Monitoring and Evaluation

MDG United Nations Millennium Development Goals

MEO Mission Environmental Officer

MIS Management Information System

MOU Memorandum of Understanding

NGO Non-Governmental Organization

OCSE/SAN Sector Coordination Office for Food Security and Nutrition Statistics

PAT Poverty Assessment Tool



AID-520-C-13-00001


iv

PLANOCC Western Highlands Food Security Plan

PERSUAP Pesticide Evaluation Review and Safe Use Action Plan

PESAN National Food and Nutritional Security Strategy

PMP Performance Monitoring Plan

REO Regional Environmental Officer

RF Results Framework

SEGEPLAN Secretariat for Planning and Programming/Office of the President

SESAN Secretariat for National Food Security and Nutrition

SINASAN National System for Food Security

SJSRP Security and Justice Sector Reform Project

STTA Short Term Technical Assistance

UPIE Strategic Policies and Information Unit, Ministry of Agriculture

USAID United States Agency for International Development

USG United States Government

WEAI Women’s Empowerment in Agriculture Index



AID-520-C-13-00001


v

This page is intentionally left blank.



AID-520-C-13-00001

Prepared by DevTech Systems, Inc., IARNA, ASIES and Geotecnologica.

1

EXECUTIVE SUMMARY

The M&E Program is designed to support USAID/Guatemala’s Economic Growth Office

(EGO) by providing the necessary information to facilitate informed program management

decisions to improve the effectiveness of development assistance interventions, and to shape

longer-term strategic direction and decision making, in particular, in regard to the

implementation of the Feed the Future (FTF) and Global Climate Change (GCC) initiatives.

The Program is working to develop a Monitoring and Evaluation (M&E) mechanism that

provides continued performance monitoring, verification and impact evaluation of program

results, including environmental compliance.

The Program has a 3-pronged strategy which includes monitoring and evaluating EGO

programs, measuring USG “whole-of-government” progress towards EGO programs, and

measuring country-level progress towards M&E of EGO priorities.

Overview of achievements by Result for the quarter:

Result 1: Statistical tools for Baseline, Mid- and Final-term Data Collection

Developed.

o The Program held a number of meetings to initiate program implementation.

This included: (1) A team integration meeting with local partners (IARNA,

ASIES, Geotecnológica); (2) Two planning meetings focusing on the

development of the Monitoring and Evaluation Program conceptual

framework that integrates Feed the Future and Global Climate Change

initiatives with relevant national public policies and (3) Meetings with

implementing partners (Agexport, ANACAFE, and Rainforest Alliance) to

initiate Performance Monitoring Plan (PMP) review for their projects.

Result 2: Local Capacity in Monitoring and Evaluation, including Data Collection of

National and Sub-National Indicators Increased.

o Survey on administrative capacity with local partners conducted.

o Training workshop held for subcontractors on administrative and financial

management, both for technical and administrative staff members.

Result 3: Clearinghouse, Management Information System (MIS), Geographic

Information System (GIS), Environmental Compliance, and other M&E Services

Developed and Delivered.

o Program subcontractor traveled to Colombia to learn more about

USAID/Colombia’s MONITOR system and evaluation functionality for

Guatemala.

o First meeting with implementing partners’ IT technicians (ANACAFÈ and

AGEXPORT) to discuss compatibility of their IT systems with USAID’s

M&E Clearinghouse system.1

1 The Clearinghouse will include: (a) a web-based Management Information System (MIS)

which provides comprehensive information on the status and progress of USAID activities;



AID-520-C-13-00001


2

o First meeting with EGO Implementing Partners to introduce

USAID/Colombia’s MONITOR System and discuss modifications for local

use.

o Meeting at USAID with Mission Offices (HEO, DGO, and EGO) to introduce

USAID/Colombia’s MONITOR System and discuss modifications for local

use.

o Meeting at ANACAFÉ to present USAID/Colombia’s MONITOR System to

USAID partners involved in the Feed the Future initiative and solicit

feedback.

o Assessment of lessons learned from Universidad de Antioquia, Colombia

consultancy on the development of USAID/Colombia’s MONITOR

Environmental Compliance Module.

Result 4: Mission Programs and Project Evaluations Conducted and Results

Reported.

o Not applicable.

Result 5: Monitoring and Evaluation Services for other USAID/Guatemala offices,

Upon Request

o The Program responded to the Democracy and Governance Office’s Request

for Proposal to develop an evaluation for the Security and Justice Sector

Reform Project.

Opportunities

In the first quarter of Program implementation, the team has identified two major

opportunities to leverage. Firstly, several EGO projects have recently begun implementation

– namely the rural value chain programs implemented by ANACAFE and Agexport (initiated

in October 2012) and the environmental program implemented by Rainforest Alliance

(initiated in February 2013). Therefore, it is the opportune time to initiate work with the

implementing partners on their respective Monitoring & Evaluation/Performance Monitoring

Plan (PMP) and indicator development in order ensure they have solid indicators from the

beginning of project implementation.

In addition, USAID/Guatemala has expressed the requirement to demonstrate results as soon

as possible for the Management Information System (MIS) that the Monitoring and

Evaluation Program will develop. As such, there is a major opportunity to adapt

USAID/Colombia’s MONITOR system to Guatemala’s needs by taking the IT platform,

achievements, and lessons learned into account in the MIS adaptation process.

(b) a Geographic Information System; (c) Mission-wide environmental compliance; and (d)

M&E services.



AID-520-C-13-00001


3

Activities Planned for the Next Quarter

Review the EGO’s programs and projects, with the purpose of being able to offer

M&E services that meet USAID projects’ needs.

Continue reviewing Implementing Partners’ M&E/PMPs, indicators and goals.

Carry-out an analysis of M&E capacity present within GOG institutions and local civil

society organizations, academia and private sector.

Continue adapting USAID/Colombia’s MONITOR Environmental Module to local

needs.

Gather feedback from USAID on the Mission’s needs in order to continue adapting the

MONITOR system to local requirements.

Initiatethe design of performance and/or impact evaluations including baseline

surveys.

Start-up of Component 5, upon request



AID-520-C-13-00001


4

I. INTRODUCTION

Many of the activities carried out during this first quarter of the Monitoring and Evaluation

Program were associated with launching the multi-year program. Due to its entirely

Guatemalan team (including its subcontractors IARNA, ASIES and Geotecnológica), the

Program was able to engage in immediate technical and operational start-up.

The first weeks of Program start-up were devoted to initial introductions and meetings at the

Mission. These early meetings served to clarify Mission expectations regarding the scope of

the services and support the Program will provided. Over the course of the quarter,

DevTech’s Chief of Party initiated work and a Contracts, Grants and Operations Manager

was recruited. The Program also engaged in recruitment to identify a replacement for its

M&E Specialist key personnel position. The Program also issued Letters of Authorization to

its three subcontractors.

The Program’s operational start-up included an initial visit of the COP (Juan Carlos Mendez)

to Arlington, VA for orientation (travel was funded by the contractor). He was trained on

DevTech policies and procedures, met with technical staff to discuss the contract and worked

with home office staff to discuss start-up operations. The Home Office Program Manager

also made a two week trip to Guatemala to attend the Program’s post-award kick-off meeting

with USAID on March 11, 2013, initiate the registration process for DevTech’s local branch,

provide training to the local Contracts, Grants and Operations Manager, meet with

subcontractors to discuss operational start up, initiate an assessment of their administrative

capacity, and provide training in these area.

The Quarterly Report is organized by Result, with reference to the Work Plan for each area,

and includes (1) Activities, achievements, and deliverables; (2) Opportunities, problems,

delays, and proposed solutions; (3) Key meetings; and (4) Activities planned for the next

quarter. The technical report is accompanied by the quarterly financial report on invoiced

expenditures by line item and projected expenditures for the next quarter, submitted under

separate cover.



AID-520-C-13-00001


5

II. MONITORING AND EVALUATION PROGRAM RESULTS

The following information summarizes the advances made thus far under each Result. On a

cross-cutting level, much of the quarter was devoted to work planning sessions with USAID,

the Government of Guatemala (GOG), implementing partners, and subcontractors to identify

priority needs and develop strategies to address them.

Result 1: Statistical Tools for Baseline, Mid- and Final-term Data Collection Developed

During the life of the Program, the team will work with USAID’s Office of Economic

Growth (EGO) implementing partners (IPs) in Guatemala to engage in baseline data

collection at the community and municipal level and establish targets.

Activities, achievements, and deliverables:

Over the course of the quarter, the Program engaged in discussions with its subcontractors

(IARNA, ASÍES and Geotecnológica) on the development of a conceptual framework for

evaluating USAID programs in Guatemala. Based on discussions with counterparts, this

framework will be based on the Feed the Future (FTF) and Global Climate Change (GCC)

initiatives, taking into consideration the national context, namely the GOG’s Zero Hunger

Pact, the National Policy on Food Security and Nutrition, and the National Policy on Global

Climate Change.

The Program also initiated the review of results frameworks, indicators and goals for EGO’s

rural value chain programs (implemented by ANACAFÉ and AGEXPORT) and its

environmental program (implemented by Rainforest Alliance). Over these preliminary

meetings, the Program reviewed draft PMPs to verify the coherence between USAID/EGO

requirements and the Implementing Partners’ proposed plans. This work will continue next

quarter.

Opportunities, problems, delays, and proposed solutions:

GOG collaboration is key to achieving the goals laid out under several Program results. As

such, the Program feels there is an opportunity to engage GOG support from the moment

Program implementation starts by designing a results framework that clarifies how USAID

and GOG policies and program interact to achieve food security and climate change goals in

the Western Highlands.

Key meetings:

February 18 and 20, 2013 - Meeting with AGEXPORT and ANACAFÉ, respectively,

to initiate PMP review.

o AGEXPORT meeting participants: IP: Ivan Buitón, COP; Carlos

Urízar, Director, Office of Agriculture and Angel Lopez, M&E



AID-520-C-13-00001


6

Specialist, and from USAID: Josefina Martinez, M&E Program COR,

and Glenda Paiz, Rural Value Chains Project AOR.

o ANACAFÉ meeting participants: IP: Mario Aragón, COP; Mario

Gaytán, Technical Director and Carlos Chacón, M&E Specialist, and

from USAID: Josefina Martinez, M&E Program COR, and Glenda

Paiz, Rural Value Chains Project AOR.

March 1 and 8, 2013 – Planning meetings with Sub-contractors (IARNA, ASÍES and

Geotecnológica) to address Work Plan development and the design of the Program’s

conceptual framework.

March 14 – Meeting with Rainforest Alliance (Beatriz Avalos, HQ M&E Coordinator;

Jorge Mario Matute, M&E Specialist in Guatemala) and USAID (Josefina Martínez,

M&E Program COR; Dani Newcomb, Communities, Nature and Climate Program

AOR, and Jenniffer Tika, Alternate AOR) to initiate PMP review.

Activities planned for the next quarter:

The Program will hold meetings to gain a detailed understanding of the structure and

needs of each Program under the Economic Growth Office in order to best adapt and

offer M&E services to implementing partners.

The Program will continue to review M&E Plans/PMPs, indicators and goals for

EGO’s Implementing Partners in order to lay the groundwork for indicator selection

and evaluation baseline development.

Result 2: Local Capacity in Monitoring and Evaluation, including Data Collection of National and Sub-National Indicators, increased

Under Result 2, the Program works to strengthen institutional and statistical capacity of GOG

institutions and among Guatemalan academic, non-governmental and private research

organizations to monitor food security and environmental/biodiversity progress and to

institutionalize sustained and systematic monitoring and evaluation beyond the life of

Program assistance. The Program also supports GOG efforts to measure Guatemala’s

progress towards the United Nations Millennium Development Goals (MDGs).


This quarter, the Program initiated efforts to build local capacity in monitoring and

evaluation by strengthening local organizations’ technical and administrative capacity so that

they will be able to obtain direct support from international donors in the future, providing

sustainability to M&E projects and programs. As an initial step in this direction, the Program

developed and administered an administrative capacity survey for its subcontractors (which

collectively represent civil society, academia and the private sector) in order to evaluate

strengths and weaknesses in each entity, including areas for future training. The Program also

initiated training among these entities with a half day capacity building event. Subcontractors



AID-520-C-13-00001


7

were provided with an introduction to USAID policy in regards to contracting, procurement,

and financial management, and operations (timekeeping, travel policies, etc).


As part of Result 2, the Program is required to select local organizations (civil society,

academia and private sector representatives) that have the administrative and technical

potential to independently manage USAID resources and carry out M&E services in the

future. The Program identified that its subcontractors (IARNA, ASIES, and Geotecnológica),

which are already strong in EGO technical areas (rural development, agriculture, nutrition,

food security, natural resource management and environmental compliance), are ideal

candidates for the capacity development processes and to become members of a consortium

that will be able to address international- and national-level needs for M&E services in the

future.

One subcontractor, IARNA, has local presence and infrastructure in Guatemala’s Western

Highlands – a priority area for USAID and GOG FTF implementation. As such, this local

center could potentially become a regional hub for M&E activities.

Key meetings:

March 22, 2013 – Workshop at IARNA offices for subcontractors on administrative

and financial procedures.

March 25, 2013 – Meeting at the Ministry of Agriculture. The Planning Directorate

from the Ministry of Agriculture requested direct support from the Monitoring and

Evaluation Program to review their results framework, indicators and the required

baselines for their programs, particularly their family agriculture, food safety and

nutrition program. This meeting was attended by Edwin Ardón, Planning Director, and

his technical team.


The Program will carry out an analysis of GOG institutions’ M&E capacity to identify

areas for improvement.

The Program will continue to analyse the gaps in administrative capacity revealed in

the administrative capacity survey with local organizations from civil society,

academia and private sector. In order to develop an appropriate training plan.

Result 3: Clearinghouse, GIS, Environmental Compliance and other M&E Services Developed and Delivered

Under Result 3, the Program will establish a Clearinghouse for M&E services and

information tracking on the status and progress of USAID programs, including a web-based

Management Information System (MIS) which provides comprehensive information on the



AID-520-C-13-00001


8

status and progress of USAID activities (3.1); a Geographic Information System (3.2), a

mechanism for Mission-wide environmental compliance (3.3); and M&E services including

development or updates of Performance Management Plans and Data Quality Assessments

(3.4).


This quarter, the Program took the first steps towards evaluating the USAID/Colombia

MONITOR Management Information System (MIS) for use in Guatemala. The Program is

initially evaluating the adaptability of the following MIS modules for use in Guatemala: the

Monitoring and Evaluation Module, the Environmental Compliance Module, and the

Geographic Information Module. To initiate this work, the Program’s subcontractor visited

Bogotá, Colombia from February 27 to March 3, 2013 to meet with MONITOR’s IT manager

and learn more about the system, the IT platform, and functionality. Based on this visit and

further study of the IT platform, the Program made presentations over the course of the

quarter to a range of USAID technical offices and to a range of implementing partners (see

dates below). This provided stakeholders (USAID and implementing partners) with

information on how the different MIS modules would facilitate their responsibilities related

to environmental compliance and PMP/indicator management. These sessions also allowed

the Program to begin to collect feedback regarding needs and concerns from local

stakeholders.

Figure 1. Monitoring and Evaluation Clearinghouse Structure


USAID values organizational learning and works to draw on regional/global successes and

lessons learned. The Program is assisting USAID/Guatemala in taking advantage of the



AID-520-C-13-00001


9

MONITOR MIS developed for Colombia and adapting it for use in Guatemala. This allows

USAID to reach its monitoring and evaluation goals in Guatemala in a more time and cost

efficient manner.

Key meetings:

March 12, 2013 – at AGEXPORT - Initial meeting with USAID representatives and

EGO Implementing Partners to introduce Colombia’s MONITOR system and initiate

discussions on its adaptation based on local requirements.

March 18, 2013 – at USAID – Met with Mission offices (HEO, DGO, EGO) to

introduce Colombia’s MONITOR system and initiate discussion on major

modifications required.

March 21, 2013 – at ANACAFÉ - Meeting to present Colombia’s MONITOR system

to USAID’s partners involved in the Feed the Future initiative for initial discussion.


Next quarter, the Program will continue adapting MONITOR´s Environmental Module for

use in Guatemala. The Program foresees holding two meetings with the Mission

Environmental Officers (MEOs) and one with the Regional Environmental Officers (REOs)

for evaluating specific requirements for the adaptation according to official guidelines on

environmental compliance, including Regulation 216 specifications for Latin America.

Result 4: Mission Program and Project Evaluations Conducted and Results Reported

The Monitoring and Evaluation Program will work with USAID/Guatemala to help meet the

requirements generated under the new USAID Evaluation Policy. The Program will design

and conduct impact evaluations in order to measure the change in key development indicators

attributable USAID interventions, undertake Cost-Benefit Analysis (CBA) and Cost-

Effectiveness Analysis (CEA) to help USAID to analyze what it is buying with its resources

and how funds should be allocated, and at USAID’s request carry out assessments that will

form part of EGO’s results-oriented program planning and implementation activities.


USAID requested that the Program investigate the impact of Roya del Café (coffee rust) on

production and yields in Guatemala, particularly in USAID’s intervention areas (five

departments located in the Western highlands).


In June 2013, the Program will initiate development of CBA and CEA methodologies.

At USAID request, the Program will carry out an assessment on the impact of coffee

rust (Roya del Café) in May 2013, in support of USAID implementing partner,

ANACAFÉ, that is implementing EGO’s rural value chain program.



AID-520-C-13-00001


10

Result 5: Monitoring and Evaluation Services for other USAID/Guatemala Offices, Conducted Upon Request

Through Result 5, USAID Mission offices have the opportunity to buy-into Program M&E

services such as baseline, midterm and final performance and impact evaluations, engage in

geo-mapping of health/nutrition and governance projects, engage in statistical capacity

building of GOG institutions and other key stakeholders in health/nutrition and governance,

and/or provide other M&E services.


To date, the Program has received a Request for Proposal (RFP) from the Democracy and

Governance Office (DGO) to design a three-phase performance evaluation (baseline, mid-

term, and final-term) for the newly designed Security and Justice Sector Reform Project

(SJSRP) and carry out baseline analysis and data collection and phases 1 and 2. The Program

submitted its proposal to USAID in March and is awaiting a response.


Initiate evaluation design and baseline data collection based, upon USAID request.



AID-520-C-13-00001


11

III. SUMMARY AND CONCLUSION

Thanks to the involvement of local partners, the Monitoring and Evaluation Program was

able to simultaneously initiate administrative operation and technical activity. This allowed

the Program to hit the ground running and begin to develop successful relationships with

USAID and implementing partners.

However, it is important to emphasize that M&E is a concept in development at the local

institution level, as is a focus on gender equity. This represents a challenge for Program

implementation and will require a number of awareness raising activities.

For more information, contact:

DevTech Systems, Inc. 1700 North Moore St.

Suite 1720 Arlington, Virginia 22209

703-312-6038 www.devtechsys.com

Quarterly Performance Report - pdf.usaid.govpdf.usaid.gov/pdf_docs/PA00N37P.pdf · Prepared by...

Documents

Transcript of Quarterly Performance Report - pdf.usaid.govpdf.usaid.gov/pdf_docs/PA00N37P.pdf · Prepared by...