file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes....

26
BRACED MTR Evaluation Matrix: Introduction and Explanatory Notes February 2016

Transcript of file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes....

Page 1: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

BRACED MTR Evaluation Matrix:Introduction and Explanatory Notes

February 2016

Page 2: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

Introduction and Explanatory NotesThis MTR Matrix Document accompanies the MTR Terms of Reference Template and Guidance. The MTR evaluation matrix template was introduced to IPs during the February 2016 BRACED Annual Learning Event in Dakar. The evaluation matrix template is a word table, which captures the key elements of Note 7.It is designed for you to customise it to the details of your projects while meeting the requirements of the MTR. It sets out the main over-arching Evaluation Questions, some key evaluation sub-questions in line with Realist Evaluation approaches, with further columns for you to add more detailed sub-questions, data sources or planned data collection method, and proposed data analysis method.It is important to note that the evaluation matrix template is intended to be used as an MTR design and planning tool, rather than a data collection and reporting instrument. The main uses of the MTR Evaluation Matrix are to:1) Help you to plan your MTR data collection and data analysis process and in structuring your MTR report;2) Assist in thinking about the kinds of questions you need to include in your data collection instruments for the MTR to capture information to help to answer the question “How, where, when and why do BRACED interventions work, and what can be learned?”;3) Support you in thinking about things that help you to achieve your objectives and those that hinder you, to inform potential course correction of your project; and4) Help you to structure and summarise your findings in your MTR report so a realist synthesis can be carried out by the KM Evaluation Synthesis and Support Team to generate programme-level recommendations about what works, for whom, in what circumstances, and why. When you submit your Terms of Reference in April 2016, it should include a completed evaluation matrix that works for you and is tailored to your project. You are required to complete the evaluation matrix in columns 3, 4 and 5 . Column 1 sets out the overarching evaluation questions (EQ) for the MTR and Column 2 some key sub-questions under each EQ. Column 3 is for you to add further sub-questions that relate specifically to the nature and context of your project or to help you answer the EQs, and will make the evaluation matrix unique to you. Column 4 is to record the data source. You may be able to answer some aspects of the EQs and sub-questions from the information in documents such as your Annual Report to the Fund Manager. In column 5 you will list your data analysis method/process. Some examples are given in Columns 3, 4 and 5 just to start you off.You are NOT required to answer the questions posed in the evaluation matrix when you submit it with your ToR in April 2016. You will collect and analyse data during your MTR and the questions will be answered in your MTR report, to be submitted in October 2016. You need to review the key sub-questions included in column 2 to see if they cover

everything in your MTR. If not, you should add extra sub-questions as needed and adjust the wording of the questions. If particular questions seem irrelevant or beyond the scope of your MTR, please discuss them with us. You may be able to adapt the questions or remove them if need be. However, the matrix is designed to minimum requirements laid out in Note 7, so you may need to justify any changes you make.

The key task is to tailor and add sub-questions in column 3 to the nature and context of your project. For example, a project with a strong gender focus might wish to include more sub-questions about gender throughout the evaluation matrix. Tip: Listing your activity packages will help you think about how relevant the suggested sub-questions are. We strongly suggest that you do this, though it is not a requirement for evaluation matrix sign-off.

Tip: You may also find it helpful to list possible interview questions against the main sub-questions in the evaluation matrix. This will help you decide on what data you already have (e.g. from your M&E plan indicators), what your MTR data collection instruments will need to capture, and what methods will be appropriate. Again, this is not a requirement for evaluation matrix sign off.

If you want to use a different approach you are free to do so. However, you should remember that you are required to produce an evaluation matrix of some form, and demonstrate that you are covering the requirements set out in Note 7.

Guidance for BRACED Project Mid-Term Reviews, February 2016 2

Page 3: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

Guidance for BRACED Project Mid-Term Reviews, February 2016 3

Page 4: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

BRACED MTR Evaluation Matrix1 2 3 4 5

Evaluation Question (EQ)

Key Sub-questionsWhat works, for whom, in what circumstances, and why?

Project Sub-questions(for each package of activities)

Data source and collection approach

Data analysis approach

What are your project’s key intervention packages and how are they defined within your project theory of change?

C1 Please describe your main packages of activities – the activities you are actually carrying out.

a. The packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses.

b. The packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses

c. The packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and

Commercial Crop (Business Farming): Training on selling crops; Access to credit; incorporate seasonal forecast to decision making; access to commercial/local seeds.

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. How many farmers are associated with the collection center?

ii. What are the types of services that are being provided by the collection center?

iii. How many essential oil farmers are associated with the DU?

iv. What are the services that are being provided by the DU?

v. How many farmers are benefited by MUS?

vi. What are the benefits of using MUS?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. How many LAPA’s have been formed through project?

ii. How many people are benefited through LAPA?

iii. How much resources have been

Annual Reporting data Focus group discussion

Annual Reporting data Focus group discussion Key informant

interviews with programme staff/ project beneficiaries

Document review

Analysis of focus group discussion

Document review

Analysis of focus group discussion

Analysis of key informant interviews

Guidance for BRACED Project Mid-Term Reviews, February 2016 4

Page 5: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

integration into DRR approaches

d. The packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels

leveraged so far in implementing LAPA activities?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. How many women’s are in the key decision making position in project promoted technologies i.e. CC, DU and MUS?

ii. How are you ensuring women participation in project promoted committees?

iii. What mechanisms have you adopted to empower women beneficiaries of the project?

d. Research findings in relation to integration of DRR and CCA.

I. What researches have been done to generate knowledge on improving policies related to DRR and CCA integration?

C2 How is this package of activities described within the project theory of change?

a. How are the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses described within the project theory of change?

b. How are the packages of activities (LAPA and EWS)

What changes do/did you anticipate delivering? At midpoint? At endpoint?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. How much is the average annual income from the project promoted interventions?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. What are the benefits that farmers are receiving from implementation of LAPA?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain

Review of Project Documentation

Annual Reporting data

Interviews with project team members

Review of Project Documentation

Annual Reporting data

Focus group discussion Interviews with

project team members

Interviews with the government

Document Review

Analysis of interview

Document Review

Analysis of focus group discussion

Analysis of interview

Guidance for BRACED Project Mid-Term Reviews, February 2016 5

Page 6: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses described within the project theory of change?

c. How are the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches described within the project theory of change?

d. How are the packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels described within the project theory of change?

composite score.i. What are the visible impacts seen

in women decision making power in agriculture production?

ii. What are the visible impacts seen in women decision making power over productive resources?

iii. What are the visible impacts seen in women decision making power over control of the income?

iv. What are the visible impacts seen in women leadership in the community?

v. What are the visible impacts seen in women time use?

d. Research findings in relation to integration of DRR and CCA.

i. What policy influences / changes have been observed as a result of project promoted interventions?

stakeholders and project staffs

C3 What is the context of this package of activities?

a. What is the context of the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses

Conditions that influence whether mechanisms operate, for different groups of people, such as social, economic, cultural, institutional setting: Individuals - their own behaviour, biases; Interpersonal - how different people relate to each other;  Institutional – policies of an NGO; Wider external factors (infrastructural) – ISIS, Al-Shabaab, droughta. Is operating environment

conducive to allow interventions to impact on local capacity?

Review of Project Documentation

Annual Reporting data

Focus group discussion Key informant

interviews with programme staff/ project beneficiaries

Review of Project Documentation

Document Review

Analysis of interview

Document

Guidance for BRACED Project Mid-Term Reviews, February 2016 6

Page 7: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

described within the project theory of change?

b. What is the context of the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses described within the project theory of change?

c. What is the context of the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches described within the project theory of change?

d. What is the context of the packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels described within the project theory of change?

b. Do Nepal government have given high priority to climate change adaptation and disaster risk reduction? If yes how?

c. Do private sector have conducive environment for their development and growth?

d. Are the project beneficiaries ready and cooperating with the project through their active participation and involvement?

Annual Reporting data

Focus group discussion Key informant

interviews with programme staff/ project beneficiaries

Review Analysis of

Focus Group Discussion

Analysis of interview

EQ1 - To what extent have particular packages of interventions delivered in terms of strengthened resilience?

1.1 What are the main changes and results you have seen from your package of activities?

a. What are the main changes and results you have seen from the packages of

Are we seeing people using the seeds, advice, credit?

Is there a change in yield? In income? Are the things that will allow people

to manage "crisis" in place? Are you seeing changes in the social

standing of the women involved? What have been unexpected changes

and results of this package of

Key informant interviews with programme staff/ project beneficiaries

Annual Reporting Data Review of baseline

survey

Analysis of interview transcripts

Document review

Analysis of HH survey

Guidance for BRACED Project Mid-Term Reviews, February 2016 7

Page 8: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

a. To what extend the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses in terms of strengthening resilience?

b. To what extend the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses in terms of

activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

b. What are the main changes and results you have seen from the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses described within the project theory of change?

c. What are the main changes and results you have seen from the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches described within the project theory of change?

d. What are the main changes and results you have seen from the packages of

activities?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. Are we seeing people cultivating vegetables in their farm?

ii. Are we seeing people cultivating essential oil in their farm and community forest?

iii. What changes have you observed after receiving training from Anukulan project?

iv. Are people adopting climate smart technologies?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. What LAPA activities have been implemented by the communities?

ii. What information has been provided by DDRC for enhancing early warning?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. Are you seeing changes in the social status of the women involved? If yes what are the changes seen?

d. Research findings in relation to integration of DRR and CCA.

i. What has been the major findings of the researches which project has been carrying out in relation to integration of DRR and CCA?

Focus group discussion HH survey

Annual Reporting data Focus group discussion Key informant

interviews with programme staff/ project beneficiaries

HH survey Review of project

reports

• Analysis of interview transcripts• Document review• Analysis of HH surveyDocument review

Guidance for BRACED Project Mid-Term Reviews, February 2016 8

Page 9: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

strengthening resilience?

c. To what extend the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches in terms of strengthening resilience?

d. To what extend the packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels in terms of strengthening resilience?

activities that lead to improved policies integrating DRR and CCA in place at local and national levels described within the project theory of change?

1.2 To what extent has this package of activities led to the changes and results you were expecting?

a. What are the main changes and results you have seen from the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

b. What are the main changes and results you have seen from the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses described within the project theory of change?

Summary narrative description of emerging changes or pathways, expected and unexpected, explanation of intervention contribution

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. What are the emerging changes seen in the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) implementation process?

ii. What are the expected and unexpected consequences seen in the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) implementation process?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. What are the emerging changes seen in the packages of activities (LAPA and EWS) implementation process?

ii. What are the expected and

Key informant interviews with programme staff/ project beneficiaries

Annual Reporting Data Review of baseline

survey Focus group discussion HH survey

Key informant interviews with programme staff/ project beneficiaries

Annual Reporting Data Review of baseline

survey Focus group discussion

Analysis of interview transcripts

Document review

Analysis of HH survey

Analysis of interview transcripts

Document review

Guidance for BRACED Project Mid-Term Reviews, February 2016 9

Page 10: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

c. What are the main changes and results you have seen from the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches described within the project theory of change?

d. What are the main changes and results you have seen from the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels described within the project theory of change?

unexpected consequences seen in the packages of activities (LAPA and EWS) implementation process?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. What are the emerging changes seen in the packages of activities (WEAI) implementation process?

ii. What are the expected and unexpected consequences seen in the packages of activities (WEAI) implementation process?

d. Research findings in relation to integration of DRR and CCA.

i. What are the emerging changes seen in the packages of activities in relation to integration of DRR and CCA implementation process?

ii. What are the expected and unexpected consequences seen in the packages of activities in relation to integration of DRR and CCA implementation process?

EQ2 – Specifically focusing on understanding ‘mechanisms’, how and why have particular intervention packages led to observed results and changes?

2.1 How and why have the changes you’ve seen come about? [for each activity package, focus on the causal forces or powers that explain why a change happens and the context that influences this]

How and why have the changes you’ve seen come about on the packages of activities (Collection

Was it what you provided? The right seeds?Agronomic advice? Access to Credit?

Was it how you did it? The right people were selected to participate? Your trainers were excellent? Your organisations relationship to other local institutions? Government? Other NGOs?

Was it external factors? Good prices? Support from traditional leadership? Support from key local government

Key informant interviews

Annual Reporting Data IPS interview including

government officials

Key informant interviews

Annual Reporting Data IPS interview including

Analysis of interview transcripts

Document review

Analysis of interview

Guidance for BRACED Project Mid-Term Reviews, February 2016 10

Page 11: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

Specifically focusing on understanding ‘mechanisms’, how and why have the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses led to observed results and changes?

Specifically focusing on understanding ‘mechanisms’, how and why have the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses led to observed results and changes?

Specifically focusing

Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

How and why have the changes you’ve seen come about on the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses?

How and why have the changes you’ve seen come about on the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

How and why have the changes you’ve seen come about on the packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels?

people? Timing? People were simply ready for

change?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. From where you purchased the vegetable and essential oil crop seeds? What are the inputs you used in those crops? From where you purchased those inputs?

ii. How was the training facilitated to the local farmers? How it had been different from the other trainings received?

iii. If there were not project intervention what you have cultivated in the same piece of land?

iv. How was the project beneficiaries selected? How project confirms that the right project beneficiaries were selected?

v. Why the beneficiaries were interest to adopt the project interventions?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. What are the reasons for selecting the VDCs / Municipalities for LAPA preparation?

ii. Do LAPA is accepted by the community? If yes what are the benefits from LAPA?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI)

government officials Focus group discussion

transcripts Document

review Analysis of

focus group discussion

Guidance for BRACED Project Mid-Term Reviews, February 2016 11

Page 12: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

on understanding ‘mechanisms’, how and why have the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches led to observed results and changes?

Specifically focusing on understanding ‘mechanisms’, how and why have the packages of activities that lead to improved policies integrating DRR and CCA in place at local and national levels led to observed results and changes?

score measured through a five domain composite score.

i. Do community were ready to accept women leadership? If yes how?

d. Research findings in relation to integration of DRR and CCA.

i. How have the local government people support for integration of DRR and CCA

2.2 What has your project learned about delivering these packages of activities?

What has your project learned about delivering the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

What has your project learned about delivering the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses?

What has your project learned

What have you learned about your work? About resilience? About your theory of change from this process?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. What you have learned from delivering the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

ii. How you have linked the the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to

IPS interview Key informant

interviews

IPS interview Key informant

interviews Focus group

discussion

Analysis of interview transcripts

Analysis of interview Analysis of focus group discussion

Guidance for BRACED Project Mid-Term Reviews, February 2016 12

Page 13: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

about delivering the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

What has your project learned about delivering the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels?

climate related shocks and stresses with resilience framework?

iii. How the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses had contributed to the project theory of change?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. What you have learned from delivering the packages of activities (LAPA and EWS) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

ii. How you have linked the the packages of activities (LAPA and EWS) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses with resilience framework?

iii. How the packages of activities (LAPA and EWS) that lead to increase capacity of poor people to reduce their

Guidance for BRACED Project Mid-Term Reviews, February 2016 13

Page 14: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

vulnerability to climate related shocks and stresses had contributed to the project theory of change?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. What you have learned from delivering the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

ii. How you have linked the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches with resilience framework?

iii. How the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches had contributed to the project theory of change?

d. Research findings in relation to integration of DRR and CCA.

i. What you have learned from Guidance for BRACED Project Mid-Term Reviews, February 2016 14

Page 15: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

the findings in relation to integration of DRR and CCA?

ii. How you have integration of DRR and CCA?

iii. How integration of DRR and CCA had contributed to the project theory of change?

2.3 What evidence is there that the interventions and the mechanisms that support them have the potential, within specified contexts, to deliver ‘amplified results’ and/or ‘transformational impact’?

What evidence is there that the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses and the mechanisms that support them have the potential, within specified contexts, to deliver ‘amplified results’ and/or ‘transformational impact’?

What evidence is there that the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses and the mechanisms that support them have the potential, within specified

What are the actual pathways between each package and activity and results i.e. strengthened resilience?

Do you have easily measured benchmarks of progress towards resilience to make for a better quality final evaluation?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. How the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses is linked with strengthened resilience?

ii. What are the status before for the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Review of baseline

survey Focus group discussion HH survey

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Review of baseline

survey Focus group discussion

Analysis of interview transcripts

Document review

Analysis of HH survey

Analysis of interview transcripts

Document review

Guidance for BRACED Project Mid-Term Reviews, February 2016 15

Page 16: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

contexts, to deliver ‘amplified results’ and/or ‘transformational impact’?

What evidence is there that the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches and the mechanisms that support them have the potential, within specified contexts, to deliver ‘amplified results’ and/or ‘transformational impact’?

What evidence is there that the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels and the mechanisms that support them have the potential, within specified contexts, to deliver ‘amplified results’ and/or ‘transformational impact’?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. How the packages of activities (LAPA and EWS) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses is linked with strengthened resilience?

ii. What are the status before for the packages of activities (LAPA and EWS) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. How the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches is linked with strengthened resilience?

ii. What are the status before for the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

d. Research findings in relation to integration of DRR and CCA.

Guidance for BRACED Project Mid-Term Reviews, February 2016 16

Page 17: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

i. How the integration of DRR and CCA that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches is linked with strengthened resilience?

ii. What are the status before for the integration of DRR and CCA that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

2.4 Who benefits most from your activities?

Who benefits most from the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

Who benefits most from the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses?

Men? Women? Older people? Younger people? Certain ethnic groups? Certain livelihood or occupational groups? Certain political alliances? Does this change during periods of crisis?

If different groups of people are represented in your activities, do they participate equally? Why /Why Not? What are the consequences - anticipated or unanticipated?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. Do men and women equally participate in the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses? How are ethnic community and women

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Focus group discussion HH survey

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Focus group discussion

Analysis of interview transcripts

Document review

Analysis of HH survey

Analysis of interview transcripts

Document review

Guidance for BRACED Project Mid-Term Reviews, February 2016 17

Page 18: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

Who benefits most from the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

Who benefits most from the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels?

participation assured by the project? If yes how social ethnic group and women benifited?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. Do men and women equally participate in the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses? If yes how social ethnic group and women benifited?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. Do men and women equally participate in the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches? If yes how social ethnic group and women benifited?

d. Research findings in relation to integration of DRR and CCA

i. Do men and women equally participate in the integration of DRR and CCA that lead to better understanding of what works in

Guidance for BRACED Project Mid-Term Reviews, February 2016 18

Page 19: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

building resilience to climate related shocks and stresses and integration into DRR approaches? If yes how social ethnic group and women benifited

EQ3 –What have you had to change or adapt in terms of your interventionpackage design and why?

What have you had to change or adapt in terms of the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses design and why?

What have you had to change or adapt in terms of the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private

3.1 What unanticipated, positive or negative, enablers or constraints have you encountered?

What unanticipated, positive or negative, enablers or constraints have you encountered in the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

What unanticipated, positive or negative, enablers or constraints have you encountered in the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses?

What unanticipated, positive or negative, enablers or constraints have you encountered in the packages of activities (WEAI) that lead to better understanding of what works in building resilience to

What has helped the package of activities to work and/or bring about change; What has got in the way of the activities working and/or bringing about change

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. How increased income has helped to bring changes in the household livelihood?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. How LAPA and EWS has helped individual households or community to bring changes?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. How women participation has helped to bring changes?

d. Research findings in relation to integration of DRR and CCA

i. How integration of DRR and CCA has helped to bring

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Focus group discussion

Key informant interviews with programme staff/ project beneficiaries / government officials

Annual Reporting Data Focus group discussion

Analysis of interview transcripts

Document review

Analysis of interview transcripts

Document review

Guidance for BRACED Project Mid-Term Reviews, February 2016 19

Page 20: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

sector to respond and provide support to climate related shocks and stresses design and why?

What have you had to change or adapt in terms of the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches design and why?

What have you had to change or adapt in terms of the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels design and why?

climate related shocks and stresses and integration into DRR approaches?

What unanticipated, positive or negative, enablers or constraints have you encountered in the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels?

changes?

3.2 What key risks do you perceive in achieving your intended outcomes and impacts?

What key risks do you perceive for the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses in achieving your intended outcomes and impacts?

What key risks do you perceive for the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses in achieving your intended outcomes and impacts?

i. Does anything need to change to make the implementation operate better? To better deliver its outcomes? Are you clear on whose resilience you are building and whose you are not? Will that require any modification to the outcome?

a. Increased average annual income per household through climate resilient diversified livelihoods.

i. Does anything need to change in for the packages of activities (Collection Center, Essential Oil Distillation Unit, Multiple use water systems, Agriculture and Essential Oil cultivation trainings) that lead to increase capacity of poor people to reduce their vulnerability to climate related shocks and stresses?

b. Number of VDCs / Municipalities implementing local resilience strategies (LAPA) incorporating climate change adaptation and disaster risk reduction

i. Does anything need to change in

FGD project staff reflections

Annual Reporting Data

FGD project staff reflections

Annual Reporting Data

Analysis of interview transcripts

Document review

Analysis of interview transcripts

Document review

Guidance for BRACED Project Mid-Term Reviews, February 2016 20

Page 21: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes

What key risks do you perceive for the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches in achieving your intended outcomes and impacts?

What key risks do you perceive for the packages of activities (MU) that lead to improved policies integrating DRR and CCA in place at local and national levels in achieving your intended outcomes and impacts?

for the packages of activities (LAPA and EWS) that lead to increase capacity of local government, civil society and private sector to respond and provide support to climate related shocks and stresses?

c. Change in average Women’s Empowerment in Agriculture Index (WEAI) score measured through a five domain composite score.

i. Does anything need to change in for the packages of activities (WEAI) that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

d. Research findings in relation to integration of DRR and CCA

i. Does anything need to change in for the packages of activities in relation to integration of DRR and CCA that lead to better understanding of what works in building resilience to climate related shocks and stresses and integration into DRR approaches?

Guidance for BRACED Project Mid-Term Reviews, February 2016 21

Page 22: file · Web viewBRACED MTR Evaluation Matrix: Introduction and Explanatory Notes. February 2016. Introduction and Explanatory Notes