EXTERNAL FINAL EVALUATION - ONGAWA...MAMA-Programme final evaluation Final Report (July 2013) 3...

EXTERNAL FINAL EVALUATION

“MAMA-Mradi wa Maji na Maendeleo”, ACCESS TO SUSTAINABLE WATER AND SANITATION

SERVICES, HYGIENE PROMOTION AND CAPACITY BUILDING IN WATER AND SANITATION AND WATER

RESOURCES MANAGEMENT PROGRAMME in Same District, Tanzania.

(Project number 9.ACP.RPR.101)

FINAL REPORT

July 2013

This evaluation is supported and guided by the European Commission and presented by ECODE. The report does not necessarily reflect the views and opinions of the European Commission.

MAMA-Programme final evaluation

Final Report (July 2013) 2

TABLE OF CONTENTS

ACRONYMS ................................................................................................................ 3 FOREWORD ............................................................................................................... 4 1. EXECUTIVE SUMMARY ........................................................................................... 5 2. INTRODUCTION ............................................................................................... 10

2.1. Purpose and scope of the evaluation .................................................................... 10 2.2. Description of the activities of the evaluation ........................................................ 10 2.3. Organization / Structure of the report .................................................................. 12 2.4. Limitations ......................................................................................................... 12 2.5. WASH situational analysis in rural Tanzania .......................................................... 13 2.6. Background of the intervention ............................................................................ 14 2.7. Evaluation approach and methodology ................................................................. 16 2.8. Tools used for information gathering ................................................................... 17

3. ANALYSIS OF EVALUATION CRITERIA ............................................................. 19 3.1. RELEVANCE / COHERENCE and EC VALUE ADDED ................................................ 19 3.2. EFFECTIVENESS ................................................................................................. 23 3.3. EFFICIENCY ....................................................................................................... 36 3.4. IMPACT .............................................................................................................. 38 3.5. SUSTAINABILITY ................................................................................................ 49

4. VISIBILITY ...................................................................................................... 63 5. OVERALL ASSESSMENT ................................................................................... 63 6. CONCLUSIONS AND RECOMMENDATIONS ....................................................... 64

6.1. CONCLUSIONS ................................................................................................... 64 6.1.1. Strengths ........................................................................................................... 64 6.1.2. Weaknesses ....................................................................................................... 65 6.1.3. Opportunities ..................................................................................................... 66 6.1.4. Threats .............................................................................................................. 66 6.2. RECOMMENDATIONS .......................................................................................... 67 6.2.1. Common to all stakeholders involved ................................................................... 67 6.2.2. Specific to ONGAWA ........................................................................................... 68 6.2.3. Specific to SDC ................................................................................................... 69 6.2.4. Specific to village government and beneficiaries ................................................... 70 6.2.5. Specific to MOW and MOHSW .............................................................................. 71 6.2.6. Specific to EC ..................................................................................................... 71

7. ANNEXES.......................................................................................................... 72



ACRONYMS

CDO Community Development Officer CBO Community Based Organisation

CLTS Community Lead Total Sanitation COWSO Community Owned Water Supply Organization CTC Child to Child

DP Distribution point DED District Executive Director DEO District Education Officer

DHO District Health Officer DMO District Medical Officer

DWD District Water Department DWE District Water Engineer DWST District Water and Sanitation Team

DWUS District Water and Sanitation Unit Support EC European Commission ECODE Estudio de Cooperación al Desarrollo (in Spanish)

FY Financial Year GoT Government of Tanzania HRBA Human Rights Based Approach

ISF Ingeniería sin Fronteras – Uhandisi Usio na Mipaka (in Spanish) IWRM Integrated Water Resource Management JMP Joint Monitoring Programme

JWSR Joint Water Sector Review LGA Local Government Authority MAMA Mradi wa Maji na Maendeleo (in Kiswahili)

MDG Millennium Development Goals MOHSW Ministry of Health and Social Welfare

MoW Ministry of Water MKUKUTA National Strategy for Growth and Reduction of Poverty (in Kiswahili) NAWAPO National Water Policy

O&M Operation & Maintenance PBWO Pangani Basin Water Office PC Private Connection

PHAST Participatory Hygiene and Sanitation Transformation RWSSP Rural Water Supply and Sanitation Programme SDC Same District Council

SWASH School WASH TAWASANET Tanzania Water and Sanitation Network TIDI Tanzania Integrated Development Initiative

ToR Terms of Reference TWESA Tanzania Water and Environmental Sanitation VEO Village Executive Officer

VG Village Government WASH Water, Sanitation and Hygiene

WEO Ward Executive Officer WPM Water Point Mapping WSDP Water Sector Development Programme

WUG Water User Group



FOREWORD The present external final evaluation work was commissioned by ONGAWA to ECODE, in

order to meet with the donor’s (EC) requirements within the implementation of the (EC) requirements within the implementation of the “MAMA-Mradi wa Maji na Maendeleo, access to sustainable water and sanitation services, hygiene promotion and capacity building in water and sanitation and water resources management programme” in Same District, Tanzania. The work has been carried out between April-June 2013 by ECODE, an independent consultancy firm

without any previous involvement in the implementation of the evaluated intervention. The evaluation team was formed by Pedro Surja (evaluation methodology expert), Valentine M. Marandu (field evaluator / health and community development expert) and Nerea Lopetegui

(field evaluator / water and sanitation expert). Field evaluation team wants to thanks the ONGAWA Same team for all the logistic and

organizational support, in special to Silvester Beda and Emmanuel Simon for accompanying and supporting during the visit to the field. Finally thanks to PBWO, the DWD and all the departments of the SDC, and all authorities and community members from Chajo

village in Mwembe Ward, Kalemawe and Makokane in Kalemawe ward, Kadando and Maore villages in Maore Ward and Kihurio village in Kihurio ward involved in the evaluation activities, for their availability, participation and support during the interviews,

meetings and workshops and for sharing their impressions and suggestions for improvement with the evaluation team.

The views expressed in this report do not necessarily reflect the views of ONGAWA. Intellectual property rights: All documentation related to the evaluation remains the sole and

exclusive property of ONGAWA – EC. Photographs1 and graphic materials have been signed over by the evaluation team. Total or partial copy of this document is allowed by any media,

mentioning always the source and intellectual property holder. For any communication referred to this document please contact to:

ONGAWA Cristina Vela (Desk Tanzania) Email: [email protected]

1 Cover photo: Beneficiary fetching water at DP in Kalemawe village during final evaluation.

mailto:[email protected]



1. EXECUTIVE SUMMARY

1.1. Introduction

The present external final evaluation work was commissioned by ONGAWA to ECODE (www.ecode.es), in order to meet with the donor’s (EC) requirements within the

implementation of the (EC) requirements within the implementation of the “MAMA-Mradi wa Maji na Maendeleo, access to sustainable water and sanitation services, hygiene promotion and capacity building in water and sanitation and water resources management programme” in

Same District, Tanzania (Project number 9.ACP.RPR.101). The work has been carried out between April-July 2013 by ECODE, an independent consultancy firm without any previous involvement in the implementation of the evaluated intervention. ECODE has also conducted

the mid-term evaluation of MAMA in 2011. The evaluation team was formed by Pedro Surja, Valentine M. Marandu and Nerea Lopetegui.

The programme aims to the establishment of safe, affordable and sustainable W&S services in

15 prioritized villages (grouped in 3 zones) of Same District and improved governance in W&S and management of water resources at local, district and basin level with a HRBA. The overall objective is a gender balanced poverty reduction in Same District through the access to water

and sanitation services. The intervention involves right holders (beneficiary communities) and duty bearers (Local, District authorities, PBWO) since the inception of the intervention. The total number of direct beneficiaries is 40.079 inhabitants.

The programme started in January 2008 and after a request of extension it has had a real implementation period of 65 months, until June 2013. The programme budget funded by EC is 2,241,342€ (65%). The counterpart is Same District Council with ONGAWA Tanzania as

implementing NGO with the local partner, TWESA, for zone III.

According to the ToR of the evaluation, the final evaluation provides the programme

management in ONGAWA, the delegation of the European Commission and the wider public with sufficient information to: a) Relevance and fulfilment of general/specific objectives and expected results (specially reach of gender objectives); b) Achievement of impacts and

verification of sustainability of the programme’s benefits (especially in zones I and II where there has been a stabilization period); c) Validity of conclusions and recommendations of intermediate evaluation; d) To contribute to the design of similar future interventions, including

providing input for setting ONGAWA intervention methodology in WASH programmes.

1.2. Methodology and limitations By using common evaluation methodologies, a project/program evaluation has been considered (focused in detail in the proposed actions); in terms of time, it has been a final evaluation,

considering the geographical zones where activities have been implemented; in terms of usefulness the evaluation has been considered as a learning process for future similar

interventions or replicability of the evaluated project by any of the stakeholders involved; in terms of the evaluation agent it has been an external evaluation but with a high participatory approach, the initial ToR related to the evaluation criteria has been designed by ONGAWA and

then completed with the recommendations from ECODE. The methodological aspects and evaluation procedures have been the entire responsibility of the external experts from ECODE; In terms of the subject of the evaluation, the formulation (coherence of the foreseen actions

and design), the products (results and effects) and the processes (starting up, monitoring and effort) have been considered. The scope of the evaluation has not been merely to accomplish the criteria defined during the desk phase; additional effort has been put on explaining the

reasons of the deviations and the difficulties and/or synergies found during its achievement.

http://www.ecode.es/



As per ToR prepared by ONGAWA, the evaluation has assessed the questions referred to the

OECD-DAC evaluation criteria following the EU evaluation guidelines: Relevance, Effectiveness, Efficiency, Sustainability and Impact. In addition Coherence and EC value-added has been

integrated under the relevance criteria and visibility has been assessed as independent criteria.

The current evaluation report has followed the structure and format recommended by the DAC and EC. The evaluation has intended to be as systematic and objective as possible. Therefore,

the different criteria have been always assessed based on reliable data. During the desk phase, an evaluation matrix was designed by ECODE and shared with ONGAWA which included all relevant questions, indicators and sources for the evaluation (see evaluation matrix in Annexes). Following DAC’s principles on reliability and usefulness of evaluations, specific mechanisms for sharing the conclusions and recommendations with the stakeholders involved in the management of the intervention were put in place by ECODE. More specifically, feed-back

sessions with ONGAWA and SDC were hold in the field and also in Spain before proceeding to produce the final version. Comments/changes suggested by ONGAWA and SDC in those cases freely decided by the evaluation team, have been incorporated into the document.

The methodological approach proposed for the evaluation combined, in a flexible manner, tools providing quantitative data (surveys, narrative and financial reports, statistics and beneficiaries, created/reactivated structures’ registration data, etc.) with different qualitative tools used

during the evaluation. Mainly: Documentary analyses, interviews, meetings, workshops, focus discussion groups, questionnaires, and direct observation.

No serious limitations were found during the evaluation that could have invalidated the conclusions reached on it. On the contrary, ONGAWA collaboratively and transparently made available all relevant documentation and openly shared their views about the strengths and

weaknesses of the evaluated programme. Some minor difficulties or limitations were found during the evaluation along with their consequences for the study such as lack of proper gender baseline for measuring the real impact on gender, the no presence of TWESA staff in Same

anymore and the bias in fulfilling the questionnaires by the beneficiary families with support of PHAST promoters. In a very low scale some medium-low relevant stakeholders were not available during the evaluation.

1.3. Conclusions and lessons learned

The main conclusions follow, while a detailed set of conclusions and lesson learned can be found in section 6.1. of the evaluation report.

A) STRENGTHS

Considering a stabilization period for each zone after ending the implementation in order to

consolidate results and put in place further guarantees towards sustainability. This has been possible during MAMA implementation period for zones I&II as MAMA was still under implementation in other zones. For zone III, additional funding has been secured from Spanish

donors for one year stabilization phase taking the advantage that ONGAWA will continue working in the same district after completion of MAMA.

Strengthening local capacities at government and non-governmental level: SDC, PBWO and

TWESA and TIDI (the new NGO launched by ONGAWA national staff) respectively, contributing to sustainable development plans. Having the pilot experience in the whole country of setting-up the DWUS by following national policies but using more operative resources.

ONGAWA experience with committed and skilled staff, enough knowledge of the institutional and local context, incorporating volunteers for specific tasks, and using integrated and innovative approaches for WASH interventions like the basin-approach for geographical

concentration or the HRBA and its flexibility and capacity to adapt itself to the different intervention contexts give added-value to its interventions in Tanzania.



Visibility and role of ONGAWA and EC is undoubtedly being strengthened in Same (and

Tanzania) as a consequence of the project.

B) WEAKNESSES

Lack of gender impact indicators built-in the design making it difficult to measure relevant impacts in women’s life (though some indirect indicators have been considered).

The terms of the partnership agreement with the local implementing partner has not agreed supervision activities from side of ONGAWA and not sharing sufficiently the roles and responsibilities from TWESA and ONGAWA have affected the smooth implementation of zone III

activities.

Difficult to assure quality of supplied materials for construction works due to high competition among supplier at national level participating in tendering processes.

C) OPPORTUNITIES

Set-up of DWUS as a realistic team at District level for support WASH related structures in the

communities. MOW has a great opportunity to replicate and scale it countrywide. Participation of ONGAWA in national platforms like TAWASANET and its advocacy and lobby role to the MOW might facilitate it.

Created COWSOs are motivated and ready to share experiences and learn from each other to make water schemes sustainable with acceptance of the community. There are already some

COWSOs having benefits and planning to expand their water schemes.

There is room for ONGAWA to deepen into the strategy of partnering local NGOs as TWESA, concentrating efforts on lobbying and capacity building with such local partners to promote local

sustainable development and strengthening of social fabric in Tanzania. D) THREATS

Legal procedures for COWSOs activity still not easy to be accomplish due to bureaucracy and logistics limitations: by-laws approval, Water Use Permit; some of COWSO functions also not yet set in place, e.g. water quality monitoring at DPs. These aspects might be relevant for a

smooth O&M of the water schemes and for avoiding conflicts between and within communities.

Community is not used to “Pay as you fetch” principle for water consumption payment at Private Connections jeopardizing the financial sustainability of the water schemes; there is no

experience of installing and using water meters for the private connections.

Budget instability at District and National level threats and not much influence of the District at National budget decisions might influence negatively the future of a relevant structure such the

DWUS at District level.

Difficulties to strictly assess the cause-effect relation and net impact of the project in reduction

of morbidity and mortality for water-borne diseases as the indicators used are not always coinciding with official health indicators from the District.

Definitive sanitation policy not yet in place and strategy for school sanitation, SWASH, very

new. Sanitation services for pupils still very poor.

Periodical drought and hunger periods (as has happened at the end of 2005 and beginning of 2006 and repeated in 2009) and/or heavy rains (as in the beginning of 2012) in Same District

can be a serious obstacle for sustainability of the systems, both financially and socially.



1.4. Recommendations The main recommendations follow, while a detailed set of recommendations can be found in section 6.2. of the evaluation report.

A) COMMON TO ALL STAKEHOLDERS INVOLVED

To make sure at the early stages that the community understands the main objective of the

project, especially regarding water supply and private connections; in multi-village water supply schemes ONGAWA and DWUS to make sure that agreement for installation of water meters is done between PC owners and COWSOs.

DWUS to coordinate jointly with DWD, PBWO and COWSO getting the Water Use Permit, water quality monitoring plans at source and DPs, installation of new private connections assuring the water schemes’ capacity.

Joint efforts to promote allowances for making sure that PHAST facilitators continue with their activities at the hamlets (possible options: community contribution in kind, COWSO or sanitation CBOs contribution in cash or VG through village action-plans budget, DHO through sanitation

funds). DWUS to ensure proper identification (T-Shirts, working tools) of PHAST promoters and Sanitation CBOs in the community with ONGAWA support.

Primary schools: ONGAWA and DWUS to advocate and remind their responsibilities on providing water-treatment tools (parents) and other hygiene & sanitation tools (VG and DEO capitation grants); promote low-cost technologies proposed in SWASH guidelines (2010) specially for

hygiene practices.

O&M of water systems: leave DWUS and COWSOs alone to run them during stabilization period; ONGAWA to act only as advisor planning supportive supervisions and giving technical and

managerial support just when needed; DWUS and COWSOs to emphasize the periodical review of the water tariff in each water supply system with community participation.

To keep on increasing efforts towards jointly design of disaster risk mitigation and preparedness

plans (community, local authorities and ONGAWA) for minimizing the negative impacts of periodical droughts and consequent hunger epidemics and heavy rains and consequent floods damaging the infrastructures in the future.

B) SPECIFIC TO ONGAWA

Related to program design: try to use official indicators in collaboration with District

departments and strengthen specific gender indicators in the logframe for measuring achievements and impacts easily; avoid too many results in the logframe and group them

accordingly to assure internal coherence and to make the monitoring easier; at least 1 year of stabilization phase for each zone to be included in WASH programme implementation timeframes. Consider the specific needs of vulnerable groups (PLWHIV, disabled, elders,

children at school) promoting representatives of vulnerable groups as COWSO members and PHAST members during their establishment to assure their needs and taking advantage of existing CBOs to approach vulnerable groups (e.g. KIWAKUKI for PLWHIV).

If technically feasible, multi-village water system seems to be the approach to be prioritized in the future; although being more complex to organize at the beginning, it is more coherent (water-basin approach) and much easier to implement since it avoids conflicts of interests

between neighbour villages.

Related to the implementation: Use “easy language” adapted to community understanding level when discussing technical issues, specifically in regard to construction and policy issues.

Implementation timetable to be adapted to seasonal community activities to avoid affecting their participation.

Related to intervention strategy in Tanzania: Deepen into partnership strategies with local NGOs

by sharing experiences, improving from previous lessons learnt and strengthening local capacities.



Keep on framing interventions into national WASH policies and advocating at Ministry level for

scale-up and/or replication based on lessons learnt; Keep-on sharing experiences with other NGOs working in the geographical area and/or sector to learn from different successful

experiences and share for advocacy.

Although water is not a direct priority for EC within the next 5 year period, still explore funding opportunities for WASH interventions linked to agriculture, energy and/or environment/climate

change instruments.

Try to minimize the negative impact of periodical draughts and other climate external factors, by strengthening collaboration, coordination and previous contacts with other relief-based

organisations in the area, developing a systematic protocol for this kind of situations and promoting disaster risk preparedness and mitigation measures (at local authorities and community levels).

C) SPECIFIC TO SDC

Mainly focus on the activity of DWUS supporting the WASH structures in management, O&M

and governance at village/ward level; and also keeping the necessary institutional links with higher authorities for this objective, e.g. PBWO and regional and national authorities.

SDC authorities to promote and support DWUS at district level and at national level.

D) SPECIFIC TO VILLAGE GOVERNMENT AND BENEFICIARIES

VG to support the created WASH structures and community agents (COWSO, WUGs, PHAST, CTC, and Sanitation CBOs) but respecting their roles and responsibilities. Motivating the communities to follow with the use of WASH infrastructures (source protection, water payment,

hygiene habits) and cooperating or mediating if conflict arises.

COWSO and Village Authorities to keep-on promoting the participation of women in decision-making platforms. Although this recommendation is directly linked with the cultural-traditional

habits it might have an effect in the daily life of men and women in the village and their roles, promoting the transition to the recognition of women in the society with completely respect.

COWSO and VG to monitor the need of safe water points for cattle and plan them with

assistance from DWE.

E) SPECIFIC TO MOW AND MOHSW

To unify ratio pupil/latrines for boys and girls, which is not clear in the different policies, thus it needs to be reviewed specially for rural context primary schools in order to have realistic standards; maximize DWUS experience in Same District and try to promote it countrywide.

F) SPECIFIC TO EC

Strengthen reporting on partial/process indicators for this kind of projects, always with a result-oriented approach.

Explore granting the NGOs with more flexibility when subcontracting local enterprises, in order

to allow them promoting small local business and development not always big companies well established and cornering most of the market.

Keep on allowing NGOs to adopt a flexible approach for the implementation of this kind of

projects, in order to get adapted to unforeseen circumstances (as droughts, global economic crisis, etc.) which may cause some amendments on the activities, their schedule and/or financial capacity to implement additional components due to scarcity of financial resources.

Consider funding post-project small stabilization phases (1-2 years) for this kind of interventions.



2. INTRODUCTION

2.1. Purpose and scope of the evaluation

The project evaluated was the “MAMA-Mradi wa Maji na Maendeleo, Same; Access to Sustainable Water and Sanitation Services, Hygiene Promotion and Capacity Building in Water and Sanitation and Water Resources Management Programme, Same District, Tanzania”

(Project number 9.ACP.RPR.101) funded mainly by the EC (and other Spanish donors). The Programme (hereinafter MAMA) started in January 2008 and after a request of extension it has had a period of implementation of 65 months, until June 2013.

According to the ToR of the evaluation, the final evaluation, which has been foreseen in the Contract’s Special Conditions, should provide the programme management in ONGAWA, the delegation of the European Commission and the wider public with sufficient information to:

a. Relevance and fulfilment of general/specific objectives and expected results (specially reach of gender objectives);

b. Achievement of impacts and verification of sustainability of the programme’s benefits

(especially in zones I and II where there has been a stabilization period);

c. Validity of conclusions and recommendations of intermediate evaluation;

d. To contribute to the design of similar future interventions, including providing input for

setting ONGAWA intervention methodology in WASH programmes.

2.2. Description of the activities of the evaluation

Following phases have taken place with their regarding products / deliverables to ONGAWA:

PHASE DATE AND PLACE TASKS

Kick-off meeting 23 April

Skype

Discuss and agree on the approach, scope,

methodology, objectives and agenda

First overview on the program, strengths and weaknesses by ONGAWA and discussion

Provision of relevant documents

Analysis of docs 24 April – 24 May

Spain

Read and synthesise docs delivered by

ONGAWA

Clarify any eventual doubts with ONGAWA

Search additional info/docs by ECODE

Preparation of

agenda

24 April – 24 May

Spain

Extract all potentially relevant

stakeholders/informants by ECODE

Propose activities with each one by ECODE

Feed-back from ONGAWA about the proposed

stakeholders/activities and consolidation of



tentative field agenda

Questionnaire for program

managers

24 April – 10 May

Spain-Tanzania

Send questionnaire by ECODE

Answer from program managers ONGAWA

Preparation of

evaluation matrix

10 May - 24 May

Spain

Draft evaluation matrix by ECODE

Feed-back from ONGAWA about the proposed matrix and consolidation of final evaluation matrix

Desk-phase

report

27 – 28 May

Tanzania

ECODE desk-phase report

Field visit 27 May – 6 June

Same

- ECODE Evaluation feed-back report

First draft report 10 – 27 June

Spain

- Evaluation final report draft

Comments from ONGAWA and

partners

28 June – 12 July

Spain

- Meeting with ONGAWA in Madrid

- New versions of draft reports

Seminar / Discussion

meeting with ONGAWA

24 July

Spain

- Draft PPT Summary

Final report 5 August

Spain

- Final report and Final PPT summary

Work process followed during the evaluation:

Preparation,

scope and

methodology

of the work

Discussion

with

involved

stakeholders

Discussion

with

involved

stakeholders

Desk phase

Information

gathering on the

field

Information

analysis

Seminar

(discussion

with

ONGAWA)

and

Final report

Evaluation

matrix,

Desk report

Draft

feedback

report



2.3. Organization / Structure of the report

The report is divided into four parts. First section starts with an introduction which includes

short background, scope, methodology and limitations of the evaluation as well as a brief description of the programme to be evaluated and related stakeholders.

The main section analyses in an exhaustive and individualize way the evaluation criteria

proposed jointly by ONGAWA and ECODE, followed by and overall assessment and main conclusions. Based on this, in the last section recommendations in common and specific to each stakeholder are made including lessons learned for an engagement at district and

community level for future intervention and for the stabilization phase of Zone III.

Finally, the report is completed with several annexes, which facilitate the deep understanding of the evaluation or some of its activities.

2.4. Limitations

No serious limitations were found during the evaluation that could have invalidated the conclusions reached on it. On the contrary, it must be remarked the availability,

collaboration and transparency shown by ONGAWA (Spain and Tanzania) during all the evaluation process, sharing all the available documentation and their views about strengths and weaknesses of the evaluated program.

Some minor difficulties or limitations found during the evaluation, along with their consequences in the study, were the following:

Limitations

Consequences Relevance

No direct access to Zone III implementation full team from

TWESA as they are not based in Same or in the region anymore.

Solved partially with telephone/email interview with former program manager

and face to face interview with one of the CDOs and cross-checking with

existing partnership agreements, minutes and opinions from other stakeholders.

Medium

Questionnaire for beneficiary families guided by PHAST promoters

The presence of a person directly linked to the project may have influenced the

answers of the beneficiaries. Whenever possible triangulation combining different data sources has been used

for confirming this information.

Medium

Absence of a proper gender baseline with real data at the beginning of the project

Project’s gender impact could not be strictly measured. Alternative indicators have been considered for monitoring by

ONGAWA

Medium

No village from other areas out of the interventions zone in Same District were visited (specially related

to Result 8 of the logframe), trying to focus the resources and efforts of the evaluation in areas where most

of activities had taken place so far.

Some minor activities from Result 8 outside of the intervention villages could not be checked directly on the

field, but DWE provided some relevant information on this.

Medium

Some community groups were missing during the evaluation workshops, specially vulnerable

Solved with massive questionnaire for families which includes participation of all community group members.

Low



groups

Impossibility of meeting with some key informants (local & international

NGOs), since they were not available during the field visit

Difficulties to check the synergies with similar interventions of other NGOs in

Same District and in the Water and Sanitation sector

Low

2.5. WASH situational analysis in rural Tanzania

The United Republic of Tanzania located in East Africa has a HDI 152 out of 186 countries

worldwide with a value of 0.476 against the average of 0.475 in Sub-Saharan Africa and 0.694 in the world. According to last UNICEF/WHO JMP review (2012) the situation on access to water supply and sanitation in rural areas in Tanzania is as follow:

Table 1: Access to water and sanitation in Rural Tanzania

Shar

ed

Un

imp

rove

d

Op

en

de

feca

tio

n

Tota

l

imp

rove

d

Pip

ed

on

pre

mis

es

Oth

er

imp

rove

d

Un

imp

rove

d

Surf

ace

wat

er

1990 25.479 81 6 3 81 10 46 0 46 30 24

2000 34.038 78 7 4 76 13 45 2 43 32 23

2011 46.218 73 7 4 73 16 44 4 40 33 23

USE OF DRINKING-WATER SOURCES

IN RURAL AREAS

(% of population)

Unimproved

USE OF SANITATION

FACILITIES

IN RURAL AREAS

(% de población)

TANZANIA

Imp

rove

d

Unimproved

Population

(thousands)Year

Improved

Pe

rce

nta

ge

rura

l po

pu

lati

on

Source: UNICEF/WHO JMP Report 2013

The Government of Tanzania, GoT, identified and started with the water sector reforms as per the necessity to link sector plans with country’s socioeconomic policies such as the Tanzania’s

Development Vision 2025, which targets at increasing access to clean and safe water in rural areas to 90% by 2025; MDGs (by 2015) and MKUKUTA II (2010-2015) which targets at

increasing access to clean and safe water in rural from 57.8% in 2010 to 65% in 2015. Sector reforms is being implemented through formulation of the National Water Policy (NAWAPO 2002), formulation of the National Water Sector Development Strategy (NWSDS 2006-2015),

preparation of the Water Sector Development Programme (WSDP 2006-2025); and enactment of two water legislation; the Water Resources Management Act Number 11 of 2009 and the Water Supply and Sanitation Act Number 12 of 2009.

GoT and Development Partners agreed on a Sector Wide Approach and WSDP was designed as a comprehensive programme covering water resources management, rural water supply, urban

water supply, sanitation/sewerage as well as institutional strengthening, sector capacity development and monitoring at all levels. The WSDP also supports a household sanitation initiative as well as improvements in school sanitation infrastructure, under the overall direction

of Ministry of Health Social Welfare (MOHSW) in collaboration with Ministry of Education and Vocational Training (MoEVT). The Ministry of Water as the overall manager and coordinator of the WSDP. The financing of the WSDP is shared between the GoT (19.31%), Development

Partners (80.7%); in September 2012 the performance was of 61%. Since the commencement of the WSDP in 2007/2008 a total of 7 successfully Joint Water

Sector Review (JWSR) meetings were held. The meetings involved water Sector stakeholders, including representation of Government at all levels, Development Partners (DPs), Civil Society



Organizations (CSOs) and private sector. The meetings discussed the follow up on the

implementation of agreed actions and undertakings, discussing physical progress for programme implementation and receive various reports from other stakeholders who work

together with the Ministry.

After MKUKUTA (2005-2010) established targets on water and sanitation has not been achieved

and new targets has been set in MKUKUTA II (2010-2015). In line with this, according to the 2012 Water Sector Status Report initial targets for Rural Water Supply and Sanitation has been reviewed and more realistic targets have been set as per 2014 from baseline date of July 2010:

Table 2: Reviewed targets for 2014

Measurement Theme Indicator/ Target

Water Supply access 60.5% of people within 400m of an improved water point

Water Supply out put 14,790 water points constructed or rehabilitated

Management 100% of water schemes in programme village have actively functioning COWSOs

access of improved sanitation facilities

and hygiene by household

1,300,000 households with improved latrines

being retrofitted, constructed under the program as a result of program promotion

access of improved sanitation facilities and hygiene by school pupils

700 schools with improved latrines being retrofitted, constructed under the program as

a result of program promotion

Source: Water Sector Status Report 2012

2.6. Background of the intervention2

Intervention Area: Same District, Kilimanjaro Region. The program covers 3 geographical

areas: Zone I: Chajo, Bangalala (Mwembe ward) and Mkonga (Kisiwani ward) villages. Zone II: Majengo and part of Mgandu (Bendera ward), Makokane and Kalemawe

(Kalemawe ward) villages. Zone III: Maore, Mpirani, Kadando and Mheza villages in Maore ward and Uzambara,

Kankokoro, Mvure, Kongei and Mbuyuni in Kihurio ward and part of Mgandu3 village in

Bendera ward.

Duration of the project: 60 months starting in January 2008 plus 5 months extension (65

months) INGO: ONGAWA

Counterpart: Same District Council

Local partner for implementation of the action in Zone III: Tanzania Water and Environmental Sanitation, TWESA NGO

Target groups: General population of Same district, with special focus on women, Local authorities (VG), Same District Council (SDC) and Pangani Basin Water Office (PBWO)

Beneficiaries: General population of 15 high prioritized villages in Same district (40.079 inhabitants). Indirect beneficiaries: Total population of Same District is around 235.000 people

2 The original logframe matrix is described in this chapter as it was formulated by ONGAWA and approved by EC.

Changes and improvements proposed by the evaluation team are not reflected. 3 For the questionnaires for families Mgandu village as a whole has been considered in zone III.



Programme budget: 3,445,333€

Financed by EC: 2,241,342€ (65%) Brief summary of the intervention:

The MAMA programme is the second intervention of ONGAWA in Same District after the so called pilot phase implemented in 3 villages since 2006. The intervention of ONGAWA is an integral programme in access to water, sanitation and hygiene services with a high participatory

approach in 15 prioritized villages in agreement with SDC. The intervention involves right holders (beneficiary communities) and duty bearers (Local, District authorities, PBWO) since the inception of the intervention. The intervention is based on a HRBA focused in the rights to

water and sanitation of rural communities in Tanzania. Memorandum of Understanding (MoU) has been signed with Same District Council, as main

counterpart of the project. Negotiation and signature of MoUs with each beneficiary community have been carried out in the first part of every phase of the action.

The programme intends to be a kind of consolidation of ONGAWA intervention strategy in Same District and to some extent countrywide; ONGAWA make use of the challenges and

achievements of its interventions in the field to make advocacy for policy-making jointly with other international and national development stakeholders.

Objectives of the intervention Overall objective: Gender balanced poverty reduction in Same District through the access to

water and sanitation services. Specific objective: Establishment of safe, affordable and sustainable W&S services in the

selected villages and improved governance in W&S and management of water resources at local, district and basin level.

Expected results: Result 1: Construction/Rehabilitation of water supplies systems attending to people’s demand and providing sufficient amount of safe water in each of the selected villages.

Result 2: The water supply services are properly managed by the community.

Result 3: The hygiene promotion program has been implemented successfully.

Result 4: Improvement of basic sanitation achieved in each of the selected villages. Result 5: The DWD has the appropriate knowledge, skills and means to assist the communities

and control the water services management in Same District. Result 6: The District Council has improved its capacities and resources for decision making,

investment planning and fund raising related to water and sanitation programmes. Result 7: The District Water Department together with PBWO has the appropriate knowledge,

skills and means to promote IWRM in the selected areas. Result 8: The civil society has improved its knowledge on water related rights and procedures

to obtain assistance towards WS&S services. Situation of the programme at the time of the final evaluation: Zones I&II: activities implemented and stabilization phase ended.

Zone III: activities ended but some water infrastructures under putting into service; stabilization phase for one year period in process of implementation.



Stakeholders involved: ONGAWA: ONGAWA, Engineering for human development (former ISF-ApD) is an international

NGO (INGO) based in Spain with a long experience in Tanzania, where it has been working in water-sanitation and hygiene promotion projects since 1996. Firstly with volunteers, nowadays is a medium size INGO registered in Tanzania as ONGAWA-Tanzania with a head office in Dar-

es-Salaam for administrative and national coordination issues, internally within the projects and externally with the MOW and other stakeholders working in the same sector. It has previously worked in Mangola Valley, Karatu District and Kigoma Rural District. At the time of the final

evaluation ONGAWA is finishing the WASH project in Same District, Kilimanjaro Region and launching an environmental project in the highlands of the same District in partnership with other local environment related NGOs. The main strategy of intervention of ONGAWA is based

in 4 pillars: service provision, community strengthening, institutional development, advocacy for policy-making.

TWESA: Tanzania Water and Environmental Sanitation, it the implementing partner for MAMA Programme Zone III. Tanzanian NGO focused on emergency and refugees context in their origins; it has developed its work to WASH sector development interventions in Tanzania, based

on NAWAPO. It has previous experience in partnership with other INGO (OXFAM, Concern...)

SDC: Same District Council is the main counterpart of the project with the District Water, Education and Health departments mainly involved. It is a decentralised governmental body depending directly from Regional and Central Government.

2.7. Evaluation approach and methodology

By using common evaluation methodologies below are the main features of the evaluation:

a. Evaluation approach: a project/program evaluation is considered (focused in detail in the proposed actions).

b. In terms of time, it has been a final evaluation, covering the period from January 2008 to June 2013).

c. In terms of usefulness the evaluation has been considered as a learning process for following WASH interventions but also for the still ongoing activities during the stabilizations phase of

Zone III; the lessons learned of the evaluation will be used as inputs for the design and implementation of them by SDC, MOW and MOHSW, VG and ONGAWA.

d. In terms of the evaluation agent it has been an external evaluation but with a very strong participative approach. The initial ToR related to the evaluation criteria were designed by ONGAWA and then completed with the recommendations from ECODE. The methodological

aspects and evaluation procedures have been the responsibility of the external experts from ECODE. The ONGAWA team (Spain and Tanzania) has been participating during the whole process. The draft of the final report has been responsibility of ECODE while inputs were

provided by ONGAWA during a feedback session on the preliminary findings. e. In terms of the subject of the evaluation, the formulation (coherence of the foreseen actions

and design), the products (results and effects) and the processes (starting up, monitoring and effort) have been considered. The scope of the evaluation has not been merely to accomplish the criteria defined during the desk phase; additional effort has been put on explaining the

reasons of the deviations and the difficulties and/or synergies found during its achievement.

As per ToR prepared by ONGAWA, the criteria proposed for the evaluation has been the

standard criteria defined by the OECD-DAC:



i. Relevance

ii. Effectiveness

iii. Efficiency

iv. Sustainability

v. Impact

As per EC requirement Visibility, coherence and EC Added-value has been also included. The

Evaluation Matrix included all relevant questions, indicators and sources for the evaluation criteria (See evaluation matrix in Annexes).

As per ToR also relevant cross-cutting issues have been considered:

Gender

Vulnerable groups

HRBA

Additionally ECODE has considered general external factors defined by OECD-DAC

affecting or not the development intervention:

a. Policy context and support

b. Institutional capacity

c. Socio-economic and cultural factors

d. Technological factors

e. Environmental factors

The current evaluation report has followed the structure and format recommended by the EC. The evaluation has intended to be as systematic and objective as possible. Therefore, the

different criteria have been assessed always based on reliable data. During the desk phase, an evaluation matrix was designed by ECODE and shared with ONGAWA. That matrix included all relevant questions, indicators and sources for the evaluation.

Following DAC’s principles on reliability and usefulness of all evaluations, specific mechanisms for sharing the conclusions and recommendations with ONGAWA were put in place by ECODE. More specifically, feed-back sessions with ONGAWA were hold on the field and also in Spain,

before proceeding to produce the final version. Suggestions from ONGAWA, in those cases freely decided by the evaluation team, have been incorporated to the document.

2.8. Tools used for information gathering

Different tools were used for information gathering during the evaluation. Whenever possible triangulation, combining different data, sources has been used. The evaluation team has tried to combine always the quantitative sources (surveys, reports, and statistics and registration

data) with qualitative perceptions during field visit. Following data collection tools have been used:

1. Records and documentary analysis. Based on the existing documents all information regarding the intervention has been collected and analyzed. A list of documents has been to be

reviewed and completed by involved stakeholders (ONGAWA, ONGAWA Tanzania). Additionally



ECODE analyzed information available in internet regarding intervention framework, public

policies, other relevant projects with best practices, etc.

2. Interviews. In-depth interviews were carried out in order to gather information of the intervention. Mainly: a. Semi –structured interviews to ONGAWA technicians and officers (HQ and in Tanzania).

b. Semi-structured interviews to TWESA staff b. Semi–structured interviews to local and District authorities and other local stakeholders involved.

c. Interviews to other key informants. A list with relevant organizations/persons was created (including local authorities, other NGO/agencies in the area / sector) to be reviewed, discussed and agreed later on with ONGAWA taking in account other relevant stakeholders that could be

suggested as well. (See list of key informants in Annexes). 3. Workshops, meetings and focus groups discussions using participatory

techniques with direct beneficiaries, communities and people who has participated directly in the program. Participatory methods were used to facilitate the collaboration according to the context of the intervention, avoiding any complex methodology.

4. Surveys. Two kind of questionnaires have been used: a) first one sent to the responsible

and team members of ONGAWA before the visit in order to get their opinion about key factors (strengths and weaknesses) regarding design, implementation and future of the project (a typical format is attached in the annexes); b) second, a brief survey for a representative sample

of families of the communities of intervention, (a total of 183 questionnaires: 48 in Zone I, 36 in Zone II and 99 in Zone III) has been done with the support of PHAST promoters to be able to collect quantitative data about the intervention as extra source to the already given data in the

formal reports. A specific assessment of the satisfaction level of the direct beneficiaries and Project users, hygiene practices, water supply service and any other relevant factor has been included in this survey.

5. Visit to project components guided by beneficiaries. During the visit to all villages, short walks were done around the villages for direct observation of the latrines, DP built and

the cleanness of the environment as well as water scheme storage tanks. In addition, informal conversations with different inhabitants were carried out during them in each village.

6. Direct observation on the structures, teams, procedures used for the intervention: observation of the way of working, planning and decision making meetings, observation of the

organizational chart (theoretical and practical), checking of available support teams and structures, conversations with all staff at the different levels. etc. At the same time, the way of living of the communities is observed mainly through visits to the site and informal

conversations.



3. ANALYSIS OF EVALUATION CRITERIA

3.1. RELEVANCE / COHERENCE and EC VALUE ADDED

Are the objectives of the intervention adequate to the context and problems identified for the men and women targeted? 1. The initiative of the intervention is based on the priorities of Same District Council in water and sanitation sector according to the study conducted by the District Water Department in

2006 “Serious Water Shortage”. 22 villages out of 83 in the whole District were identified in serious risk as only 18% of their population were benefited with the existing water schemes. The rest 82% of people were either getting service above 1km distance or not served by any

defined schemes getting water from local springs and run off sharing them with cattle tracks and other users. Main factors contributing to this situation were identified as: i) Serious drought for the past 4 years; ii) Rapid Population increases vis-a-vis inadequate population forecasting

in planning for Social services provision; iii) Deforestation for a number of reasons; iv) Poverty and ignorance; v) Lack of proper periodic maintenance.

2. The target communities (in all 3 zones) spontaneously feel and declare that water supply was a main priority for the population as well as sanitation which were affecting their health.

Since identification phase participatory activities communities faced this problem as priority. Almost the 100% of community groups participating in the joint workshops declared water being the highest priority jointly with sanitation and education.

3. Same District has been chosen by ONGAWA after a comprehensive district selection process countrywide based on its previous experience and lessons learnt in the country (previous long-

term interventions in Kigoma and Karatu Districts). ONGAWA started in Same district with a WASH pilot phase covering 3 of the 22 identified villages by the District. The MAMA programme intervention villages have been selected as a result of the Water Shortage study mentioned

before and taken into account the villages that were already having some support from other stakeholders. The Water shortage study have been updated by the District with support of ONGAWA in 2009 with the so called MAMA Coverage Study in order to avoid overlapping with

new WASH interventions started by other stakeholders between 2006 and 2009. 4. In each zone PRA has been conducted, taking into account almost all community groups.

Though all the participatory tools used in the selection process it seems like the needs of some vulnerable groups have not been deeply analysed and designed accordingly. As per the evaluation workshops’ findings, the disabled have been considered, in the way that they get

water service for free, and their families or neighbours help them to access to the service. However, no specific actions to address their needs have been analysed (e.g. accessibility to

the DPs and latrines)4; similar case happened with PLWHIV.

Is the design of the intervention logical and coherent (objectives, results, indicators)? 5. The design of the intervention is logical and coherent. Participatory approaches have been used to identify the needs and problems of the village members in all 3 zones. This approach has been optimum to design the intervention in combination with the different chronogram for

each zone, learning from previous experiences and running common activities, mainly at District

4 National Guideline for School Water, Sanitation and Hygiene (SWASH) in Tanzania, First Draft for Piloting & Consultation, GoT, 2010.



and Basin level during the complete timeframe of the programme. In each zone combination

and timing of awareness activities, management trainings and construction/rehabilitation activities seems to be balanced and allows for a proper implementation. Gender baseline has

not been conducted in detailed during the design. 6. The objective and results of the intervention are clear and properly formulated though the

high number of results and indicators accordingly make the monitoring of the programme not easy. The indicators proposed are overlapping in some cases between objective and results indicators; in addition, though gender balanced has been considered in the overall objective,

there was no specific baseline with gender data, thus only some indirect gender indicators are included. The reason is that data were not easily available at District level and ONGAWA had not enough resources to conduct it. Specific identification of women groups or CBOs in the

intervention villages or at District level might facilitate this exercise. The evaluation team proposes an alternative logframe for a more coherent design and easier M&E. (See alternative logframe matrix in Annex D). 7. The programme design is based in the ONGAWA intervention strategy, combining 4 components: a) service provision for beneficiaries, b) community strengthening, c) institutional

development and d) advocacy for policy-making. This approach is fully in line with the HRBA involving the right holders (communities) and duty bearers (local and district authorities, Basin

office and Ministry at national level) among all intervention results. The gender approach has been considered as per national policy guidelines (e.g. 50% of presence in COWSO as per NAWAPO).

8. Multi-village systems rehabilitation in zone III have some challenges in the design of the community based O&M strategy in addition to the technical and economical feasibility but also

some advantages in regard to the environmental impact. There is a need of effective communication between representatives of each community in the COWSO as per the distances and transport constraints between the villages. Illegal connections and water losses are difficult

to control and it needs a more systematically routine maintenance due to its complexity and because any breakdown may affect many villages due to larger systems and many areas without population rather than in a single village system. On the other hand, one treatment

plant and one water source reduce costs of operation and maintenance; it is possible to find more efficient staff for dealing with the O&M. It seems that all this aspects has been taken into account during the design of the intervention in zone III with the formation of only 2 COWSOs,

one per ward/multi-village system. But unfortunately political interests and conflicts between villages have result in 2 COWSOs in the Maore Ward. The problems stated by families in zones

I&II might probably not be present during the O&M of the multi-village systems in zone III, e.g. water source in other village, pipe cuts, water source damage.

9. Key relevant risks have been identified since the beginning and contingency measures applied to minimize them (e.g. construction period to avoid rainy season). The implementation in phases (each zone starting in a different stage of the project), has also allowed the

intervention to detect and be adapted to risks not able to be identified at the inception phase. During the implementation also changes have been done from the original design to avoid overlapping with other interventions in the same areas by other donors (CARE, QUICKWINS,

RWSSP from World Bank) started after the MAMA-Programme was already designed and implementation had started.

Are the comparative advantages and added value of ONGAWA and TWESA being maximized when designing and implementing the program?

(comparing with other development actors in the intervention area)

10. The project goes in line with the strategic-plan of ONGAWA, with a clear Human Rights Based approach to Water interventions in rural areas. In spite of TWESA having its background in refugee and emergency contexts, its mission has evolved in the last years towards

development interventions in the water sector, becoming very experienced with NAWAPO.



Apparently there is a coincidence on both strategies but some management and coordination

problems happened5 during the intervention that indicate their comparative not being properly combined.

11. ONGAWA has an added value in comparison to other stakeholders that have been working in the same area: the inclusion of stabilization phases for consolidating results and assure

sustainability, conducting evaluations (mid-term and final) and the commitment of its staff have been continuously mentioned during final evaluation workshops in the communities.

12. There is no overlapping of the programme with interventions from other stakeholders; ONGAWA, in continuous coordination with DWE, has avoided this situation. On the contrary, capacity building at District Level might create some synergies with interventions of other

stakeholders in the WASH sector, like World Vision and CARE, as per explanation of DWE and DWUS members.

Are the intervention design and implementation in accordance with the legal

and institutional framework in Tanzania?

13. The project design complies with the principles and priorities of the National Water, and

hygiene and sanitation related policies and regarding regulations. The decentralization in the water sector promoted by the MoW is being truly considered in the design of the intervention.

The main development and poverty reduction strategy documents of GoT. Vision 2025 and

MKUKUTA, have among their main targets and goals the high quality livelihood ensuring the universal access to safe water for achieving it and the quality of life and social well being through Increasing access to affordable clean and safe water; sanitation and hygiene

(MKUKUTA Cluster II, Goal 4). According to the last MKUKUTA monitoring report 2011/12 available data the currents trends are as follow:

Table 3: MKUKUTA II targets and trends

CLUSTER II: QUALITY OF LIFE AND SOCIAL WELL BEING

Goal 1: Ensuring equitable access to quality education at all levels for males and

females, and universal literacy for adults, both men and women

2009 2010 2011 2012 Target 2015

% of schools having adequate sanitation facilities as per policy Higher

Males 1:64 1:25

Females 1:51 1:20

Goal 4: Increasing access to affordable clean and safe water; sanitation and

hygiene

2009 2010 2011 2012 Target 2015

Proportion of population with access to piped or protected water as their main drinking water source:

Rural 58.7% 58.7% 47.9% 65%

% of households with basic sanitation facilities

93% 99%

Source: Mkukuta Annual Implementation Report 2011/12

The NAWAPO (2002) includes the principles of decentralisation and subsidiarity of water supply management whereby this should be devolved to the lowest appropriate level. The Water

Sector Development Program (WSDP) is set to achieve in the mid-term the MDG and MKUKUTA II (2010-2015) goals and in the long-term the Vision 2025 targets.

5 More detail on this matter in chapter 3.3. of this report (Efficiency).



The sanitation component of the WSDP is leaded by the MOHSW. There is a policy draft in

Sanitation and Hygiene and different pilot approaches are being introduce since the intervention started: CLTS at community level and SWASH for schools.

The intervention is following all these policies and regulations and has granted enough flexibility as to get adapted to the policy changes during the implementation.

14. ONGAWA Engineering for human development, former Ingeniería sin Fronteras (ISF)- Uhandisi usio na Mipaka, is officially registered under the Societies Act at the Ministry of Home Affairs with Certificate of Registration Number S.A. 11503. The initial register of ISF-ApD was

done in 2002 while the change of name took place in 2012. TWESA is registered as a private low body in Tanzania since June 1997.

Has the role, coherency and added value of EC been maximized?

15. The MAMA Programme is consistent with the EC Tanzania country strategy as beneficiary of the 1st ACP-EU Water Facility Component C – Civil society and decentralised cooperation

Initiatives. Although the CSP 2008-2013 has two focal sectors: (i) Infrastructure, Communications and Transport; and (ii) Trade and Regional Integration, there is an additional non-focal component, supporting the Non-State-Actors. On the sector support, the ACP-EU

Water Facility, is strategically contributing to the achievement of the MDG in relation to Water and Sanitation (MDG7) and indirectly improving the education (MDG2), child health (MDG4) and poverty in general (MDG1). In the case of Tanzania this instrument contributes to two of the

priority goals of the MKUKUTA. 16. The intervention considers gender and HIV/AIDS as cross – cutting issues in the design, in

line with EC cross-cutting priorities. As mentioned before although they were well considered in the design, the effective implementation of them has not been as strong as it was expected.

17. The Water Facility seems to be an appropriate instrument to support this kind of intervention covering the areas of (A) Improving the water management and governance; (B) Water and Sanitation Infrastructure; (C) Civil Society initiatives. Enough flexibility has been

shown by the water facility instrument in terms of extensions, budgets and redesign of intervention areas timely informed by ONGAWA.

18. The intervention is complementing other development agencies’ WASH initiatives in Tanzania, being coordinated mainly by World Bank (RWSSP) and KfW national programmes. EC,

itself, is not active on the WASH sector in Tanzania but is member of the coordination and steering committees.



3.2. EFFECTIVENESS

Has the specific objective of the program been achieved in the communities of the intervention? 19. The table below summarizes the achievement of the indicators for the specific objective. The indicators included in this chapter are the ones proposed by ECODE in the reviewed logframe matrix in annex D. The evaluation team decided to consider this alternative logframe in the analysis of the effectiveness

criteria as in the original logframe some relevant indicators were missing/overlapping. Table 4: % of achievement of specific objective indicators

Indicator Baseline6 % of achievement7

80% of targeted population increases consumption of safe water from 10-16

to 24-30 litres /person/day (split up for drinking, cooking, washing)

Average consumption is between 3 and 5 buckets per family, it means approximately

between 10 and 16lts./person/day. Although it depends a lot on the community, the specific sub-village, the season and the

family. There are even ratios of 1 bucket/day/family in some cases.

100% in Zones I&II: according to closelines, consumption has increased around 15-20% in the moment of collection this

close information. Currently it´s estimated that the consumption has been doubled.

Zone III: pending to measure after a period with the systems running

Evaluation team’s comments8

People are using water from DPs and/or PCs so that there is a tangible change. According to the physical aspect of the population during the field visit they use it in enough quantity to assure cleanness and good health; they have improved since the mid-term evaluation and they stated it during the evaluation workshops.

According to interviewed families, more than 90% of families in all zones confirmed to be using between 2 to 6 buckets per day (i.e. 40 to 120lts/household) while only 7% in each zone use 7 or more buckets (more than 140lts/household/day). The consumption has decreased specially in 2

villages in zone I: Bangalala and Mkonga where during mid-term evaluation 100% of the interviewers declared be using more than 7 buckets while in the final evaluation only 14% of Bangalala families and 0% in Mkonga were declaring the same amount.9

6 Data from 2006 baseline or PRAs conducted in each zone at the beginning of the intervention. 7 Source: ONGAWA monitoring reports. 8 Figures included in this column in all this table are extracted from Questionnaire for families conducted during final and mid-term evaluations. 9 To note the possible bias of questionnaire for families: conducted in the same villages in zone I for both mid-term and final evaluation in a random basis, i.e. to a different number of families and

not necessarily to the same families.



Average water consumption has increased in compare to the baseline data but it is still very low if we compare it with the NAWAPO standards: household size average in Same district is 4.5 (Tanzania Census 2012) people with 2 to 6 buckets means a minimum consumption of 8.8lts/person/day to a maximum of 27.27lts/person/day.

Indicator Baseline % of achievement

75% of women avail more 2-4

hours/day of spare time as a direct consequence of having the new water and sanitation systems

According to the PRAs exercises women were

spending between 2 to more than 6 hours fetching water, although it also depends on the community, hamlet and the moment of

the year.

Not available

Evaluation team’s comments

Figure 3:

During the evaluation workshops all women stated that they

have more time to deal with other domestic activities or for going to the farm, as they save time with the new water systems. Before MAMA they were waking up early in the

morning to fetch water; even the families participating in the evaluation survey were from the same opinion:

60 % of the families declared they used between 0.5 and 2 hours before the intervention while at the final evaluation in

zones I&II 96% of families declared to use less than 15 minutes and in zone III 73% of families.


Each public water point (2 taps) supply good quality water (Tanzanian standard) to less than 400 people (600

for hand pumps) within 20 minutes for

35% (according to WPM 2006 data) 98% considering 250 people/tap as stipulated in the NAWAPO, and more distance (300m.) in the case of hand-pumps in special isolated places in some communities.



distance after project completion


Figure 4:

DPs are located according to NAWAPO standards and

beneficiaries don’t make any comment about lack of water in DPs, but they are satisfied with the service provided.

Beneficiary families declared that water quality at home is good or very good for 87% of the interviewers in zones I&II

improving since the mid-term evaluation and 65% in zone III. Water quality at DPs is not being monitored in a periodical basis.


Routine and preventive maintenance and repairs undertaken satisfactorily on

water systems and latrines by the communities at the end of the project

At the beginning of the project, in the communities where a water system already

existed more than 50% of the DPs were not functional.

100% At the end of the DPs are running, even if some reparations

were needed. COWSOs managed to solve these problems. There are a total of 29 plumbers (fundis) in zone I; 24 in zone

II and 30 in zone III fundis trained and working. There are also routine and preventive maintenance procedures

in place.


Since water schemes are functioning specially in zones I and II there have been some small breakdowns and COWSO have been able to deal with them

alone or with support of DWD.




Water sources are properly protected and in a safe environment according to Tanzanian standards by the end of the

project

0%

100% In all gravity water systems, where the source was an open one (mainly rivers), an improved intake has been constructed,

in order to avoid contamination. Other specific measures implemented have been the protection

of the catchment’s area with a fence and the restoration of the area after construction with tree planting.


There was no possibility to visit any water source during the evaluation. However, CBOs in zone III and PHAST in zones I and II have been planting trees in water source areas and they are aware of its importance.


Gender equity aspects in water &

sanitation are incorporated in all decision-making bodies, campaigns and

plans (% of women in COWSOs and other related bodies; gender issues mentioned in sanitation plans; water

tariffs adapted to vulnerable women's needs; DWD strategic plan includes gender issues; % of women aware of

water rights; exclusive latrines from women and/or women-friendly design of latrines; hygiene and sanitation roles

increasingly shared by men-women)

No available information from PRAs or

baseline.

100%

All water entities have equitable representation of women and men.

The strategic plan of DWD included as Objective of Same District to Improve Social Welfare, gender and community

empowerment. The objectives of gender are based on the following target: gender opportunities increased from 38.9% in 2010/2011 to 50% in 2016.

In zones I&II, latrines exclusive for girls used had been constructed in all beneficiary primary schools.

In zone III, increase the percentage of man who are responsible of cleanliness the latrines. Still the number is really

low but there is an improvement. This is a sample of the changes on hygiene and sanitation roles in the communities.




Figure 5:

Women have also participated in all WASH related trainings/workshops during the intervention; all COWSO have equitable representation of women and men; there is also high

representation of women at PHAST and CBOs, water tariff are adapted to disabled and PLWHIV, elders (including women);

In Chajo village one COWSO plumber is a woman, which is a rear case in such a culture where usually this kind of positions is covered by men;

District strategic plan gender approach for WASH sector is based on NAWAPO policy.

In public buildings not always exclusive latrines for women of

women-friendly design are available. As it was declared by the families in the evaluation survey

women are using the “free time” for business or productive activities rather than for family care which shows an improvement in the change of roles men-women: in the mid-

term evaluation in one village was declared they use the free time for family care and 2 villages stated using it for farming activities; at the final evaluation farm activities and not

specified business were mentioned in 3 villages and fishing in 2 while family care was not mentioned in any of the villages; on the other hand in zone III 4 villages mentioned using the time

for farming and 2 for business against 1 for family care and 1 for washing clothes.



Have the results of the program been achieved for the communities of the intervention?

20. The expected results of the program have been achieved by the time of the final evaluation. The stabilization phase of zone III is serving as for consolidating those results in zone III. The beneficiaries in all zones are satisfied with the results of the project as well as it was stated in the survey conducted

during the evaluation. Figure 6: community satisfaction about programme results

The level of satisfaction of the beneficiaries is high, more

than 80% in all three zones confirmed they are much satisfied while no one stated to be not satisfied. It is to consider the improvement in the perception of families in

zones I&II since the mid-term evaluation where less than 4% were declaring being much satisfied but 10% were satisfied to some extent only and almost 5% were not

satisfied at all. By that time some of the villages in these zones were not yet having water supply service regularly functioning and others were not yet having water; these

might be the main reasons for this non satisfaction. In zone III, though in the moment of the field visit, the

putting into service was just under implementation and works were still on-going, they are much satisfied because they now progress is being done and it is just a

matter of time to finish the pending works and get the service.

Source: Questionnaire for families conducted during final and mid-term evaluations



21. The table below summarizes the achievements on the indicators of the results of the program. The indicators included in this chapter are the ones

proposed by ECODE in the reviewed logframe matrix in annex D. The evaluation team decided to consider this new logframe in the analysis of the effectiveness criteria as in the original logframe some relevant indicators were missing/overlapping.

Table 5: Achievements on results of the program

Result 1: Water and sanitation infrastructures in the target villages enhanced


Capacity of system from source to

public water points ensure 25 l/p/day for 80 % population after 20 years (growth rate estimation) after project

completion

Not applicable 100%

Designs have been done considering these parameters (25 l/p/day and after 20 years), and the current real flow running confirm this hypothesis.

Evaluation team’s comments10

Water schemes visited during the evaluation were functioning in all zones; in zone III where systems were still in process of putting in service some

adjustments and finishing were being done (mainly in part of the distribution line and some DPs). DPs were providing enough pressure to ensure the 25l/p/day in all visited villages.


Number of new improved latrines in used in each community (split up by houses, schools, hospitals, markets)

According to the PRAs and baseline information, the coverage of latrines is quite high, around 80% although the most part of

them has been constructed in order to avoid fines and punishments. A great part of the latrines are not improved and are in very bad

situation of maintenance.

Zone I: Demonstrative Private latrines: 19 latrines; 7 latrines constructed in primary schools.

Zone II: 3 new latrines constructed and 3 rehabilitated in primary schools. No data of demonstrative latrines. According to the closeline the coverage in private households is around

95%.

Zone III: estimated 7 out of 10 communities coverage of 75%. Pending the closeline

10 Figures included in this column in all this table are extracted from Questionnaire for families conducted during final and mid-term evaluations




Figure 7:

Improved latrines in private households visited during evaluation were in good condition, but there are still many households with not-improved latrines. In any case, open

defecation seems to be disappearing except when people are farming out of the village areas.

School latrines were in use but the hygiene condition on the ones visited during evaluation, Nasuro primary school in Chajo and primary school in Makokane, was not good enough and

they were not so suitable for girls in terms of hygiene and privacy. According to questionnaire to beneficiaries, 98.8% of families in zones I&II declared W&S have improved at schools

since MAMA.

Result 2: Hygienic habits and practices among the communities promoted


At least 40% of population have improved management of children's stools, hand-

washing practices and water management at household level after project completion

According to baselines data: 60 % boiling water, 24% washing hands with

soap before eating, and 81% washing hands after defecation. (Note: percentages about hygiene practices have been estimated as an average among data from different communities, where percentages differ).

100% achieved. Zones I&II: 90% boiling water (30% of increase), 70%

washing hands with soap after before eating (46% of increase); 92% washing hands after defecation (11% of increase);

Zone III: PENDING FROM CLOSELINE DATA




Figure 8:

Households visited during the evaluation showed high level of cleanliness; proper tools for water management and hand-washing were in place and in use. During final evaluation

families declared having proper hygiene habits as shown in figure 8; mainly boiling water is mentioned in 4 villages in zones I&II (100% of families against 95% in mid-term

evaluation) as well as in zone III (98% of families in the final evaluation), followed by use of latrines that was mentioned in 2 villages in zones I&II and hand-washing by three villages in

zone III. There is a considerable improvement specifically in hand-

washing habits in zones I&II where 41% of the families was using nothing for it and 56% only soap during the mid-term

evaluation while during the final evaluation 63% use soap and 37% soap and towel.


Sanitation action plans elaborated and

implemented following adapted Community Led Total Sanitation Approach

There was no action plans linked to

sanitation. There were CBOs working in communities of zone III although not related to sanitation and hygiene

activities.

100 %

Specific action plans for sanitation situation improvement in communities of zone III were developed.

Also along the program a total of 260 PHAST facilitators were trained (75 in zone I, 65 in zone II and 120 in zone III).


Hygiene and sanitation action plans from the implementation period available in zone III; CBOs are active up to now and they have the technical support from the ward health officer; there is a need of linking the CBOs with the COWSO and/or VG in order to get some budget support to follow implementing

these plans.



Result 3: Sustainability, transparency and good governance of water and sanitation systems increased


Legal water user associations have been

democratically constituted, trained, registered and owning the systems in each

village by the end of the project

No water entities in place, in the case of

water systems already existent, the ownership was on VG´s side, not

following the water policy.

98%

Zones I&II: COWSOs and WUGs formed, registered and trained. All systems transferred (but Kalemawe).

Zone III: COWSOs formed, trained and registered. Systems not yet transferred. Maore ward COWSO split in 2.


COWSOs are recognised by the communities, registered, working and fulfilling their responsibilities of accounting and transparency at least in zones I and II. COWSO members are coming from the WUGs constituted at each DP in every village. All infrastructures handed over in all villages in zones I&II except

in Kalemawe as community doesn’t want to accept the system with the bypass in the water tank. In zone III registration of COWSO has been also finished, meanwhile handing over was on process in the moment of the final evaluation. Maore ward COWSO has been split into 2 independent COWSOs (1 for Maore village and 1 for Mheza and Kadando) due to internal conflicts between communities


Cost recovery system is established and functioning assuring O&M costs with less

than 25% of water rate unpaid after project completion

In the case of water systems already in place, there was normally a flat payment

per year and household, normally not related to the real consumption. Related to the cost, in the great part of

the communities villagers had to pay till 600-800TZS/20lts. in some cases.

90% More than 25% unpaid yet in the case of villages of Zones I&II

with flat rate (Mkonga and Chajo). In the rest of communities unpaid rate is less than 25%.

In Zone III tariff set up in all wards and currently is under implementation. In all the cases it will be a PAYF tariff.


Zones I&II water users are paying for water. PAYF in all villages except flat rate in Mkonga and Chajo, where the unpaid rate is still >25%; COWSO monthly reports show having some savings in the bank in all COWSOs after monthly expenditure-income balance. COWSOs have the financial capacity to

face breakdowns and periodical maintenance including allowances for DWD technicians. In Zone III, tariff already set-up in Kihurio ward, Kadando y Mheza villages and under discussion in Maore village.




By-laws approved by District Council have been established in the new water systems of the selected villages, including water

sources protection

Not applicable 98% By-laws approved in all the COWSOs of all zones I and II, at community and ward level, waiting from the last approval from

District. In the case of zone III a first version of by-laws was presented and approved at community level although currently is under

revision, linked to the changes in the COWSOs. In all the cases a specific by-law is included stating the restrictions and conditions related to the protection of the

water sources and the water quality. COWSOs are registered in all the cases.

Evaluation team’s comment

At the time of the evaluation, by-laws are still pending of approval at District Level to complete the whole process for all COWSOs.


District Water Department and village authorities have developed strategic plans to increase sustainable access to

water&sanitation in Same District at the target villages for the period 2011-2015 by the end of the project

Not applicable 100% District Strategic Plan include water and sanitation plan.


DWD has elaborated a strategic plan 2011-2015, included in the SDC strategic plan, based on the initial WPM (2006) and the second updated version of 2009; ONGAWA has acted as advisor for this activity.

The WASH strategic plan is in line with the decentralization plan and national water and sanitation policies. DWD head and interviewed technicians are trained, know the regulations and have the skills to develop plans for sustainable access to WASH.


A Geographical Information System with relevant data for W&S (access, health indicators, etc.. ) in Same District territory is

being used in the District Council towards effective resource allocation by the end of

the project

There was no proper information for planning by the District. Geo database from WPM 2006 was

available, although without any specific use for planning.

100% Water point mapping from 2006 updated in 2009. The DWD used the 2009 results to elaborate the New Water and

Sanitation Priorities document, that later fed the District Strategic Plan.

Apart from the database itself, there are some specific products as the proposal of specific index for planning, and the



simplified procedure for WPM update, that could help for a more practical use of the information.


First WPM was done in 2006 with support of ONGAWA, and updated as part of the MAMA program in 2009; the MoW took the mandate of including the WPM at national level and it is working (wpm.gaf.de) since 2010/11.


A monitoring plan established in the District to assist COWSO in water systems

management by the end of the project

Only DWST was supposed to be the responsible for supervision of COWSOs.

There was no real coordination or supervision. COWSOs had an important lack of support.

100% DWUS fully active in Zones I&II and starting in Zone III.

There is a reporting system that is arriving more or less quarterly to the District.


DWUS functioning with a working –plan for assisting COWSOs. COWSO monthly reports being reported to DWUS but in quarterly basis. Apart from the reporting system, periodical meetings between DWUS and COWSOs were held along 2012 and last part of 2011 (around 4 meetings were held). In each

meeting critical issues were identified to be treated. After that, looking for the economical follow-up of this initiative, it was revised in order to avoid dependence from District funds. So, the proposal has been to define a periodical COWSOs meeting (Umbrella COWSO), that is planned to be conducted in Same Town to fill the gap of DWUS not going to the field due to lack of funds. From each COWSO only secretaries should participate. Till the moment of

the evaluation visit, only one meeting was held in the last quarter of 2012. DWUS declared it as a very useful meeting but it was not again called.


4 technical proposals elaborated targeting

key national/international potential

donors/partners by the end of the project

DWD counted mainly on the “Serious

Water Shortage in Same” document as the unique and main tool for submitting proposals to donors.

100%.

DWD is applying the Strategic plan activities.


Some proposals prepared and submitted (Malindi and Mhezi) to other donors (e.g. Rotary Club) as per DWE information.


10% of Same villagers are aware of the rights, procedures and requirements needed

to demand for water services according to National Water Policy by the end of the project

Not applicable

100% A total of around 35,000 people targeted through campaign

developed in a total of 12 villages (the highest priorities according to the data).


Some new villages seem to be ready for water system process following policy procedures as per DWE information. DWE is also coordinating with CARE and World Vision to spread knowledge about NAWAPO in the villages they are planning to work.



Result 4: IWRM promoted at Same District plan


A water resources study is available for the

targeted areas, including an estimation of current and future demands, localization of

usage conflict areas and environmental problems, by the third year of the project, and shared with PBWO and DWD

N/A 100%

Water resource study available and shared with PBWO and DWD.


Environmental study of SOKOINE UNIVERSITY AND PBWO done at the early stages of the implementation period.


Specific good practices in planning for promoting IWRM are identified jointly with District and PBWO

N/A

100% COWSOs and DWD trained in basic concepts about IWRM; as the IWRM itself and how to take it account in the regulations

and activities linked to water for human consumption. Also, the foundation for the relationship between COWSOs and PBWO has been built up during the project.


PBWO have been training in WUP and IWRM at District and Community level during constitution of COWSOs and strengthen capacities of DWD.


IWRM concepts are included in the by-laws of the new COWSOs formed.

N/A 100% Basic concepts already included in the bylaws, mainly related to the specific measures for ensuring no contamination of the

water, and especially in the sources.


COWSO by laws includes IWRM concepts like water source management and protection.



During the implementation of the program unforeseen results and/or

activities have taken place? 22. There have been some changes in the results/activities as per external factors observed and experienced during the implementation (already identified in the mid-term evaluation) of the programme: i) new villages of intervention after a coverage study due to new interventions

in the sector by other governmental and non-governmental actors; ii) Result 4: inclusion of CLTS in zone III as per new approach promoted by the MOHSW; iii) Result 7: activity change as per the new Integrated Water Resource Management act from 2009 iv) inclusion of a local

implementing partner for Results 1 to 4 in zone III; vi) build of additional DPs in some hamlets of neighbourhood villages as they are on the way of the water scheme distribution line and to avoid continuous conflict between villagers (e.g. Kalemawe water scheme included a DP at one

hamlet from Ndungu village); vii) a participatory evaluation conducted by ONGAWA in July-August 2010 brought as result the planning of stabilization phase in each zone in order to

consolidate achieved results (for zone I and zone II only in this water facility). Stabilization phase for Zone III is under implementation and will continue after ending the water facility and it is funded by other Spanish donor.

23. Before the intervention the institutional support defined in the policy and the WSDP for the monitoring of W&S schemes was the DWST formed by the head of departments with DWE as

chair-person. The fact was that this team is not operative in the field as their members have too many compromises to be dealing with such ground level activities. Therefore the DWUS was formed within the institutional capacity development component of the intervention dealing with

the DWD but also the education, health, community development and planning departments of SDC. This unit is formed by technicians of every department, 8 in total and being the DWE the chair-person.

3.3. EFFICIENCY

Are the technical, material, human and budgetary resources of the intervention

adequate, for the achievement of the proposed objectives and results? 24. Distribution of roles/functions within ONGAWA Tanzania (Same and Dar-Es-Salaam) and

ONGAWA Madrid (headquarters) teams is clear and adequate for the intervention: working procedures and communication channels are efficient; teams in each location are very stable and professional with long working experience in the organization, and also working in the

sector and with the national and local governments. The roles/functions and responsibilities between ONGAWA and TWESA were not so clear during the first partnership agreement in 2011: weaknesses in the supervision/support obligations of ONGAWA towards TWESA work. For

the second year’s agreement some positive changes have been included to redirect the partnership and this has succeeded to some extent.

25. ONGAWA and TWESA implementation teams are adequate in number and skills. TWESA has included in its team previous ONGAWA staff (e.g. CDO). Both of them have enough skills in

infrastructure, capacity building and governance as well as national policies and gender mainstreaming related to WASH; however TWESA seems to have weak managerial skills which have affected to the smooth implementation of the programme; changes in its management

team at the early stages of the zone III implementation may have contributed to this situation, as well as the poor experience in managing multi-village systems with a relevant budgetary scope.

26. ONGAWA continues taking advantage of incorporating qualified volunteers (national and international) in their interventions following organisation principles. Field team in Same

recognized the positive contribution from the volunteers to the intervention through their assignments that are usually identified on a participatory approach between field team, country



team and headquarters. In Tanzania ONGAWA has already an agreement with some

educational institutions like the University of Dar-es-Salaam.

27. The original intervention’s timeframe has experienced some changes with an extension of 5 months basically due to the increase in number of beneficiaries, through multi-village water schemes in zone III, which means more complexity and therefore more time. The programme

has adapted adequately to this new situation and ONGAWA has ensured additional funds (Spanish donors) for covering it. In addition, ONGAWA has adequately introduced stabilization phases at the end of the implementation of each zone for consolidating the results of the

programme taking the advantage of being still in the district implementing the project in other zones with the same staff. As per the original timeframe of the program, some additional timing and financial resources have been raised and properly planned for the stabilization phase of

Zone III.

Has the turnover of resources into results been cost-effective?

28. The procurement processes internally in ONGAWA are well established, launched and coordinated by the country representation team in Dar-Es-Salaam. Following EU procurement procedures it might has experienced some inconveniences in some of the tendering processes

and contracts: i) impossibility of hiring local skilled labour for infrastructures component of the project as there is no local habit from small enterprises for being officially registered in the Tanzanian Revenue Authority thus only bigger enterprises may have access to contracts; ii) in

material procurement to assure quality is not easy in the national context as the competition makes that corruption exists. Coordination in procurement and tendering processes between TWESA and ONGAWA has gone properly.

29. The intervention has applied two different implementing procedures: direct implementation by ONGAWA – Tanzania in Zones I&II and through TWESA, local NGO, in zone III. Both

approaches seems to be adequate, but the second is preferable at strategic level, for contributing to the strengthening of local capacities, by using the previous experience gained by ONGAWA in the country and in Same district specifically in WASH in rural areas. In terms of

timing, as the partnership with TWESA started in 2011, 3 years after the beginning of the project, that provided enough time for ONGAWA to get experienced and known in the district, getting directly exposed to the challenges and advantages. TWESA recognised they have learnt

a lot from the experience though there have been some punctual discrepancies with ONGAWA.

30. In zones I&II, a PRA has been conducted guided by external consultants. In zone III a WASH situational assessment has been conducted including also participatory approaches by a well known and experienced social local enterprise in the sector, WEDECO. Ideally the

implementation of the zone III in partnership with this not for profit Tanzanian enterprise should be the best option but the similarity on the roles to be adopted between ONGAWA and WEDECO might affect to the efficiency of the program. Therefore the option of TWESA has

been chosen for the implementation of zone III, as per their previous experience in Kigoma water facility and other projects where TWESA worked as a subcontractor for ONGAWA.

31. Affordable technologies have been used: most water schemes are gravity schemes in each village, appropriate as per their cost-effectiveness; there are also pumped schemes like Makokane and Mgandu-Majengo systems in zone II and also hand-pumps in isolated hamlets.

The technical solution depends on the feasible options first, the operational costs and then the willingness and capacity of the users and it is agreed between DWE, VG and beneficiaries. In zone III multi-village system rehabilitation has been chosen instead of new single-village water

supply schemes; it should noted that these multi-village systems with large number of PCs may have more room to make profit and assure cost-recovery of O&M. In the sanitation component,

example improved latrines have been built with local improved materials but guidelines for locally available and more affordable materials have been given and community is using them in zones I&II; the CLTS approach and SWASH guidelines in zone III promotes also affordable

solutions.



32. There is a balanced and coherent budget distribution within the different components of the intervention:

Figure 9: Distribution of programme budget by components

Source: ONGAWA formulation and reformulation reports 33. There has been fluent and frequent communication between ONGAWA and EC through

quarterly and biannual reports. There have been 4 modifications during the whole timeframe of the intervention, they have been submitted on time and all have been approved by EC. According to the view of the EC’s representative in Tanzania the communication with ONGAWA

has been correct. Some more flexibility from EC in the procurement processes is demanded by ONGAWA although the guidelines clearly delegate the total responsibility of the quality to the

funded organisation.

3.4. IMPACT11

This criterion has been analysed in-depth only for Zones I&II, as they have already water supply systems operative since 2010 and 2012 respectively and hygiene and sanitation activities

have been completed since 2011 and 2012 respectively. Therefore impact might be more tangible in these zones rather than in Zone III, where project activities were still being finalized at the time of the final evaluation field visit. However, in some cases also findings from Zone III

have been included in the analysis.

Has the program contributed to the achievement of the general objective proposed? 34. Head and staff of Chajo dispensary (functioning since 2011) confirmed that water borne diseases had decreased, though occasional skin infection cases have increased. There is no

proof of direct link of such infection with water supply in the village, as by the time no water breakdown has been reported since it is operative; health data from official sources don´t allow

11 The indicators included in this chapter are the ones proposed by ECODE in alternative logframe matrix in annex D.



extracting a clear conclusion about a downward trend in water-borne diseases in Zone I. It has

to be noted that this is a common consequence on this kind of interventions: after an improvement of the WASH situation and beneficiaries making more use of health centres than

before, trends show just the opposite effect as in the context previous to the intervention many beneficiaries used to visit the traditional doctors and/or get self-treatment rather than visiting the health centre when having symptoms related to water-borne diseases (see data Annex J), In addition, there are also other external factors affecting this indicator such as the climate, heavy rains and droughts which are periodical in the intervention area. Only malaria incidence rate have been drastically reduced according to the DMO office:

Figure 10: Malaria incidence rate in zone I

Source: DHO (DMO Office)

Direct observation reveals that in zone I and zone II population looks healthy. The evaluation

workshops the beneficiaries declared that their health had improved as a direct impact of the availability of water and sanitation. It was mentioned at least 5 times by 4 community groups amongst villages in zones I&II answering to changes in the community after the intervention..

The families answering to the questionnaires found also that their health condition has improved since the implementation of the MAMA: in zones I&II it has increased from 94% to

99% the percentage of families accepting that their health condition is better between mid-term and final evaluations (the remaining families declared it has been same improvement only), while in zone III 90% also stated their condition in health is better (Figure 11).

However, families also stated that water-borne diseases continue being the most common diseases in the communities of the programme, that it would mean the existence of some

external factors affecting it in addition to safe and clean water and best hygiene practices. As it is shown in figure 12 diarrhoea has been the most mentioned disease in villages in all zones, followed by schistosomiasis, malaria and skin diseases. In zones I&II where project is already

completely finished diarrhoea was mentioned in 2 villages during the mid-term evaluation and it has increase to 3 villages during the final evaluation. In zone III has been mentioned in 5 villages but it has to note that in this zone project is being finished in June 2013 and still the

stabilization phase were the relevant topics from the project are being still emphasized in other to get higher impact in the mid-long-term.



Figure 11: Health condition of family members


Figure 12: Common diseases mentioned in the villages




Even when the families were asked about the reason why they visit to the doctor last time

(Figure 13), most diseases identified by the community members have been related to safe water and best hygiene practices to some extent. Main reason was declared as Malaria in all

three zones (in 2 villages in zones I&II in both mid-term and final evaluation and in 4 villages in zone III), followed by fever and skin diseases in zones I&II (mentioned in 1 villages in the final evaluation) and diarrhoea in zone III (mentioned in 2 villages).

Figure 13: Reason to last visit to the doctor by any family member


In conclusion malaria and diarrhoea are the most familiar diseases in zone III in final evaluation and in zones I&II mid-term evaluation, while the trend in zones I&II is not clear as schistosomiasis, malaria, diarrhoea and skin diseases were identified in the final evaluation.

35. According to beneficiaries, their children now go to school because they can take shower

every day thus they are clean and are accepted at school, not like before that children could skipped classes to avoid being thrown out of due to dirty appearance; one community group mentioned the increase in school attendance by children as direct change due to the

programme. Primary schools of Nasuro (in Chajo) and Makokane (in Makokane) were visited and the evaluation team confirmed this statement. Head of Nasuro primary school in Chajo confirmed that due to the availability of water at school, they can cook and the children get one

healthy meal midday, therefore the performance of pupils is also better; nevertheless they complain not having an storage tank, which is a problem as the water is not flowing in the school DP in daily basis. He also stated that attendance has increased from 70-80% before the

intervention to 90-92% nowadays. Similar explanation was given by Makokane primary school in-charge regarding attendance and availability of healthy food and safe water at school. Beneficiaries have the same perception in terms of school attendance and performance in zones

I&II specially (over 80% of the families in both indicators) and also zone III families have a positive overview (60 to 70% of them) though they have water just since 2 to 3 months ago:



Figure 14: Primary school attendance and performance


36. There is no clear evidence of significant number of women participating in income-

generation activities as result of having more free time (used for fetching water before the



intervention). Women’s groups in the evaluation workshops referred to having more time for

other activities and getting the children cleaner as an impact of the project. But other community groups which also included women in their teams (e.g. COWSO, PHAST, Sanitation

CBO, VG, Elders) stated that women now dedicate their free time for farming activities in the private farms and consequently they do not need to ask for much money to husbands; they also do fishing activities or home gardens. As mentioned in table 4 in point 19 of this report the

free time saved from fetching water near home is being used for different activities by women.

Small impacts in women’s life were also observed during the evaluation and confirmed by many beneficiary groups in the villages visited: women in general look very healthy and their children

as well; the visited households were very clean; some of the houses have small gardens with flowers and trees. The masika season (heavy rains), just finishing before the evaluation, might have also contributed to a nicer environment around households.

In the meanwhile, in Zone III, beneficiaries in the several workshops stated they are expecting to plant vegetables at home, mainly for domestic use, saving the money spent currently for them. In Maore the women stated to be benefiting from this activity when they have private

connections; similar plan had a woman in Kalemawe (Zone II) village with a family garden in the backyard that was not in use as she has not yet private connection and the DP close to her house is not supplying water in continuously basis.

Figure 15: Backyard garden in Kalemawe waiting for a PC for being operative

Source: evaluation field visit

Has the intervention achieved other unforeseen but positive impacts?

37. The project has served as a pilot model for other interventions and policy – makers at local/international level, e.g. establishment of DWUS. Same District has been the first district in Tanzania establishing the District Water and Sanitation Unit Support, DWUS. They have already

received the visit of at least other district members in order to gain experience from their work.

38. The project has contributed to the improvement of the environment in the intervention

area, especially with the promotion of protection of water sources; planting trees at water source areas has been one of the activities to protect them. Almost all the beneficiaries in

Zones I&II considered that environment in their villages has improved (Figure 16).

Not even one environmental problem has been identified by them during the final evaluation, though some of them were detected during the mid-term evaluation in zones I&II (Figure 17).



Figure 16: Environmental change in the villages


Figure 17: Current environmental problems in the villages

Source: Questionnaire for families conducted during final and mid-term evaluations Hygiene promotion best practices have also contributed to environmental improvement

reducing waste in household surroundings and surface water contamination in the villages. Improve of sanitation, cleanness of the community and overall cleanness were mentioned by communities as relevant changes due to the project. In Zones I & II 100% of the population

used to burn the home rubbish since mid-term evaluation and they keep-on with this practice while in Zone III a 6% practice burying the rubbish instead of burning. 88% and 87% of the



interviewees used to burn the home rubbish more than once per day during final evaluation in

Zones I&II and Zone III respectively:

Figure 18: Handling rubbish at home


39. Gender equity is, to some extent, present as cross-cutting issue in the new SDC strategic-plan 2011/12-2015/16, which identifies key issues affecting the district and provides strategies

to reinforce the vision and values of the Council and community: Improving social welfare, gender and community empowerment is among the 10 objectives of the District strategic plan; Gender quality and women empowerment is a direct objective under the Community

Development Service Area. Set up indicators strive to measure the gender participation in



development opportunities, community advocacy to marginalized groups including women. For

WASH related activities national policies gender approach is taken as reference.

40. The intervention has contributed to a more cohesionated community in the villages of intervention and to some extent in the whole Same District. Workshops conducted with the different community groups in the intervention villages during the evaluation have shown a

good group spirit and committed with public issues. This perception is confirmed by the data of qu9estionnaire to families (Figure 19): i) 97% of families in zones I&II declared that relation between families have improved (similar % as in the midterm evaluation), and; ii) Participation

in community issues through CBOs, general meetings and committees has improved in zones I&II since mid-term evaluation:

Figure 19: Villagers participation in community issues

Source: Questionnaire for families conducted during final and mid-term evaluations In addition to the collaboration with ONGAWA, District collaborates with other NGO and PBWO

creating new COWSOs and WUAs among Same District. DWUS coordinates with the existing water related projects in the District and take the advantage to reach the communities were the

organizations are working to follow implementation of NAWAPO. 41. Most of the VG are committed and collaborative with the community activities in relation to

WASH. Communities’ perception regarding better performance of village authorities in zones I&II high (82%) and has improved significantly since the mid-term evaluation when the 80% of the families in this zone considered the performance of them was equal:



Figure 20: What do you think about village/hamlet authorities performance?


42. The programme has contributed to reduce water expenses for beneficiaries using DPs. The price of water seems to be cheaper than before the intervention when many of the beneficiaries

had to make use of water vendors; e.g. in zone II Kalemawe beneficiaries were paying 500TZS/bucket before the intervention that was brought from Ndungu, nowadays they pay 25TZS/bucket from the village water supply scheme; on the other hand most of the private

connections from Zones I and II were getting water for free but with a irregular service while nowadays they have to pay a flat amount and the service is more regular. In general the beneficiaries in zones I&II found that the water service is now expensive than before:

Figure 21: Water tariff comparative




but at the same time they find the current water tariff normal (92%) even comparing with what

they were thinking during the mid-term evaluation (61% cheap):

Figure 22: Opinion about current water tariff


As in zone III during the time of the evaluation only in Kihurio ward was fixed the price and not yet in Maore ward and there has been not yet time for experience the data cannot be taken as

much realistic.

Has the intervention created any unforeseen negative impacts? 43. The project has not created serious conflict within the communities: e.g., participation,

communication, politics. Some conflicts founded in Zones I&II during mid-term evaluation have been already solved (e.g. Chajo and Kalemawe pipes being cut by Ndungu villagers where system crossing by their village, misusing of funds by COWSO in Makokane); For Zone III there

is no experience yet, but it should be noted that the multi-village system in Maore Ward was supposed to have one COWSO formed with members of the 3 villages supplied by the system.

However, due to conflicts among them during the implementation it was decided to split it in 2 COWSOs: one for Maore village and one for Mheza and Kadando villages, and that both share one office located in Maore village. During some evaluation workshops it also came out the

need of water points for cattle to avoid interfering in DPs but it does not seem to be a serious issue yet.

44. In some villages VG has shown interest in interfere in the COWSO management (e.g. Chajo in zone I and Maore in zone III); fortunately COWSO and the community in Chajo understood well the role and responsibilities of VG and COWSO in the water supply systems’ O&M as it was

stated during the evaluation workshop and the problem seems not to hamper the proper management of the water scheme. In Maore, as the intervention is still not get to the end and a stabilization phase of one year long is already planned, it seems like the potential conflict will

be solved within the intervention period with the support of DWUS and ONGAWA. In Maore the



conflict involves VG, COWSO and private connections of Mheza and Maore villages as some PCs

of one village are located in the second village administrative limits and it is not clear to which COWSO they should pay.

45. According to workshops and questionnaire to families the project has not overloaded women with new tasks; their roles have been strengthened. Most of the beneficiary women

declared that they are having more time for other activities as they have access to water close to their houses. Mentioned activities are all related to development and improve of live conditions from them and their families: working in farming activities, cleanness of the house,

planting trees and vegetables at home, children more healthy, more income, no domestic problems with their husbands among others.

46. The impact of TWESA among the community in zone III has not been such strong although some former ONGAWA staff was recruited by them. It seems that the top-down approach used by TWESA has made difficult the smooth implementation of the activities of the programme in

this zone; their poor experiences in such big multi-village programmes might contribute also to this situation.

3.5. SUSTAINABILITY

Are the positive effects of the project financially sustainable? 47. The DWD is active in supporting the COWSO in terms of major maintenance and/or breakdowns. The continuous support to W&S infrastructures and management teams is

responsibility of DWUS. Though this unit has been already fully recognized by the full council of Same District, it has not yet its own budget. Its working plan is included in the DWD budget under Other-Charges, OC budget-line. OC is the only budget-line that is not audited under the

district and department budgets and it is usually use for covering running costs (e.g. tea and other small office expenses). In the case of DWD in Same District this budget-line is also used

for paying the fuel expenses of the Same urban water supply system. According to the DWUS members and its chair person, DWE, it is not realistic and not even coherent to cover DWUS activities under this budget-line. In addition, the last 4 financial years the OC has been the

same (153 million/year) without increasing and not always the complete amount is effectively been deposited in the department account. At the time of the evaluation, DWUS activities are still supported, to a large extent, by ONGAWA.

48. Most of DP users in all zones I&II apply the principle of “pay-as-you-fetch”, PAYF, paying according to the amount of water they get, except in Chajo and Mkonga paying flat rate,

though water meters are in place and PAYF would be feasible and more realistic. The current water tariffs have been agreed by the communities themselves according to their affordability and willingness rather than focusing in cost-recovery of water schemes as it was originally

planned by the intervention. During the evaluation, the users from zones I and II stated their agreement with the current tariffs that are being reviewed by the community themselves to some extent, like in Chajo village where they have already doubled the flat tariff since the mid-

term evaluation by their own decision. Table 6: Water tariff at DPs

Zone Village Tariff approach Tariff at DPs

Zone I Chajo Flat rate 1,000TZS/month

Mkonga Flat rate 500TZS/month

Bangalala PAYF

Zone II Makokane PAYF 30TZS/bucket

Majengo PAYF

Kalemawe PAYF 100TZS/4



buckets/day

Zone III Kihurio ward (6 villages)

PAYF 100TZS/ 7 buckets/day

Maore PAYF Under discussion

Kadando PAYF Under discussion

Source: evaluation community workshops and info from ONGAWA

On the other hand, since they water supply schemes are operative in Zones I&II no water meters have been installed at Private Connections, PC, and they continue paying flat rate; in

zone III in Maore where there is a large number of PCs, rather than DPs, water meters are supposed to be installed as soon as the systems are operative and introduce the “pay as you fetch” principle in their tariff in order to assure the O&M of the water schemes.

The monthly water expenses in zones I&II have not increased significantly since midterm evaluation; from 55% to 62% for the group expending between 500 and 1,000TZS per month;

while the group of families expending more than 1000TZS maintains almost the same value (from 39% to 38%):

Figure 23: water expenses per family


Comparing this indicator with the monthly family income (Figure 24) it has to note that in zones I&II the monthly income has decreased since mid-term evaluation: while the 50% of the families were having a monthly income of between 20,000TZS and 50,000TZS during the mid-

term evaluation at the final evaluation this group has been reduced up to less than the half, 20%; in addition the reduction on the group of families earning more than 50,000TZS have been from 8% to 2% and currently the majority of families are in the group that have an

average monthly income of less than 20,000TZS, 67% in compare with the 11% during the mid-term evaluation. That means expending maximum between 2.5% to 5% of their income in water.



Figure 24: Monthly family income


49. Cost-recovery for O&M of water schemes is functioning in zones I&II. As per the COWSO monthly reports that include income-expenses summary and the bank/cash balance, all of them have a small surplus in their accounts. They are being able to pay for the O&M and repair costs

of the water schemes including the field travel costs and allowances of DWD-technicians when needed, as it happened during the evaluation visit with the breakdown of the pumping machine in Makokane during the evaluation.

50. The nearest bank branch to the intervention villages (including all 3 zones) is located in Same town. All COWSOs in zones I&II have a bank account there; no difficulty has been stated

by COWSO members during evaluation activities related to the fact that Same town is far. Only one COWSO structure, in Makokane village, has been renewed due to misuse of COWSO funds. In zone III they have also open the necessary bank account for COWSO but they have not yet

started with the payment/saving process.

51. Beneficiaries have the resources to keep on with basic hygiene as it has been observed during evaluation visit. They have been able to put in practice the use of local and low cost resources for applying hygiene habits learnt during the intervention as shown in figure below.

Beneficiaries in all zones visited were familiar with the tip-tap (Kibuju chirisi) for hand washing or local table for drying the kitchen dishes.



Figure 25: hand-washing in Kihurio and Kalemawe households, dish-drying in Kadando and Kalemawe, water for toilet in Kalemawe

Source: Evaluation field visit

However, at primary schools the provision of necessary tools to assure basic hygiene does not seem to be assured. Lack of information regarding the responsibilities of each stakeholder supplying them (parents, village government and DEO according to national guidelines) and

weaknesses in establishment of priority needs at school level seems to be the main cause of it. At the time of the evaluation, Nasuro primary school in Chajo had the tools for boiling and for drinking water but in the toilets there was no sink or other hands-washing facilities not even

water for the toilet; in Makokane primary school no tools for boiling water neither for hand-washing and toilets were available during the visit. According to the national policy and guidelines, parents are supposed to contribute for water-boiling tools same as they do with food

for cooking lunch for pupils at school, DEO provides annual capitation funds to each school (5000TZS/pupil) and school teachers supposed to set-up priorities of school needs including basic hygiene tools in annual basis.

Are the positive effects of the project technically sustainable?

52. Water schemes have been theoretically designed in all villages for a 20 years lifetime based

on calculations done with realistic population growth estimation (with 2002 census data, 1.6% annual growth and 211,738 inhabitants). The recent 2012 population census reveals a real population growth average in Kilimanjaro region of 1.8%, but should not influence the design

data according to theoretical flow of the systems. According to the design number of DPs built in each village ensured this service. National water regulations and policy state the need to



assure water supply service for beneficiaries at public point level (DPs) though PC are allowed

when technically possible. At the time of the evaluation all visited water supply schemes were working and in a good condition. Community are satisfied with the service provided in almost all

Zones I&II and they have not experienced big breakdowns since they are operative; in Kalemawe the water scheme built by the project has been modified bypassing the water storage tank as using it tank built for supply all the village was not giving enough pressure for

it. The technical solution has been proposed and conducted by ONGAWA in collaboration with DWE. Community and VG seem to be not fully satisfied with the solution though all affected hamlets are getting water with good pressure in shifts. In zone III the pipes broken due to high

pressure and apparently poor quality of material are being replaced and the necessary tests to assure proper functioning by ONGAWA. Communities needs and request to get adequate explanation about the progress and interferences of the ongoing activities in their villages.

53. In zone III the sanitation CBOs are working in CLTS and the PHAST facilitators in hygiene promotion approaching to the whole community including vulnerable groups but it seems like

no clear plans for them are still in place: stigmatization still exists in regard to PLWHIV in the community and it is not easy to reach them and know their specific needs; though in evaluation workshops all community groups affirmed that no community group had been forgotten in the

intervention, only in one village a group of disabled participated in the evaluation workshop; they declared to be satisfied with the approach used, that is getting help from family members

or neighbours to have access to programme results like water and sanitation services. No adapted latrines or any other specific actions for disabled have been observed or explained by sanitation CBOs or community members.

54. The type of latrines used by beneficiaries were observed during the evaluation visit: traditional latrines with just an unprotected hole in the ground and an external structure made

by local materials, unimproved latrines with a concrete slab and an structure made by blocks and cement, improved latrine with a improved slab and improved external structure. The type of soil is not an obstacle to build an improved toilet as it was observed in a neighbourhood in

Kihurio where a good quality improved latrines have been built in a very rocky soil, it is more a matter of economical affordability and culture the decision of changing from traditional to improved latrines. In this way for example in Makokane (zone II) a very isolated rural village

most of the latrines were traditional with a poor structure; on the other hand many latrines observed in zone III where the locations also are bigger and more populated have an improved structure, though still many are not ventilated.

Figure 26: type of latrines (Kihurio, Kadando, Kalemawe and Makokane,)


According to beneficiaries improved private latrines are not being affected by heavy rains, the

main cause of breakdowns of traditional latrines; the ones visited during the evaluation look in good condition; the 100% of the surveyed families have a latrine at home and is working properly in zones I&II; in zone III at least the 93% of the families have improved it and 86%

declared that it is working properly. Around 70% of them have improved their latrine since the intervention in all three zones:



Figure 27: Improving latrines

Source: Questionnaire for families conducted during final and mid-term evaluations Latrines in schools are working but their maintenance look very poor. In the case of schools,

mostly all latrines visited that were built by the programme (latrines for girls in Nasuro School in Chajo and Makokane primary school) have the doors damaged obstructing privacy for pupils, especially for girls. In Kalemawe the front wall of the toilets, that fenced somehow the toilets

building, is cracked since shortly after its construction and no repair plans from school or VG side has been planned. Community and VG are expecting ONGAWA to repair it and are not

ready to get it handed over. Figure 28: Latrines for girls in Makokane primary school




55. The COWSO, PHAST facilitators, CtC teachers and CBO at village level and DWUS and DWD

at district level have enough skills and tools to conduct the O&M of the W&S infrastructures; COWSOs have trained plumbers to deal with it; they have also the means for get the spare parts needed in each case as it has been shown also during the evaluation visit in Makokane

village: the water pump from the borehole pumping scheme had a breakdown a week ago and COWSO in collaboration with DWD water technician have managed to identify the problem, get the spare part from Dar-Es-Salaam and they expect to replace it the day after the evaluation

visit. In the meanwhile community of Makokane had no safe and clean water and they were getting it alternatively from the lake while at school the odour in the latrines was quite unbearable and no water for toilets was in place. On the other hand for Zone III multi-village

system it make sense that the COWSOs contract a professional staff for its management as per its complexity with the large number of PCs in different villages and the probably financial affordability as result of the high monthly income.

PHAST are working in all zones though they have reduce their activity in zones I&II as they have to plan now by their own since ONGAWA is not there anymore but they are active and recognized by the communities.

CtC teachers are still present in the primary schools and they have a subject related to hygiene promotion and sanitation in their school timetable. School clubs still working but not such active

as during the MAMA implementation period in zones I&II. In zone III, for the community sanitation component CLTS approach has been used through existing sanitation CBOs. A specific schools hygiene and sanitation component has not been implemented, since a more

community-answer based approach has been used, in coherence with new draft SWASH guidelines (according to these guidelines schools committees and parents should mobilize funds for latrines construction/rehabilitation), therefore there are not CtC teacher neither school clubs

in the primary schools.

DWUS is working alone as an independent entity in the district and ONGAWA is acting as adviser. DWUS have already organised periodical COWSO meetings in Same Town, for

experience sharing and lessons learned between the different COWSOs active in the District. In this way the Njoishi COWSO from Njoro, village, pilot phase from ONGAWA in Same District, is being taken as a reference for the new COWSOs for its well done work, both technically and

financially. The DWUS members have taken their new duties as such members after being informed and trained about their new tasks, but their previous duties are also included in their job description that means they could be overloaded with the additional tasks in DWUS when

COWSO number increases among the District.

56. PCs are a present fact in most of the villages, increasing since the running of the water schemes in zones I&II. The technical advice of DWD to COWSO is being necessary for ensure proper functioning of the water schemes with the new PCs and capacity limitations has to be

well defined in order to avoid systems collapse. At the time of the evaluation, coordination between COWSO and DWD was good and no technical problems due to large number of PCs’ request were found.

Is there enough institutional support for the project? 57. Water and Sanitation in one of the five core priorities targeted in the Tanzania Five Year Development Plan 2011/12-2015/16. The aim of these development plans is to contribute to

the achievement of results of Vision 2025. Improving access to safe water and improving water supply infrastructure in rural areas as well as strengthen water resources management remains a priority to GoT for the FY 2013/1412.

12 Mapendekezo ya mpango wa maendeleo wa taifa kwa mwaka 2013/14 (GoT Development Plan for 2013-14), Planning commission, President’s office, March 2013



58. According to the participation of village, ward and District authorities in the evaluation and

their knowledge of the project activities, it is confirmed that they have been participating actively in the implementation of the intervention and are supporting the stabilization phases to

consolidate results and have more warranties towards sustainability. As mentioned before, the implementation has been done hand by hand with district authorities. Full Council has been also involved in the recognition of the main coordination entity at District level (DWUS). In addition,

the new Strategic Plan of the Same District Council in which ONGAWA has participated advising for the WASH sector includes the following purpose, goals and strategies for the sector (Table 7). The main goal of this plan in the water sector is to achieve coverage of 65% from the 43%

in 2005. Table 7: Same District Council Water Management strategic plan

Main

Objective

Quantity and Quality of social Economic, social Services and Infrastructures

improved

Targets

Clean and safe drinking water available within 400m for households by June

2016

Strategies • To design and prepare BOQ for 20 water supply projects by 2016.

• To rehabilitate and construct 24 water supply projects by 2016.

• To increase the number of improved water points by 2016.

• To construct 25 rainwater harvesting infrastructures in public institutions by

2016

• To conduct 25 sensitization meetings in rainwater harvesting at community

level

• To facilitate auditing of 25 water projects by 2016

• To establish 15 water users entities by 2016

• To facilitate the training of 8 members of District Water and sanitation team

by 2016.

• To train 60 water schemes attendants by 2016.

Source: SDC Strategic-Plan 2011/12-2015/16

59. There has been also involvement of other district departments like education and health for

PHAST and CtC. Both programs are working in zones I&II. There is a good perception of the communities in regard of the District performance (Figure 29): 75% in zones I&II and 65% in zone III declared that their performance is better now.

There is a CtC coordinator at the DEO office trained for technical support to the primary school teachers participating in CtC for hygiene promotion. The District Health Officer, from DMO office

has been active during the implementation of PHAST activities and since the intervention there is a PHAST coordinator at district level trained and active as technical support to village PHAST facilitators. DHO has been trained in CLTS as well. CtC coordinator is member of DWUS but

PHAST coordinator is not.



Figure 29: Has the performance of District authorities changed?

Source: Questionnaire for families conducted during final and mid-term evaluations 60. The PHAST facilitators in zones I&II are following with the hygiene promotion activities at community level although they claim the lack of incentives with support from VG and District level. They are recognized at village level but they claim the need of official certificates from

their trainings and work done with ONGAWA to make this recognition effective. Similar situation is happening in Zone III with the sanitation CBO as explained by UKIMAU members (sanitation CBO in Maore). In most of the cases CBO members are also PHAST facilitators, but not all

PHAST facilitators are sanitation CBO members. PHAST facilitators are working since the zone I at the beginning of the MAMA and are well known by local and district authorities. In Zone III, CBO members are also active in the villages since some years ago, though not specifically in

sanitation and CLTS; it seems to be more room to improve on support from District and local authority during the stabilization phase. The technical support is already there through the ward health officers as it was confirmed during the interview with the Maore ward health officer who

has been also trained during the intervention and it is a representative figure from the SDC. Budgetary support is needed from District and/or Local authorities. .

61. ONGAWA is very active in the main coordination platforms/forums at national and international level. ONGAWA is one of the advisory committee members of TAWASANET (the

other members are Water Aid, UNICEF, SNV and Concern). At the Informal International Group (IIG) quarterly meetings they share experiences and lessons learned and coordinate interventions with other international NGOs working in development in Tanzania and also invite

Tanzanian governmental bodies (TRA, Immigration...) to review regulations, rules, laws affecting international organization in the country. It is also active jointly with other NGO in advocacy for policy making and improvement of the sector through the MOW (CARE, SNV,

WATERAID, PLAN). It is also member of the District Advisory committees acting as advisor for District development challenges and strategies. ONGAWA has awarded a new environment project which includes a water and sanitation component in the Same District for a 3.5 years

period starting in 2013 being an opportunity to consolidate their position as development stakeholder in Same District. For this intervention the technical office will be located in Maore; but there is already an agreement for sharing premises of DWD with a small office for



administrative and logistic issues in Same. This will allowed reducing costs of hiring an extra

office in Same and even more proximity to the District.

62. TWESA is also an active member of TAWASANET and it is a well known Tanzanian organisation with experience with other national and international NGOs and agencies. The previous manager of TWESA, the one who signed the partnership with ONGAWA, is nowadays

the chair-person of this network. There has been no possibility of contacting the management of TWESA to know in depth about their involvement and activity in such platforms and its future plans in Same and/or potential collaboration opportunities with ONGAWA.

Are the positive effects of the project environmentally sustainable? 63. Specific actions for water sources protection have been included in the by-laws of COWSO: planting trees, fencing the area, not allowing latrines in a certain area around the water source

are the main protecting actions. This is supposed to be managed by the COWSOs with support of DWUS though the final monitoring responsibility of water sources is of PBWO. COWSOs and

DWD have been trained; the first COWSOs of the intervention (Zone I) were trained directly by PBWO and the followings by ONGAWA with PBWO training materials. PBWO complains not getting yet the request of Water Use Permit, WUP, processing from COWSOs created during the

intervention. This permit is a legal support to COWSO to avoid conflicts around water sources. It seems like there is the knowledge of the WUP at COWSO level but there is no direct support from DWUS for supporting them in the request. PBWO assists COWSOs in this matter after

request but there are no specific follow-up plans from them. This is a common problem in the District, as costs linked to the transport between Same and Moshi, location of PBWO, is quite high. PBWO is in the process of opening a new office in Same town, that will contribute to solve

this problem and other related to coordination with PBWO. 64. New private latrines built after the project Hygiene and Sanitation campaigns and trainings

are located in the surrounding of the households; the construction type and location is correct and there is no danger for contamination of DPs, surface water or boreholes in the areas of intervention.

65. PBWO monitoring plans for water resources include a water quality test at water sources, at least twice a year: one in the dry season and one in the rainy season. Water quality test at

DPs should be monitored by COWSO with the support of a laboratory (nearest at PBWO in Moshi and in Arusha). This action is not included in the COWSO activities and it seems to be

very expensive and probably not affordable with the current water tariffs. DWD have been trained on water quality and could support COWSO on this (e.g. with a portable kit), but they still have no enough expertise neither the resources (portable kit test is not officially recognized

and the DWD doesn´t have the mandate for that). PBWO might also assist COWSO, through DWD, after request.

Is there enough ownership of the project by local communities?

66. Since project inception all activities at community level have been started through a village general meeting; all entities created during the intervention and its active members have been democratically elected within the communities and they participate freely in WASH activities in

their communities. The perception during the evaluation workshops is that communities know the role and responsibilities of COWSO and WUG in regard to water and also of PHAST promoters and CTC facilitators in regard to hygiene and sanitation; they are recognised by the

villages and can work without pressure for their communities; beneficiaries declare accordingly in the questionnaire for families:



Figure 30: Villagers opinion about water management civil society structures


In the case of water related entities, COWSO and WUGs (Figure 30), the already positive

perception has considerably increased since the mid-term evaluation in zones I&II: COWSOs are qualified as very good (70%) and good (30%) and WUGs are very good (43%) and good (54%) in the final evaluation from over 90% only good at the mid-term evaluation.

In the case of hygiene and sanitation promoters, PHAST and CtC in the community and in schools (Figure 31), the positive perception from mid-term evaluation has improved among the beneficiaries of zones I&II:



Figure 31: Villagers opinion about hygiene promotion civil society agents


73% of the families qualified PHAST promoters as very good and 23% as good in the final evaluation against 98.5% as good and 1.5% as very good in the mid-term evaluation. CtC

improved also in a similar way in zones I&II (69% very good and 28% good) but there is still a 2% qualifying them as not good. In Zone III PHAST promoters are well accepted by the families (60% as very good and 37% as good) as well.



On the other hand the sanitation CBOs involved for the sanitation component with the CLTS

approach in zone III are seen as good for more than a half of the families that participated in the evaluation survey:

Figure 32: Villagers opinion about sanitation promotion civil society structures

Source: Questionnaire for families conducted during final and mid-term evaluations Taking into account that most of PHAST promoters are also members of sanitation CBOs it is to

note that while PHAST promoters were seen as good or very good by a total of 96% of the families only 74% were from the same opinion in regard to sanitation CBO and even 11% were

the opinion that this CBO is not good. It might be because CLTS is a new approach at national level and there is not yet much experience on it. Nevertheless during the evaluation workshops no negative statement from sanitation CBO members or from community members was

declared, but a good perception of the sanitation approach used. But since some weaknesses have been identified in this component, a specific action in primary schools has been also included in the stabilization phase with support of local CBOs for zone III.

67. Though the willingness of community to pay for water services seems clear while systems are already operative, during the project design and implementation, the cash contribution

agreed for development projects (5%) at village level has not been covered completely; on the other hand it has to be noted the commitment in the in kind contribution with unskilled labour during construction and rehabilitation; the community members themselves have motivated

each other in order to follow with the programme of water supply and sanitation (e.g. in Kihurio, men were not ready to collaborate and women themselves started digging trenches as they see water supply as a very hard issue in their daily life, once the men observed this action

they react and started contributing as well).

The cash contribution, though initially is thought as to cover part of the capital costs, definitively is for the cost-recovery of the O&M of the water-schemes to be managed by

COWSO. As at the beginning of the intervention the VG is the only one with a bank account the contribution goes to them and it supposed to be transferred to the COWSO bank account as

soon as they open their account. The contribution is not done at once and depends on the affordability of the beneficiaries and community internal agreements on the amount and the way to be completed. Once the COWSO received the contribution in its account it should pay an

annual agreed “fee” to the VG as it is a body within its village and to avoid future conflicts between them.



68. Primary school teachers in Zones I&II continue with the CtC lessons from Std. 3 to Std. 7; school clubs are active but not that much as during the implementation phase of the

intervention. As mentioned before specially toilets hygiene condition was not the most adequate during the evaluation visit. This situation might be due to weak support from DEO and weak motivation of school teachers but can still be improved working with the District during the

stabilization phase of Zone III. 69. There is no evidence of traditional leaders/beliefs obstructing the key WASH messages

promoted by the intervention. Some of them participated in the evaluation workshops and their opinions were aligned with the ones from other community groups in regard to the intervention components.

70. The analysis of CBO has been done in zone III for electing sanitation CBOs for the CLTS component of the intervention. Nevertheless a deepen CBO assessment has not been observed

and advantages of involving them for the cross-cutting factors, like vulnerable groups, has not been maximized. For example, KIWAKUKI, working with PLWHIV has presence in each village and could be used for approaching this community group. In Zones I&II no deeper knowledge

of existing CBOs has been observed at community level. There is no opinion about other CBOs at the final evaluation while during the mid-term evaluation were mentioned as good by 25% of

the families though the CBOs they were referring to were not specified. 71. Though the HRBA used by the intervention and the multiple trainings at the communities

with the different community groups, the obligations and rights of community members seems to be not much clear for most of the beneficiaries. There has been no answer from families in regard to their duties as community members while in the mid-term evaluation they already

mentioned the water source protection and the payment of water at least in one village of zones I&II; in regard to their rights the participation in community activities was mentioned in one village also during the mid-term evaluation in the same zones but no answer during the

final evaluation. Men and women groups participating in the evaluation workshops showed also few knowledge about their rights/obligations; however the COWSO, PHAST and sanitation CBO members participating in the evaluation workshops showed a good knowledge of their duties

and rights as well as the VG members, confirming that community members with more level of education are aware of them but still it is difficult to make it understand to the members without or with a very low education level.

72. COWSO, PHAST facilitators and sanitation CBOs have included women in equitable number

and they are being part of decision making forums in some of the villages, though in some others they have still not much voice. In PHAST and sanitation CBOs members are mostly women while in COWSO, following the national policy, they are balanced to 50%. In Chajo

COWSO even one woman plumber and she is accepted and recognised by the community and local authorities.

73. Whenever possible the civil society organizations created during the intervention have tried to create synergies with other CBOs existing previously in the villages: the sanitation CBOs have been selected from already existing CBOs working in environment, many of the PHAST

members have been already community health workers or members of the village health committee, the COWSO have include village plumbers as members.

74. There seems to be no socio-economic conflicts within communities in any of the intervention zones. All community members are satisfied of having water for a cheaper price than before at the DPs. This matter is still open for PCs and the need of installing water meters;

this issue might cause some conflicts especially in zone III due to large number of PCs, as mentioned before; stabilization phase period should be used to keep supporting this aspect.



4. VISIBILITY

75. Proper communication and information channels have been used to explain the Project, and the roles of ONGAWA to the District, Village Authorities and community through DWD and

village sensitization meetings at Zones I&II; in zone III, though roles of ONGAWA and local partner, TWESA, have been explained since the beginning, the end of the partnership between ONGAWA and TWESA at the end of 2012 before finishing the activities have caused some

confusion among the communities in Kihurio and Maore ward afraid of the project not being finished; since beginning 2013 ONGAWA is working hardly in this zone in order to accomplish

the activities and achieved the expected results up to the end of the stabilization phase in this zone.

76. ONGAWA has informed the target community and other stakeholders about EC as main donor of the intervention. All constructions in the field have the logo of ONGAWA and EC painted and there are numerous signs with both logos as donor and implementing organization

respectively at the outskirts of villages/wards of intervention. Cars have also EC logo and other visibility materials such as T-Shirts and stickers have been created for visibility purposes with project name and/or key hygiene promotion messages with logos from both, EC and ONGAWA.

However the beneficiaries recognized EC as donor but are not able to explain easily about EC. They see ONGAWA in zones I&II and TWESA and ONGAWA as “donors” of the intervention.

77. Experience role and visibility of ONGAWA, specially, but also from TWESA and EC have been reinforced against communities and local authorities as well as SDC. The combination of their work at local level as well as their participation in national and international forums

contributes to their consolidation in Same District and Tanzania. the change of the name of the NGO from ISF to ONGAWA in 2012 has caused some confusion between the communities and even some district members but during evaluation some more clarification has been given in

order to make it clear for the future.

5. OVERALL ASSESSMENT

The Water Facility from EC has contributed to improve the basic services in the Same District

and indirectly the MDG in Tanzania. The education component on WASH has deeply got into the community. Changes at community and District level after the intervention are tangible in addition to the infrastructures provision expected by the programme. The integral intervention

design with a HRBA used by ONGAWA has contributed to a big impact in both right holders and duty bearers. Institutional development and capacity building at this level has made a change

as the SDC is now skilled and capable to design and manage new WASH interventions according to its new Strategic Plan. High participatory approach at all intervention levels has resulted in a very good coordination, understanding and effectiveness of the intervention

outcomes.

Well committed counterpart has facilitated the work as well as long-term intervention from ONGAWA in the district helps counterpart to increase confidence and stability. But for

sustainability matters traditional difficulties and limitations from Tanzanian Central Government to secure their local financial contributions (less amount than expected, significant delays for releasing funds, etc. to the District), might affect to the planning and management of the whole

intervention (e.g. enough financing of DWUS).



The strategy of strengthening local capacities of implementing partners is an added value from

which ONGAWA should take advantage to be installed as advisory in WASH at national level. However it has been identified that more reflection on this kind of partnership needs to be done

for better success in future interventions. ONGAWA’s networking capacity and field experience make possible its advocacy role in the sector being nowadays an international reference for this African region.

In terms of intervention design the experience of including such called “stabilization period” at the end of the implementation of each zone for consolidating results put in place further guarantees towards sustainability.

In addition quantitative impact indicators on gender are limited in the design, while qualitative impacts are tangible but not measurable. Advantage of the good relationship with the counterpart, SDC, and the deep knowledge of the District context, should give room for more

specific agreement in terms of impact indicators.

6. CONCLUSIONS AND RECOMMENDATIONS

6.1. CONCLUSIONS

6.1.1. Strengths

Considering a stabilization period for each zone after ending the implementation in order to consolidate results and put in place further guarantees towards sustainability. This has been possible during MAMA implementation period for zones I&II as MAMA

was still under implementation in other zones. For zone III, additional funding has been secured from Spanish donors for one year stabilization phase taking the advantage that ONGAWA will continue working in the same district after completion of MAMA.

Ongoing implementation, of a local Tanzanian NGO (TIDI), launched by ONGAWA

Tanzania in order to increase fundraising possibilities for future projects in Tanzania

and also to expand the experience of ONGAWA through a local NGO making it more sustainable through other possible partnerships and with access to other resources.

Periodical participation of ONGAWA at District Advisory committees is an added value in

terms of advising for strategic and development plans for the District on WASH and

IWRM.

The new intervention of 3.5 years in Environmental sector with a Water and Sanitation

component, funded by EC, in the upstream villages of the MAMA intervention in Same District.

The intervention goes fully in line with the country strategies, policies and legislation of the sector.

Opportunistically following decentralization process promoted by the government, ONGAWA has pioneered and piloted the creation and set-up of the first DWUS in the SDC to support the water supply and sanitation systems at local level.

Project design combines, in a balanced way, community capacity building, institutional

development and construction of infrastructures, all with an integral and right-based approach. The significant previous experiences of ONGAWA in Tanzania and lessons



learned from previous interventions have contributed to a highly improved design of the

current MAMA programme.

ONGAWA avails experienced staff knowing in depth the context of the water and sanitation sector in Tanzania, the governmental context and the strategy and way of working of ONGAWA itself. All this advantages make the implementation of the project

much effective and efficient. All technical designs are overall correct and appropriate for the local context.

The effective incorporation of ONGAWA volunteers in the project (both from Tanzania and Spain) strengthens the organization’s principles and values, and contribute to increase the efficiency of the project.

ONGAWA has adopted a sound multi-stakeholder approach for this intervention:

virtually all relevant groups at all levels (public-private-community) have been somehow

involved and contributed positively to the project. The focus has been properly set on district and community level, which allows the NGO to maximize its comparative advantages and added value.

Geographical concentration and design in phases, all following local authorities’

priorities and with a basin-approach, has proven to be highly appropriate, in order to maximize impact and effectiveness, and secure incorporation of lessons learned from previous experiences.

ONGAWA has shown enough flexibility as to get adapted to all unforeseen factors

happened during the implementation (other NGOs implementing projects in the same

target areas, legal and institutional changes on water sector in Tanzania, etc.), minimizing the risk of overlapping and other negative impacts on the project.

Environmental issues directly linked to sustainability of water sources have been properly and effectively incorporated in the design of the project.

Visibility and role of ONGAWA and EC is undoubtedly being strengthened in Same (and Tanzania) as a consequence of the project.

6.1.2. Weaknesses

The terms of the partnership agreement with the local implementing partner has not agreed supervision activities from side of ONGAWA and have affected the smooth implementation of zone III activities.

Construction problems during implementation of Zone III due to poor quality of some

materials supplied (e.g. pipes); though there was a good reaction of ONGAWA by

replacing them still with the implementation-stabilization period.

Lack of gender impact indicators built-in the design, makes it difficult to measure

relevant impacts in women’s life (though some indirect indicators have been considered).

Roles and responsibilities from TWESA and ONGAWA have not been sufficiently shared and made clear to VG and beneficiaries.



6.1.3. Opportunities

DWUS have been approved and acknowledged at SDC and by the Council of Councillors (Baraza la Madiwani). DWUS’ members are trained and count with a realistic working-

plan.

Some COWSOs have already taken the initiative to review and increase their water

tariffs in order to ensure their capacity to cover their own O&M costs. This could be an example for the rest of the villages.

Njoishi COWSO from pilot phase is working very well. DWUS keep on using it as example to promote COWSO best practices in the MAMA programme COWSOs and along the District.

MOW could build into the experience of DWUS in SDC, to replicate and scale it up to

the rest of the country. The advocacy role of ONGAWA at national level seems to be

one of the keys for this.

There is room for ONGAWA to deepen into the strategy of partnering local NGOs as

TWESA, concentrating efforts on lobbying and capacity building with such local partners to promote local sustainable development and strengthening of social fabric in

Tanzania.

ONGAWA to share their experience into platforms as TAWASANET and MOW, in order

to facilitate replication/scale-up by other stakeholders in other similar regions in Tanzania.

6.1.4. Threats

Legal registration of COWSOs completed in all zones but by-laws not yet approved.

No Water Use Permit (WUP) has been requested yet by COWSOs to PBWO. Thus, the

prerogatives and benefits of that WUP (WUP gives COWSOs the legal right to use the

water source, making them more involved in protecting it with the support of PBWO and helping when conflicts between villages due to water sources appear). The cost of WUP (initial fee of 40.000 TZS plus 35.000TZS/year) may refrain COWSOs to apply.

No water meters installed in private connections in any of the water schemes and

owners not used to pay according to “pay as you fetch” principle, which may jeopardize

the financial sustainability of the water schemes.

DWUS activities’ budget is included in DWD budget under OC-budget-line; it might weaken they recognition at District level, since they could not be able to secure enough funds for doing their job.

Development NGO coordination forum at District level not active enough and might

cause potential overlapping of interventions.

Lack of a proper baseline with official health data, difficulties to select control groups

and/or get accurate data from health centres (lack of health centres in all the villages

and/or lack of registration in other centres by origin of the patient, number of cases of water related diseases are not treated at the health centres, but with traditional healers, etc.) may all create difficulties to strictly assess the cause-effect relation and

net impact of the project in reduction of morbidity and mortality for water-borne diseases.



Even not being exclusive for this project, traditional difficulties and limitations from Tanzanian Central Government to secure their local financial contributions (less amount

than expected, significant delays for releasing funds, etc. to the District), are seriously affecting to the planning and management of the whole intervention (e.g. enough financing of DWUS).

No monitoring plan established for water quality control at DPs, from COWSO with

DWUS/DWD support.

Some punctual social and political conflicts regarding the water sources on concrete

villages (currently in Maore ward) may negatively affect to the achievement of results,

so as to the global visibility of the project.

Periodical drought and hunger periods (as has happened at the end of 2005 and

beginning of 2006 and repeated in 2009) and/or heavy rains (as in the beginning of 2012) in Same District can be a serious obstacle for sustainability of the systems, both financially and socially.

Weak contribution of beneficiaries in unskilled labour during construction activities of

Zone III might have caused delays in the implementation timeframe.

Definitive sanitation policy not yet in place and strategy for school sanitation, SWASH,

very new. Sanitation services for pupils still very poor.

6.2. RECOMMENDATIONS

6.2.1. Common to all stakeholders involved

In multi-village water supply schemes ONGAWA and DWUS to make sure that agreement for installation of water meters is done between PC owners and COWSOs.

ONGAWA and DWUS to make sure at the early stages that the community understands the main objective of the project, especially regarding water supply and private connections.

Promote allowances (posho) for making sure that PHAST facilitators continue with their

activities at the hamlets (possible options: community contribution in kind, COWSO or

sanitation CBOs contribution in cash or VG through village action-plans budget, DHO through sanitation funds).

Primary schools: ONGAWA and DWUS to advocate and remind their responsibilities on

providing water-treatment tools (parents) and other hygiene & sanitation tools (VG and

DEO capitation grants); promote low-cost technologies proposed in SWASH guidelines (2010) specially for hygiene practices.

DWUS with ONGAWA to prepare monitoring plans of water supply systems’ capacity and inclusion of new PCs to ensure the 20 years lifespan of the water schemes.

DWUS to support COWSOs for getting the Water Use Permit issued by PBWO. This will minimize conflicts between villages due to shared water sources.

Design a feasible and affordable water quality monitoring plan at DPs involving COWSOs, DWD and PBWO. Assess the possibility of a contract or collaboration



agreement with PBWO with the DWUS to support all COWSOs in water quality at lower

cost.

DWUS and COWSOs to emphasize the periodical review of the water tariff in each water supply system with community participation.

O&M of water systems in all zones: leave DWUS and COWSOs alone to run them during stabilization period; ONGAWA to act only as advisor planning supportive supervisions and giving technical and managerial support just when needed.

Promote that COWSOs have their own office avoiding its share with VG office, in order

to keep on differentiating their roles and responsibilities in regard to WASH in the

villages.

DWUS to ensure proper identification (T-Shirts, working tools) of PHAST promoters and

Sanitation CBOs in the community with ONGAWA support.

To keep on increasing efforts towards jointly design of disaster risk mitigation and

preparedness plans (community, local authorities and ONGAWA) for minimizing the negative impacts of periodical droughts and consequent hunger epidemics and heavy

rains and consequent floods damaging the infrastructures in the future.

6.2.2. Specific to ONGAWA

Avoid overlapping/missing indicators for objectives & results and strengthen specific gender indicators in the logframe for measuring achievements and impacts Avoid too many results in the logframe and group them accordingly to assure internal coherence

and to make the monitoring easier (see alternative logframe proposed).

Make use of already existing indicators at District level, in order to measuring the

impact in a efficient and smooth way, avoiding extra work for District staff in health and education, and increasing reliability of data (e.g. school attendance and performance indicators for the general objective were not available at DEO; specific water related

indicators were not available at DMO office). Make direct follow up at schools and health centres to get very specific data or use other reliable sources.

At least 1 year of stabilization phase for each zone to be included in WASH programme implementation timeframes.

Make sure communities understand the technical infrastructures and decisions related

to problems happened, e.g. bypass in Kalemawe system storage tank. Use easy

language adapted to community understanding.

Specific needs of vulnerable groups (PLWHIV, disabled, elders, children at school) to be

considered during the design of the intervention: i) Promote representatives of vulnerable groups as COWSO members and PHAST

members during their establishment to assure their needs

ii) Take advantage of existing CBOs to approach vulnerable groups (e.g. KIWAKUKI for PLWHIV).

If technically feasible, multi-village water system seems to be the approach to be prioritized in the future; although being more complex to organize at the beginning, it is more coherent (water-basin approach) and much easier to implement since it avoids

conflicts of interests between neighbour villages.

Zone III: ONGAWA to conduct meetings in each village and at District level for

clarifying objectives of stabilization phase and agreeing on joint activities.



Deepen into partnership strategies with local NGOs by sharing experiences, improving from previous lessons learnt and strengthening local capacities.

Keep on framing interventions into national WASH policies and advocating at Ministry

level for scale-up and/or replication based on lessons learnt.

Although water is not a direct priority for EC within the next 5 year period, still explore

funding opportunities for WASH interventions linked to agriculture, energy and/or

environment/climate change instruments.

Keep-on sharing experiences with other NGOs working in the geographical area and/or

sector to learn from different successful experiences and share for advocacy.

Local partner: Make sure of sharing the same mission, vision and values and check

carefully its previous management and financial capacity/experience in similar interventions; Partnership agreement: Include detailed roles and obligations of each implementing and contracting partners and also conflict solutions mechanisms;

MoU with District: in case of partnership agreement of ONGAWA with a local partner leave room to local partner for working directly with SDC having regular supervision

from ONGAWA when implementing activities (and not just accessing SDC always through ONGAWA.

When working with local implementing partners increase transparency to communities and local authorities on roles and responsibilities of each partner.

Keep on working with volunteers and universities as a way of adding value especially for technical components.

Use previous lessons learnt in terms of cost-effectiveness for decision-making in applying low-cost technologies (e.g. rope pumps vs. manual hand pumps).

Material procurement procedures: ONGAWA to make sure that a proper quality is secured at reasonable market prices.

Provide with certificates to PHAST facilitators in all intervention villages.

Implementation timetable to be adapted to seasonal community activities to avoid affecting their participation.

Try to minimize the negative impact of periodical draughts and other climate external factors, by strengthening collaboration, coordination and previous contacts with other relief-based organisations in the area, developing a systematic protocol for this kind of

situations and promoting disaster risk preparedness and mitigation measures (at local authorities and community levels).

6.2.3. Specific to SDC

DWUS, with support of ONGAWA, to make clear to VG and beneficiaries that community

cash contribution is for O&M cost-recovery funding.

DWUS, with support of ONGAWA, to push DWD and PBWO to make sure all COWSOs

created during MAMA get the water rights and by-laws completely approved by the end of the intervention.



DWUS, with support of ONGAWA, to make sure that VG have enough understanding

and information on constitution and responsibilities of COWSO in water systems management; also to coordinate with District Legal Officer during conflict resolution.

DWD and SDC to promote DWUS experience at regional and national level, seeking for

replication/scale-up.

DWUS working-plan needs to have an independent budget line at SDC level (O.C.

budget-line within DWE budget seems to be less realistic).

DWUS to facilitate COWSOs auditing process through District Auditor.

SDC to appoint a new DWUS member from DMO office and establish permanent replacement mechanisms.

DWD to continue giving technical advice to COWSO in relation to new PC to ensure water system capacity and water supply at DPs.

DWUS to plan for supportive supervisions of WASH management and to conduct refresher trainings in a yearly basis.

DWUS to follow-up & monitor COWSOs through established reporting system at least in

quarterly basis.

DWUS and District PHAST coordinator to assure recognition of CBOs and PHAST

Facilitators (PF) by VG in Zone III.

DWUS and District PHAST Coordinator to support PFs and CBOs in hygiene and

sanitation action plans at village level.

DMO and DEO to get more committed with collection and provision of impact

monitoring indicators for the intervention at health centres and primary schools.

6.2.4. Specific to village government and beneficiaries

VG to encourage community cash contribution by beneficiaries during project

implementation period explaining its importance.

In individual village water schemes COWSOs to plan implementation of water meters at

PC in order to guarantee sufficient and fair payments.

COWSO and Village Authorities to keep-on promoting the participation of women in

decision-making platforms. Although this recommendation is directly linked with the cultural-traditional habits it might have an effect in the daily life of men and women in the village and their roles, promoting the transition to the recognition of women in the

society with completely respect.

VG in collaboration with PHAST facilitators and Sanitation CBOs to prepare, keep and

regularly update systematic official registers of vulnerable families in their respective villages (particularly PLWHIV, elderly and handicapped), who might be prioritized and/or offered special assistance regarding these kind of projects (and water supply in

general).

COWSO and VG to monitor the need of safe water points for cattle and plan them with

assistance from DWE.



6.2.5. Specific to MOW and MOHSW

Ratio pupil/latrines for boys and girls not clear in the different policies, thus it needs to be reviewed specially for rural context primary schools in order to have realistic

standards.

Maximize DWUS experience in Same District and try to promote it countrywide.

6.2.6. Specific to EC

Strengthen reporting on partial/process indicators for this kind of projects, always with a result-oriented approach.

Give more regular feedback to the NGOs granted, on their quarterly monitoring reports.

Explore granting the NGOs with more flexibility when subcontracting local enterprises,

in order to allow them promoting small local business and development not always big companies well established and cornering most of the market.

Keep on allowing NGOs to adopt a flexible approach for the implementation of this kind of projects, in order to get adapted to unforeseen circumstances (as droughts, global

economic crisis, etc.) which may cause some amendments on the activities, their schedule and/or financial capacity to implement additional components due to scarcity of financial resources.

Consider funding post-project small stabilization phases (1-2 years) for this kind of

interventions.



7. ANNEXES

A. Terms of reference

B. CV evaluation team B.1. Nerea Lopetegui B.2. Valentine Mafula

B.3. Pedro Surja C. Detailed evaluation method: C.1. Evaluation matrix

C.2. Questionnaire for program managers C.3. Questionnaire for families

D. Logical framework matrices D.1. Original program logframe matrix D.2. Reformulated logframe matrix

D.3. Logframe matrix proposed by ECODE E. Maps of project area F. List of key informants and activities of the evaluation

G. List of documents used for/during the evaluation H. Other technical annexes H.1. Evaluation community workshops’ data summary

H.2. Systematization data from questionnaires for families H.3. Zone i water-borne diseases (source: DMO-Same) I. The standard DAC format for evaluation report summaries

The annexes to the report are included in a separate pdf document.

EXTERNAL FINAL EVALUATION - ONGAWA...MAMA-Programme final evaluation Final Report (July 2013) 3...

Documents

Transcript of EXTERNAL FINAL EVALUATION - ONGAWA...MAMA-Programme final evaluation Final Report (July 2013) 3...