Document of The World Bankdocuments.worldbank.org/curated/en/... · Ministry of Transport,...

Document of The World Bank

Report No: ICR00003210

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-7142 IBRD-7978)

ON A

LOAN

IN THE AMOUNT OF US$ 80 MILLION

TO THE

REPUBLIC OF PERU

FOR THE

NATIONAL RURAL WATER SUPPLY AND SANITATION PROJECT - PRONASAR

June 2014

Sustainable Development Department Peru Country Management Unit Latin America and Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 2, 2014) Currency Unit = Peruvian Nuevo Sol

2.78 PEN = US$1

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

CIDA COSUDE CPS

Canadian International Development Agency Swiss Agency for Development and Cooperation (Agencia Suiza para el Desarrollo y la Cooperación) Country Partnership Strategy

DGSR Rural Division of the General Sanitation Directorate (Dirección General de Saneamiento Rural)

DRA ERR

Demand-Responsive Approach Economic Rate of Return

FONCODES IE INEI IPRs ISR

Social Compensation and Development Fund (Fondo de Compensación y Desarrollo Social) Impact Evaluation National Statistical Institute of Peru (Instituto Nacional de Estadística e Información) Independent Procurement Review Implementation Status Report

JASS M&E

Sanitation Board (Junta Administradora de Servicios de Saneamiento) Monitoring & Evaluation

MINSA MTHC

Ministry of Health (Ministerio de Salud) Ministry of Transport, Communications, Housing and Construction

MVCS Ministry of Housing, Construction and Sanitation (Ministerio de Vivienda Construcción y Saneamiento)

NGO NPV OES O&M OT OVC

Non-Governmental Organization Net Present Value Specialized Operator (Operador Especializado) Operation and Maintenance Supervisor Operators (Operadores Supervisores) Citizen Monitoring Entities (Organismos de Vigilancia Ciudadana)

PAD PDO PMU PNSR

Project Appraisal Document Project Development Objective Project Management Unit (Unidad de Gestión del Proyecto - UGP) National Rural Water and Sanitation Program (Programa Nacional de Saneamiento Rural )

PRONASAR National Rural Water Supply and Sanitation Project (Proyecto Nacional de Agua y Saneamiento Rural)

RO RWSS SEDAPAL SNIP SO

Regional Operators Rural Water Supply & Sanitation Lima’s Water and Sewerage Service (Servicio de Agua Potable y Alcantarillado de Lima) Public Investment System (Sistema de Inversión Pública) Social Operators

STO TA

Social Technical Operators Technical Assistance

UE UTG

Implementing Unit (Unidad Ejecutora) Technical Management Unit (Unidad Técnica de Gestión)

UTM VMS

Municipal Technical Unit (Unidad Técnica Municipal) Vice Ministry of Sanitation

VMCS Vice Ministry of Construction and Sanitation (Viceministerio de Construcción y Saneamiento)

WSP WSS

Water and Sanitation Program Water Supply and Sanitation

Vice President: Jorge Familiar Country Director: Livia Benavides (Acting)

Sector Manager: Wambui Gichuri Project Team Leader: Christophe Prevost

ICR Team Leader: Ella Lazarte

PERU

NATIONAL RURAL WATER SUPPLY AND

SANITATION PROJECT - PRONASAR

CONTENTS Data Sheet A. Basic Information i B. Key Dates i C. Ratings Summary i D. Sector and Theme Codes ii E. Bank Staff ii F. Results Framework Analysis iii G. Ratings of Project Performance in ISRs vii H. Restructuring viii I. Disbursement Graph viii 1. Project Context, Development Objectives and Design................................................................1 2. Key Factors Affecting Implementation and Outcomes...............................................................7 3. Assessment of Outcomes...........................................................................................................13 4. Assessment of Risk to Development Outcome..........................................................................20 5. Assessment of Bank and Borrower Performance......................................................................21 6. Lessons Learned.........................................................................................................................24 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........................... 27 Annex 1. Project Costs and Financing...........................................................................................28 Annex 2. Outputs by Component...................................................................................................29 Annex 3. Economic and Financial Analysis..................................................................................33 Annex 4. Bank Lending and Implementation Support/Supervision Processes.............................35 Annex 5. Beneficiary Survey Results............................................................................................37 Annex 6. Stakeholder Workshop Report and Results................................................................... 41 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR......................................42 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders........................................43 Annex 9. List of Supporting Documents.......................................................................................44 MAPS

i

A. Basic Information

Country: Peru Project Name: National Rural Water Supply and Sanitation Project

Project ID: P065256, P117314 L/C/TF Number(s): IBRD-7142-PE, IBRD-7978-PE

ICR Date: ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF PERU

Original Total Commitment:

USD 50.00M Disbursed Amount: USD 74.99M

Revised Amount: USD 80.00M

Environmental Category: B-Partial Assessment

Implementing Agencies: Ministry of Housing, Construction and Sanitation

Cofinanciers and Other External Partners: Canadian International Development Agency (CIDA)

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 11/21/2001 Effectiveness: 03/12/2003

Appraisal: 04/29/2002 Restructuring(s): - 05/05/2006; 11/20/2008; 12/13/2010 06/11/2013

Approval: 08/29/2002 Mid-term Review: 09/2005

Closing: 12/31/2008 06/30/2013

C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Unsatisfactory

Risk to Development Outcome: High

Bank Performance: Moderately Unsatisfactory

Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately UnsatisfactoryImplementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:

Moderately UnsatisfactoryOverall Borrower Performance:

Moderately Unsatisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments (if

any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA): Satisfactory

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory1

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Sanitation 45% 48%

Water Supply 45% 48%

Subnational government administration 6% -

Central Government administration 4% 4%

Theme Code (as % of total Bank financing)

Rural policies and institutions 23% 23%

Participation and civic engagement 22% 22%

Rural services and infrastructure 22% 22%

Water resource management 22% 22%

Other Private sector development 11% 11% E. Bank Staff

Positions At ICR At Approval

Vice President: Jorge Familiar David de Ferranti

Country Director: Livia Benavides (Acting) Isabel Guerrero

Sector Manager: Wambui Gichuri Danny Leipziger

Project Team Leader: Christophe Prevost Alexander Bakalian

ICR Team Leader: Ella Lazarte

ICR Primary Author: Ella Lazarte

1 The last ISR Rating, dated 7/22/13 was Moderately Unsatisfactory, but the system-generated rating was Moderately Satisfactory.

iii

F. Results Framework Analysis Project Development Objectives (PDO) Project Appraisal Document (PAD): The development objective of the National Rural Water Supply and Sanitation Project (PRONASAR) is to increase the sustainable use of new and rehabilitated water supply and sanitation facilities in rural areas and small towns while emphasizing improvement in hygiene practices and training in operation and maintenance.2 Loan Agreement: The objective of the Project is to increase the sustainable use of water supply and sanitation facilities in rural areas and small towns in the territory of the Borrower while emphasizing improvement in hygiene practices and training in operation and maintenance.3 There is some difference in wording of the objective in the PAD and the Loan Agreement. The words “new and rehabilitated” were not included and “in the territory of the Borrower” was added in the loan agreement for the Additional Financing in December 2010. However, this difference in wording does not change the essence of the project or how the project is being assessed. The objectives stated in the loan agreement will form the basis of the evaluation. Revised Project Development Objectives (as approved by original approving authority) Not applicable. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Number of people with access to improved water sources under the project Value quantitative or Qualitative)

0 685,000 256,400 211,323

Date achieved 08/06/2002 05/2006 12/2013 06/30/13 Comments (incl. % achievement)

The target was partially achieved: 82% achievement of the revised target number of people with improved water sources.

Indicator 2: Number of people with access to sanitation services in the communities covered under Component 1 (rural areas)

Value quantitative or

0 616,500 242,700 180,607

2 Project Appraisal Document, Report No. 24173, August 6, 2002. 3 Loan Agreement (Additional Financing for the National Rural Water Supply and Sanitation Project – PRONASAR) between Republic of Peru and International Bank for Reconstruction and Development, December 14, 2010.

iv

Qualitative) Date achieved 08/06/2002 05/2006 12/2013 06/30/13 Comments (incl. % achievement)

The target was partially achieved: 74% achievement of the revised target number of people with access to sanitation services.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from approval documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target Years

Indicator 1 : Number of new or rehabilitated systems in rural areas whose water samples show residual chlorine greater than or equal to 0.3 ppm in the farthest network point of the source

Value (quantitative or Qualitative)

0 1,153 89 N/A

Date achieved 08/06/2002 05/05/2006 12/31/2013 Comments (incl. % achievement)

This indicator was not tracked.

Indicator 2 : Number of communities in rural areas with systems providing continuous services for at least 12 hours per day


0 1,153 489 357

Date achieved 08/06/2002 05/05/2006 12/31/2013 08/31/2012 Comments (incl. % achievement)

The target was partially achieved: 73% achievement of the revised target number of communities. The partial achievement is due to the lack of information on how many additional systems were completed since August 2012.

Indicator 3 : Number of JASS operating and maintaining the systems with updated records of users showing monthly household payments


0 1,153 489 296


The target was partially achieved: 61% achievement of the revised number of JASS. Note that this figure is consistent with the endline assessment in 2014 which reports that 58.3% of JASS have updated records of users showing monthly household payments.

Indicator 4 : Number of district municipalities with updated inspection records showing at least one inspection of each of the JASSs under its jurisdiction


0 292 175 76

Date achieved 08/06/2002 05/05/2006 12/31/2013 04/2014 Comments The target was partially achieved: 43% achievement of the revised target. From 2005

v

(incl. % achievement)

and 2014, the percentage of municipalities showing at least one inspection of each of the JASSs under their jurisdiction decreased from 47% to 26%.

Indicator 5: Number of communities with increasing number of households demonstrating adequate handwashing practices


0

695 localities with 60% of households with adequate handwashing practices



Date achieved 08/06/2002 05/05/2006 12/31/2013 04/2014 Comments (incl. % achievement)

The target was partially achieved: 76% achievement of the revised number of communities with increasing number of households demonstrating adequate handwashing practices.

Indicator 6 : Number of District Municipalities that have executed Small Towns Delegation Agreements


0 25 12 15


The target was fully achieved and exceeded: 125% achievement in the revised number of district municipalities that have executed small towns delegation agreements.

Indicator 7 : Number of specialized operators with operational income greater or equal 110% its operational expenditure


0 25 7 0


The target was not achieved.

Indicator 8 : Number of new or rehabilitated systems in small towns whose water samples show residual chlorine than or equal to 0.3 ppm in the farthest network point of the source


0 25 12 7


The target was partially achieved: 60% achievement.

Indicator 9 : Number of communities in small towns with systems providing continuous services for at least 16 hours per day


25 12 8

Date achieved 08/06/2002 05/05/2006 12/31/2013 08/31/2012

vi

Comments (incl. % achievement)

The target was partially achieved: 70% achievement.

Indicator 10 : Number of systems in small towns issuing at least 80% of water bills based on micrometering


0 25 0 1

Date achieved 08/06/2002 05/05/2006 12/31/2013 05/2013 Comments (incl. % achievement)

The target was fully achieved and exceeded since the revised target was “0” and one city continued to use micrometering in billing its customers.

Indicator 11 : Water and sanitation norms and technical design standards for rural areas. Issued through Ministerial Resolution adopted by MVCS


0 4 3 4

Date achieved 08/06/2002 05/05/2006 12/31/2013 08/31/2012


The target was exceeded: 133% achievement. (i) OS.070 Potable Water Networks; (ii) Technical Option in Rural Areas and Marginal Urban Areas (UBS) RD. Nº050-2008-VIVIENDA; and (iii) Contracting Specialized Operators RM. Nº270-20090-VIVIENDA.

Indicator 12 : A sector information system (including information from SEDAPAL) and at least from 90% of the district municipalities under components 1 and 2 of the Project


No Yes Yes Yes


The target was achieved. The Information System for Water and Sanitation (SIAS –Sistema de Información de Agua y Saneamiento) has been designed and implemented, including the database of the 514 sub-projects in the first and second call for bidding.

Indicator 13 : Sanitation policy for rural areas and small towns. Issued through Ministerial Resolution adopted by MVCS


0 3 3

0

Date achieved 08/06/2002 05/05/2006 12/31/2013 08/31/2012


This was not achieved. A ten-year strategic plan (2002-2011) adopting policies that aligned with PRONASAR (sustainability, enhanced role of district municipalities in WSS, demand-responsive) was developed. In 2012, policies became more supply-driven.

vii

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements

(USD millions) 1 10/31/2002 Satisfactory Satisfactory 0 2 05/30/2003 Satisfactory Satisfactory 0.5 3 12/05/2003 Satisfactory Satisfactory 0.75 4 06/03/2004 Satisfactory Satisfactory 1.35 5 12/01/2004 Satisfactory Unsatisfactory 2.38 6 04/22/2005 Moderately Satisfactory Unsatisfactory 2.93 7 01/03/2006 Moderately Unsatisfactory Unsatisfactory 5.77 8 06/01/2006 Moderately Satisfactory Unsatisfactory 7.38 9 11/20/2006 Satisfactory Satisfactory 8.29

10 05/02/2007 Moderately Satisfactory Satisfactory 9.36 11 12/15/2007 Moderately Satisfactory Satisfactory 14.08 12 06/13/2008 Satisfactory Satisfactory 17.8313 11/26/2008 Satisfactory Satisfactory 21.5014 05/22/2009 Moderately Satisfactory Moderately Satisfactory 24.0215 11/18/2009 Moderately Satisfactory Moderately Satisfactory 28.5216 05/14/2010 Moderately Satisfactory Moderately Satisfactory 30.8417 02/21/2011 Moderately Satisfactory Moderately Satisfactory 35.3618 09/10/11 Moderately Satisfactory Moderately Satisfactory 39.5019 04/06/2012 Satisfactory Satisfactory 45.2920 11/19/2012 Moderately Satisfactory Moderately Satisfactory 53.8821 07/22/2013 Moderately Unsatisfactory Moderately Unsatisfactory 74.99

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

05/05/2006 N/A U U 7.38

Transfer implementation responsibility from FONCODES to UGP-PRONASAR and revise targets in results framework.

11/20/2008 N/A S S 21.5 Extend original project’s closing date to December 31, 2010

12/13/2010 N/A MS MS 35.36

Support implementation of additional financing for US$ 30 million to address shortage of funds; revise targets in the results framework.

06/11/2013 N/A MU MU 74.36 Close the project as the project design no longer aligned to Government of Peru's policies in

viii

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP the rural water sector; Project closed to allow all rural water and sanitation interventions in Peru to be aligned with the National Rural Sanitation Program.

I. Disbursement Profile

1

PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.

1.1 Context at Appraisal 1. Low water and sanitation coverage in rural areas. When the project was approved in 2002, Peru's total population was estimated at 26.3 million of which approximately 34% lived in rural areas defined as population centers with less than 2,000 people.4 About 62% of Peru's rural population had adequate water supply services, defined as access to either piped systems feeding household connections, yard taps or public standposts or to point sources such as wells. Approximately 30% of Peru's rural population has access to either on-site or (occasionally) piped sanitation systems.

2. Rapid increase in investments in the rural WSS sector in the 1990s but lack of sustainability. Between 1992 and 1997 total investments in the rural WSS sector increased from US$15 to US$88 million, mostly through the National Fund of Cooperation for Development (Fondo Nacional de Cooperación para el Desarrollo-FONCODES) and non-governmental organizations (NGOs), but decreased significantly by the beginning of this decade due to the closing of external loans and FONCODES’ loss of prominence. FONCODES represented more than 75 percent of all rural WSS investments between 1992 and 2000, and they reported to have built, improved and/or rehabilitated approximately 13,000 rural WSS systems. However, baseline studies carried out in 2002 as part of the preparation of the Original Project showed that around 80 percent of existing rural WSS systems (including those which had been recently built) required additional investments for rehabilitation and 20 percent were not operational, indicating a significant lack of sustainability of investments. 3. PRONASAR as a response to sector challenges. Aside from poor sustainability of rural water systems, Peru faced a number of other key sector challenges as it sought to consolidate and expand upon the progress of the past decade: (a) lack of sector policy; (b) no provision for investment in rehabilitation and expansion; and (c) no long-term technical assistance to rural communities. It was within this context that the Government of Peru requested a loan to finance the National Rural Water and Sanitation Project or PRONASAR (Proyecto Nacional de Agua y Saneamiento), which aimed to address the key sector challenges mentioned above. The project would introduce a paradigm shift in the concept of implementing rural water supply systems where the construction is only a small part of the project cycle in which training in operation and maintenance (O&M) and changes in hygiene habits would be given a much longer timeframe to be internalized by beneficiaries than had previously been the case. At the core of PRONASAR was the demand-responsive approach where the users determined if they wanted to participate and selected their preferred level of service based on their willingness-to-pay and assumed long-term responsibility for managing the WSS services.

4 Project Appraisal Document (2002). Data derived from the Census 2000 data from the National Statistical Institute of Peru (Instituto Nacional de Estadística e Información - INEI).

2

4. Importance of decentralization in increasing sustainability of rural WSS services. The emphasis on the role of district municipalities in planning, monitoring of performance, and provision of long-term technical assistance to communities was also a core element of the project. The project aimed to contribute to the ongoing decentralization process in Peru which began in the late 1990s.

5. The Original Project was identified in the Peru Country Assistance Strategy (CAS) of 1997 as a means to contribute to economic growth and poverty reduction through the expansion of sustainable services in rural areas.5

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 6. The development objective of PRONASAR was to increase the sustainable use of new and rehabilitated water supply and sanitation facilities in rural areas and small towns while emphasizing improvement in hygiene practices and training in operation and maintenance. The key indicators to measure the achievement of the PDOs are outlined in the table below: Project Development Objectives Outcome/Impact Indicators Increase the sustainable use of WSS facilities in rural areas and small towns while emphasizing improvement in hygiene practices and training on operation and maintenance

Number of people with access to improved water and/or sanitation service; 850,000

Number of water systems providing adequate service at community level; 2,500 rehabilitated and 450 new ones

Number of district municipalities participating in planning, monitoring performance and providing technical assistance to communities: 400

Percentage of beneficiaries who have adopted improved hygiene practices: handwashing at critical times and safe disposal of children’s stools: 60% in each community

VMS policies implemented in all sub-projects: all rural water and sanitation sub-projects

Outputs by Component Output Indicators Water supply and sanitation systems rehabilitated in rural areas Water supply and sanitation systems built in rural areas JASS formed and trained in O&M and community members practicing improved hygiene habits in water use

Number of new or rehabilitated systems successfully rehabilitated within quality standards

Number of rehabilitated communities with 60% or more sanitation coverage.

Number of water systems built within quality standards; and

Number of new communities with 60% or more sanitation coverage.

% community members aware of hygiene implications of clean water

5 World Bank (1997). Peru Country Assistance Strategy. Report Number 16796-PE.

3

New management model (with private operators) in place in small towns effectively participating in WSS service delivery in small towns District municipalities strengthened to provide long term technical assistance in sustaining the water and sanitation services in rural villages Vice-Ministry of Sanitation (VMS) is performing its role as sector leader

% of successful delegation of contracts and implementation of WSS plans in pilot towns

Measures of financial sustainability of operators Number of municipalities trained and with capacity to

provide TA to communities VMS is effective at offering a portfolio of services that

comprises: ‒ 5-year plans to expand coverage and improve

services; ‒ Definition of norms and standards for the WSS

sector; and ‒ A sector information system that is regularly

updated and maintained.

1.3 Revised PDO and Key Indicators, and reasons/justification 7. Although the PDO did not change during the entire life of the project, the wording of the PDO was changed, as mentioned in Section H. The words “new and rehabilitated” (referring to water and sanitation facilities) were taken out in later documents, such as in the AF approved in 2006. In addition, key performance indicators were modified during project restructuring in 20106 to reflect the achievable goals within the remaining implementation time and budget, as outlined in the table below. 8. In assessing the project, the ICR uses these revised indicators as well as some additional indicators measured in the Impact Evaluation of PRONASAR. The indicators changed in 2010 when the AF was approved, but not in a drastic manner. For instance, instead of the indicator being the “number of water systems providing adequate service at community level; 2,500 rehabilitated and 450 new ones”, as stated in the PAD, it became the “number of communities under Component 1 with systems providing continuous services for at least 12 hours per day, thus adding more specificity to the definition of “adequate service”. Project Development Objectives Key Performance Indicators Increase the sustainable use of WSS facilities in rural areas and small towns while emphasizing improvement in hygiene practices and training on operation and maintenance

Number of people with access to improved water sources under the project

Number of people with access to sanitation services in the communities covered in rural areas (Component 1, as described in the next section)

Intermediate Results

6 World Bank (2010). Project Paper on a Proposed Additional Loan in the Amount of US$30 million to the Republic of Peru for the National Rural Water Supply and Sanitation Project. Report No: 56770-PE: November 4, 2010. The PDO remained the same, but it should be noted that the specific objectives which were explicitly outlined in the PDO section of the PAD had been left off in this document. Instead, the project components were included when discussing the PDO.

4

1. Increase access and sustainable use of WSS facilities in rural areas while emphasizing improvement in hygiene practices and training on operation and maintenance

Number of new or rehabilitated systems under Component 1 whose water samples show residual chlorine greater or equal than 0.3 ppm in the farthest network point of the source

Number of communities under Component 1 with systems providing continuous services for at least 12 hours per day

Number of JASS operating and maintaining the systems with updated records of users showing monthly household payments

Number of district municipalities with updated inspection records showing at least one inspection of each of the JASSs under its jurisdiction

Number of communities with increasing number of households demonstrating adequate handwashing practices

2. Improving sustainable management of WSS facilities in small towns through the establishment of Specialized Operators and training on operation and maintenance

Number of District Municipalities that have executed Small Towns Delegations Agreement

Number of OEs with operational income greater or equal 110% its operational expenditure

Number of new or rehabilitated systems under component 2 whose water samples show residual chlorine greater or equal than 0.3 ppm in the farthest network point of the source

Number of communities in small towns (Component 2, as described in the next section) with systems providing continuous services for at least 16 hours per day

Number of systems under Component 2 of the Project issuing at least 80% of water bills based on micrometering

3. Support the National Sanitation Directorate and the Vice-Ministry of Sanitation in the normative and policy making role

A sector information system (including information from SEDAPAL) and at least from 90% of the district municipalities under components 1 and 2 of the Project

Sanitation policy for rural areas and small towns. Issued through Ministerial Resolution adopted by MVCS

Water and sanitation norms and standard technical designs for rural areas and small towns

1.4 Main Beneficiaries 9. At appraisal, the project aimed to support the implementation of a demand-responsive and sustainable basic water and/or sanitation services project for approximately 850,000 people in 2,950 rural communities through the construction and rehabilitation of water points, piped systems and sanitation facilities as well as training, operation and management of water and sanitation facilities and extensive hygiene education. Other beneficiaries included 400 municipal governments which were expected to provide technical assistance to the communities to manage their system better and the Vice Ministry of Sanitation (VMS), expected to oversee sector performance. More specifically, it was one of the four main divisions of the VMS, the General Directorate of Sanitation (Dirección General de Saneamiento Rural – DGSR), which was one of the main intended beneficiaries. In addition, project benefits defined at appraisal included the temporary and permanent employment generated by the various activities of the project through construction jobs (mainly masons and unskilled labor) as well as water system operators and health promoters.

5

10. The number of main beneficiaries changed over the course of the project and at the time of the approval of an Additional Financing (AF) in 2010, which supported a cost-overrun7. The target of the number of direct beneficiaries with access to improved water sources was revised downwards from 685,000 in 2006 to 256,400 and beneficiaries with access to sanitation services in rural areas was revised from 616,500 to 242,700 within the same period.8 The drastic decrease in the number of beneficiaries is a result of cost overruns explained in Section 1.7. 11. The project consisted of the following four components: 12. Component 1: Rural Water Supply, Sanitation, Training and Hygiene Education. Component 1 comprised of two sub-components, rehabilitation of existing systems and construction of new ones, under the responsibility of FONCODES as the implementation agency. This component aimed to build on the past experience of FONCODES in building infrastructure (“hardware”) and improve on it by emphasizing “software” - the technical assistance to the communities, establishing new and strengthening existing water user associations (Junta Administradora de Servicios de Saneamiento – JASS) and district municipalities through regional operators (RO) and social operators (SO) to be hired by FONCODES. 13. Component 2: Water Supply and Sanitation in Small Towns. Under this component the project provided both: (i) technical assistance to district municipalities interested in delegating service provision to private operators or autonomous entities (cooperatives, municipal companies etc.); and (ii) water supply system investment finance for district municipalities that have signed delegation contracts. Component 2 had been designed as a follow-up to a pilot project to be implemented by the Water and Sanitation Program (WSP)9 with CIDA funds (about US$4 million) in 8 municipalities. 14. Component 3: Capacity Building. Under this component, consultants were hired to strengthen the DGSR in its role as sector leader (e.g. by preparing policy and sector studies), train its staff to assume their assigned functions, finance the training, orientation and accreditation of the project's implementing agencies (including municipalities, regional operators, NGOs, contractors, artisans, etc.), carry out pilot studies on source protection and the implementation of the environmental guidelines, etc. 15. Component 4: Project Management, Monitoring, Evaluation and Supervision. This component financed the activities of the Project Management Unit (PMU), including equipment & software, contractual staff, external audits, the set-up and maintenance of financial management and project monitoring & evaluation (M&E) systems, vehicles for the FONCODES field offices, as well as special consultancies.

7 See Section 1.7 for more discussion on the cost overrun. 8 The targets in 2006 are revised values which were defined during the restructuring in the same year. The expected number of beneficiaries at appraisal was 850,000 for improved water and/or sanitation services, as defined in the Project Description Section and Annex 1: Project Design Summary in the PAD (Report Number 24173). 9 The pilot project was implemented by the Lima office WSP, a World Bank-executed international partnership to help the poor gain sustained access to water and sanitation services.

6

1.5 Revised Components 16. The components of the project at appraisal were not revised.

1.6 Other significant changes 17. In 2006, the Project was restructured mainly to address issues related to the inability of FONCODES – originally the main implementing agency of Component 1 – to change its methodology and implement the Project’s designed demand-driven and participatory approach to building rural WSS systems. Under this Level 2 restructuring, the implementation responsibility was consolidated under the Project Management Unit within PRONASAR (UGP-PRONASAR) operating within the Ministry of Housing, Construction and Sanitation. The restructuring also included a reallocation of funds between components and a review of project indicators, though the Project’s development objectives and components remained unchanged. While the components remained unchanged, targets were revised, as mentioned in Section 1.4 above. The Project restructuring, with an amendment and restatement of the Loan Agreement, was signed on May 5, 2006. 18. In November 2008, a second restructuring for Loan No. 7142-PE was approved to extend the original Loan closing date to December 31, 2010 in order to address delays in implementation due to problems encountered in the calls for bids and the lengthy administrative process involved in implementing each of the sub-projects (defined as a project that entailed either construction or rehabilitation of a new system and/or construction of a basic sanitation facility, coupled with social mobilization and support) in the communities. 19. In 2010, a third restructuring for Loan No. 7142-PE was approved in order to support the implementation of the AF in the amount of US$ 30 million (Loan 7978-PE). The AF was approved to complement the original loan and help address the shortage of funds which emerged due to cost overruns from: (i) underestimation of original costs at appraisal; (ii) changes in implementation approach; and (iii) inflation and the impact of the increase in the value of the Peruvian Sol given the implementation delays.10 First, the lower original costs at the time of preparation were largely based on similar experiences implemented by NGOs at a small scale in Peru, which were not representative of the national market and of the full scope of implementation of project activities. Second, the changes in implementation approach for Component 1 had cost implications since the transfer of responsibility of implementation of this component from FONCODES to the Project Management Unit (Unidad de Gestión del Proyecto - UGP-PRONASAR) resulted in additional direct supervision costs. Instead of the FONCODES teams carrying out the construction works, UGP-PRONASAR had to contract firms to monitor the work of the construction firms. Another change in implementation approach included the expansion of the technology options for sanitation from dry pit latrines to pour flush latrines and ecological toilets. These first two factors alone accounted for 80% of the cost increase. Third, accumulated inflation of about 16% led to a US$13.8 million increase in project costs and the appreciation of the Peruvian sol led to a decrease of US$8.5 million in available resources. 10 See Annex 3 of the project paper for Additional Financing in 2010 (Report Number 56770-PE) for a comprehensive analysis of the cost overrun.

7

20. To allow for the execution of the planned activities financed by the AF and complete the implementation of the Original Project, the closing date was extended to December 31, 2013. The financing per component is shown in the table in Annex 1. 21. Finally, in 2013, a fourth restructuring occurred due to Government policy changes which resulted in the Project design no longer being aligned with the Government’s policies in the rural water sector. In 2012, Government sector policy orientation changed following the creation of the National Rural Water and Sanitation Program (Programa Nacional de Saneamiento Rural – PNSR). The newly established PNSR abandoned some key PRONASAR principles aimed at sustainability, including the community demand-responsive approach and community contribution towards capital cost as these principles were perceived as limiting inclusiveness (based on the argument that only communities able to contribute in cash or in labor could benefit from water and sanitation services, thereby excluding the poorest communities and also delaying progress in construction of systems).11 22. This last restructuring consisted of: (i) Advancing the closing of the Project (Loan No. 7142 and Loan No. 7978) six months earlier, from December 31, 2013 to June 30, 2013, to allow all rural water and sanitation interventions in Peru to be aligned with the strategy of the National Rural Sanitation Program; and ii) Modifying the financing percentages for Loan No. 7978 to fully utilize the loan funds before the new closing date. The Government committed to finance any remaining works and activities until finalization with own resources after the project’s new closing date.

KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.

2.1 Project Preparation, Design, and Quality at Entry 23. Solid background analysis at preparation stage. The project preparation process benefitted from solid background analyses prepared by the Project Preparation Unit as well as from previous studies by WSP and other organizations. These studies, which looked at a sample of 66 water supply schemes in rural areas, showed that only 28.8 percent of the services in the communities visited were categorized as “sustainable”; 57.6 percent were in a state of slight deterioration; 10.6 percent in serious deterioration; and 3 percent had collapsed. Key factors for lack of sustainability in the systems include lack of community participation in the decision-making process and the lack of capacity of the communities in managing and maintaining them due partly to the lack of household contribution to cover costs. 24. In addition, the project design incorporated lessons from the FONCODES model of implementing rural water supply systems. Non-governmental organizations (NGOs) such as

11 The co-financing arrangements under Component 1 of PRONASAR were: (i) new systems - The project financed 70-80 percent of total project costs and the municipality and community financed the rest. The municipality was expected to contribute 10-20 percent in cash while the community was expected to contribute 10 percent of total project costs through unskilled labor participation; and (ii) rehabilitated systems - The project financed 60 percent of total project cost while the municipality and community financed the remaining 40 percent. The community was expected to provide 100 percent of the required unskilled labor.

8

CARE and Caritas, which were already implementing projects similar to those proposed under PRONASAR organized several stakeholder meetings in order to further scale up best practices from their work and inform the project. The project design also leveraged lessons from a small town pilot project financed by the Canadian International Development Agency (CIDA) and implemented by WSP. Furthermore, the lessons from the Bank’s Operation Evaluation Department report published in 2000 (“Rural Water Projects: Lessons from OED Evaluation”) fed into the design, highlighting the need for a strong local organization capable of maintaining water points or piped systems, the importance of institutional development and the need for offering different levels of service. 25. Project design was innovative and significant for sector advancement. The objectives and components were clear, significant, and responsive to development priorities since they sought to address key sector challenges such as sustainability and weak capacity at different levels, particularly the communities and municipalities. Innovations in approaches also characterized the initial project design such as: (i) the shift from a supply-driven approach, which was the prevalent mode of implementation of rural water supply and sanitation sub-projects, to community-driven demand-responsive approach; (ii) introduction of co-financing by the municipalities and communities; (iii) introduction of the concept of Regional Operators to be contracted to provide support for community mobilization and organization, engineering design and education activities in project communities; and (iv) piloting of alternative models of delegated management in small town water supply and sanitation, which included local private sector participation (“Specialized Operator”). 26. Realistic objectives, but overambitious targets given the nascent institutional development of the WSS sector. The targets which accompanied the project objectives were too ambitious and the project too complex at that time, making the project design rating moderately unsatisfactory. The project was too demanding on the Borrower and the implementing agencies, such as FONCODES and UGP-PRONASAR, created within the Vice-Ministry of Water and Sanitation, which was just moved from the Ministry of Transport, Communications, Housing, and Construction to the newly created Ministry of Housing, Construction and Water in mid-2002. The project preparation stage, which lasted about three years from 1999 to 2002, benefitted from adequate government commitment to tackle the issues of sustainability and lack of sector policy. During this period, the General Sanitation Directorate (Dirección General de Saneamiento" – DGS) was established as stipulated in a sector law which passed in August 3, 2000. First, the DGS was initially established in the Ministry of the Presidency and then subsequently moved under the Vice Ministry of Water and Sanitation in the Ministry of Transport, Communications, Housing and Construction (MTHC). As mentioned above, the government decided to move the Vice Ministry of Water and Sanitation (VMS) to the newly established Ministry of Housing, Construction and Water in mid-2002, after splitting MTHC into two ministries.

27. Risk assessment was not realistic, given all the innovations and new concepts. Some risks were more substantial than anticipated such as the “integration of physical investments with training in O&M, hygiene education and long-term follow up is not done”, which was rated modest. Given the novelty of the approach and the scope and depth of intervention needed in thousands of communities that the project hoped to benefit, this should have been rated as

9

substantial. A related risk which could have been identified then was related to the fact that there might not be enough capacity to implement the social and technical components of the project at the regional level, at scale, given that the pilot NGO experiences were localized and at a substantially small scale compared to what the project was trying to achieve. Furthermore, while the risk assessment at appraisal rightly points out important risks such as that of the national government not applying the DGS financial policies and norms, the mitigation measure could have been strengthened to include constant dialogue with key sector leaders and evidence to support that the policies and norms were working.

2.1 Implementation 28. Many implementation challenges arose during implementation, leading to significant changes in the project and significant delays.

29. Lack of alignment in sector policy and strategy: The policy and institutional context within which PRONASAR was prepared and implemented provided the first hurdle for the Project. Weak sector policy and institutions were identified during project preparation, which started in 1999. Policies for the rural water supply and sanitation (RWSS) sector were not yet consolidated, with various actors in the sector implementing their own policies. This continued throughout the life of the project. The sector still lacked coherence and organizations within the sector lacked consolidation. For instance, many municipalities who had initially expressed interest changed their decisions because of competing government strategies and programs that provided 100% financing for WSS infrastructure as opposed to PRONASAR’s main principle of co-financing by both the municipality and community (through unskilled labor). After the first project restructuring in 2006, the PRONASAR had around 500 sub-projects in contract under the demand-responsive approach strategy, which included participation by the municipalities and communities through co-financing and unskilled manual labor contribution, respectively. However, many mayors decided not to continue with the Project and were tempted to participate in other government programs that provided 100 percent financial subsidy. This lack of alignment between PRONASAR and national government policies affected both the rural areas and small towns components but the effect was less on the latter since it was supposed to be a smaller scale pilot initiative. 30. Inability of FONCODES to implement the component 1 of the project. The project sought to build on the experience of FONCODES in efficiently transferring funds and executing small works in rural areas by adding the social mobilization and support component. Unfortunately, FONCODES was not able to adjust to the proposed approach as anticipated and this resulted in severe project delays. Another reason why there was very little progress made in the project in the first three years was because of the difficulty in transferring resources from the MVCS to FONCODES, a process which required a Ministerial resolution. This added burden made the PRONASAR activities less of a priority for FONCODES execution teams, thus contributing to project delays.

31. By early 2006, after almost two years of unsatisfactory performance the Project was restructured mainly to address issues related to the consolidation of sector institutions and the inability of FONCODES – originally the main implementing agency of Component 1 – to

10

change its methodology and implement the Project’s demand-driven and participatory approach to building rural WSS systems. At this point, Component 1 also became the responsibility of the PMU or UGP-PRONASAR, which had been implementing the small towns component (Component 2) since the beginning. The restructuring helped improve the overall progress of the project; the rating changed from Unsatisfactory to Satisfactory, showing improvements in disbursements (jumping from US$ 0 to US$ 8.3 million from 2004 to 2006) and progress in the infrastructure development. Subsequent delays after the restructuring can be attributed to the poor performance of the firms delivering the feasibility and technical designs of the proposed systems, as project documents reported. However, an unpublished report by the Swiss Agency for Development and Cooperation (Agencia Suiza para el Desarrollo y la Cooperación - COSUDE) analyzing the implementation timeline of the PRONASAR sub-projects shows that the average time between the finalization of technical designs and the start of works was 734 days or 2 years.12 The report further suggested that this was determined by the restructuring process, which emphasized the hiring of Technical Social Operators (Operadores Técnicos Sociales – OTS) and Supervisor Operators (Operadores Supervisores – OT) and other human resource to implement in this manner, which required a No Objection from the Bank.

32. Delays in budget allocation and slow transfer process lasted throughout the life of the project. Lack of coordination between the different ministries and related agencies responsible for PRONASAR contributed to delays due to the budget allocation and transfer process. In June 2011, the pace of execution had slowed significantly as the project did not have a budget allocation corresponding to the Bank loans in the Ministry of Housing, Construction and Sanitation (MVCS). PRONASAR had to use additional emergency budget allocations from the MVCS, in order to commit a portion of the US$16 million worth of contracts that were ready to be signed. 33. Lack of capacity of private sector firms, NGOs, and national consultants to implement the sub-projects. The design of PRONASAR assumed the existence of a private and non-profit (NGO) sector that can implement the technical and social components at the regional level. However, it became apparent that the NGOs, for instance, could not respond to the rigid clauses of the contracts and the demand for a large number of sub-projects to be implemented in each department. In addition, there was lack of human resources (national consultants) to respond to the demand for the OTSs and the OTs, given the large volume of work needed to develop project profiles and technical designs, as required in the Public Investment System (Sistema de Inversión Pública - SNIP) and the large scale of a project like PRONASAR.

34. Excessive administrative and licensing regimes led to significant delays in the implementation of sub-projects. The project faced formidable bureaucratic challenges, which significantly contributed to the tripling of time it took for a sub-project to be finalized. These requirements are specific to Peru. For instance, the average time between the finalization and approval of the technical project profiles and the start of construction was 481 days. Overall, it took around 1,624 days or a little over four years for a sub-project to be completed and

12 See unpublished internal document obtained from COSUDE, the Swiss Development Cooperation Agency in Peru, titled Análisis de las fases de los proyectos del PRONASAR y la línea de tiempo de implementación (First and Second Call for Bids).

11

transferred and closed, according to official regulations (in the SNIP system).13 This was almost three times the amount of time originally planned, which was approximately a year and a half. Infrastructure projects funded by other donor agencies have also faced similar implementation challenges due to the heavy administrative burden involved in obtaining the necessary clearances and approvals.

2.2 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 35. M&E Design: A results framework with development objectives, outcome, and intermediate outcomes indicators was developed. While the values of the targets by indicator have changed during project restructuring, the indicators have remained the same. The key indicators are linked to the main objectives of the Project, as described in detail in Annex 2.

36. The Project’s M&E system also included an impact evaluation, with a baseline from 2005 and an endline in 2014 to determine whether the project has made significant impact in the area of health and hygiene behaviors and resulted in positive outcomes in coverage and sustainability. The baseline included control and treatment groups in 43 sites in 19 districts in the eight departments where the Project was initially implemented. The results framework indicators along with the impact evaluation allowed for a clear assessment of the contribution of the project activities to the outcomes expected.

37. M&E Implementation and Utilization: Project M&E was under the responsibility of the UGP and additional evaluations were contracted out to consultancies to measure project progress and outcomes. Given the difficulties in accessing most of the locations where the interventions were carried out because of their dispersed nature, the M&E of the execution of the sub-projects was carried out by the operators, PRONASAR, and the Bank. However, even though an M&E system was developed at the national level, it started to show signs of weakness, particularly towards the end of the project and little information could be obtained by the Bank on the status of the different sub-projects and the overall intermediate outcome results indicators.

2.3 Safeguard and Fiduciary Compliance

38. Safeguards. Overall, compliance to environmental and social safeguards was satisfactory throughout the project. Triggered safeguards include Environmental Assessment (OP/BP 4.01), Physical Cultural Resources (OP/BP 4.11) and Indigenous Peoples (OP/BP 4.10). An Environmental Impact Assessment was developed during preparation and properly disclosed as per Bank’s policy. The project as originally designed did not include a separate Indigenous Peoples Planning Framework or Plan given that most of the beneficiaries are indigenous peoples and thus provisions of the policy were mainstreamed into the design. An updated social assessment with emphasis on indigenous people and afro-descendants was prepared and incorporated in the Revised Operational Manual following the 2006 Project restructuring. New environmental guidelines for consulting firms were updated in 2009 and publicly disclosed in September 2010.

13 Ibid.

12

39. A review of safeguards implementation during preparation revealed that safeguards instruments had been appropriate to meet the Project’s need for compliance. Responsibility for the direct implementation of the safeguards in the field has been borne mostly by consulting firms. These consulting firms included in their proposals the required personnel for the intervention, including social and environmental specialists. Adherence to the social intervention methodology was required for issuance of partial payments to the consulting firms. This included prior informed consultation to the community, including the reporting of key technical and social indicators. The implementation unit based in Lima monitored, evaluated and supervised the process throughout and intervened where more detailed social, technical or environmental intervention was needed. The implementation unit relied on a group of decentralized technical and social specialists for managing the contracts with the consultant firms including safeguards compliance. The unit also obtained support from the environmental specialists at the Vice-Ministry of Construction and Sanitation.

40. Overall, financial management arrangements worked well during project implementation. PRONASAR benefited from the use of governmental financial management information system, SIAF, and counted on qualified and experienced staff. However, the project had complex institutional arrangements within the Ministry and UGP PRONASAR and experienced high staff turnover of financial management personnel. This caused some delays in the submission of project financial information and several reviews of roles and responsibilities of financial management tasks to record project transactions and carry out payments on a timely basis. In terms of project financial audits, they experienced delays due to General Controller’s complex processes; however, audit reports were found acceptable. The closing date was June 30, 2013 and the grace period ended on October 31, 2013. The government agreed with the Bank to complete some civil works with local counterpart funds until March 2014. Thus, the last financial audit of the project is still pending and is expected to be delivered by September 30, 2014.14

41. Procurement. The management of procurement processes by PRONASAR was satisfactory overall. Several ex-post procurement reviews were performed during the implementation and also IPRs (Independent Procurement Reviews) were conducted by an independent expert. The ex-post reports, IPR reports and the Supervision Mission Aide Memoires included recommendations that helped improve the management of procurement processes, including: (a) management of the procurement plan (PP), with a complete systematized information base for follow-up action (the Project’s PP was approved and administered through SEPA); (b) training of procurement staff; (c) improvement of filing and archiving of the procurement documentation; (d) introduction of appropriate provision for advance payments for works; (e) the adaptation of adequate bidding documents for NCB; (f) establishment of proper and reasonable time for the bidders to submit proposals; and (g) improvement of the time required for evaluating bids for works. These recommendations were included in the action plans of the ex-post reports which were carried out during project implementation.

14 As of June 25, 2014, the bidding process was still in progress in the works that were promised to be completed by the government with its own funds after the project closed and there is no other known progress to report in this ICR at this time.

13

2.4 Post-completion Operation/Next Phases 42. As mentioned earlier, the Project closed six months earlier than planned, in June 2013. By the team’s supervision mission in June 2013, there were 30 contracts still in progress and one contract with a firm that would provide consultancy services to the social component of 82 works. Before the closing date, 21 works were planned to be finalized, leaving 61 contracts for works to be completed after June 30, 2013. The finalization of these 61 works contracts was estimated to cost US$18 million, to be financed by local counterpart funds. These works were expected to be fully executed by March 2014, adding 36,873 beneficiaries to the Project.15

43. WSP has also been working with the Government in strengthening sector policies and institutions, including the rural water supply and sanitation sub-sector to improve coverage and to ensure sustainability of investments such as those made under PRONASAR, an activity which has potential to be linked to future Bank-supported program. 44. The findings from the endline assessment from April 2014 will also be further analyzed to provide additional insights on how to move forward, given the lessons learned from the last decade of implementation of the PRONASAR project. A stakeholder workshop will be organized to further discuss the Bank’s role in the rural water supply and sanitation sector in Peru.

OUTCOMES 3.

3.1 Relevance of Objectives, Design and Implementation 45. The objectives of the Project were clearly defined and their relevance is rated Substantial. The objectives addressed the issue of sustainability, which remains one of the biggest challenges in the water and sanitation sector. Even after 11 years since the Project’s inception, its objectives remain the same as the ones expressed in the Ministry of Housing, Construction, and Sanitation’s Medium-Term Plan 2013-2016 regarding the need to ensure that the rural population has opportunities to improve their health through adequate and sustainable provision (quality and appropriate technology) of water and sanitation services. It is worth noting that the focus on rural water and sanitation remains extremely relevant, particularly in line with the Bank’s twin goals of ending extreme poverty and boosting shared prosperity. As shown in the graphs below, access to WSS by the rural population is still low and this is particularly an issue for the poorest quintile, around 60% of which lacks access to improved water sources. Sanitation remains a huge challenge across all quintiles. 46. The project’s objectives and design are also closely aligned with the most recent Country Partnerships Strategy (CPS) developed in 2012,16 particularly on three of the four pillars focused on increasing access and quality of social services for the poor, connecting the poor to services and markets, and inclusive governance and public sector performance. The emphasis on the

15 Same comment as above (Footnote #11). 16 World Bank (2012). Country Partnership Strategy for the Republic of Peru for the Period of FY12-FY16. Report No. 66187-PE; February 1, 2012.

14

social and governance aspects to ensure sustainability needs to be continued as the sector develops more and addresses the pressing issue of providing WSS services to the rural poor.

3.2 Achievement of Project Development Objectives 47. The project development objectives were partially achieved and the ICR rates this as Modest. The assessment was made by referring to the level of achievement of target values for the outcome indicators and to other impacts and outcomes of the project directly related to the PDO, as provided by the impact evaluation study, M&E reports, and the UGP-PRONASAR assessment of the pilot project of the Component 2. Additional information on the findings of the beneficiary survey (the endline assessment undertaken in 2014) is in Annex 5 of this report. 48. Increasing access to WSS facilities in rural areas. This objective was partially achieved. The project helped in providing access to improved water services to an additional 211,323 people (82 percent of target) and to improved sanitation services to 180,607 people (74 percent of target). The project constructed 78 new systems and rehabilitated 282 water supply network systems. In addition, it provided 27,818 hygienic sanitation facilities and 34,299 household water connections in 380 rural communities spread over 9 regions and 51 provinces across the country.

49. The impact evaluation provides evidence that access to improved water and sanitation services did occur as a result of the project. For instance, the evaluation shows a significant positive increase of 22 percentage points in the probability of having access to continuous improved water source in the Project’s intervention areas compared to non-intervention areas (p<1%). In localities where new systems were built, the access rate jumped from 36.9 percent to 100 percent between 2005 and 2014 (p<1%). In areas where systems were rehabilitated, the coverage increased from 82.8 percent to 90 percent in intervention areas, though this finding was not statistically significant. There is a positive increase of 30 percentage points in the probability of having access to sanitation services in the intervention areas (p<1%). The table below shows a comparison between WSS coverage at the national level and in PRONASAR’s intervention areas between 2005 and 2014. Overall, access to improved water sources in the areas where PRONASAR implemented their sub-projects was much higher, 77.7 percent in 2005, compared to the national rural access figure, which was 33.4 percent. The sanitation figures were more similar at baseline. They provide further evidence that PRONASAR has had an impact in the increased access to sanitation, with the construction of improved latrines.

Main Source of Water 2005 2014

National (rural) PRONASAR Baseline

National (rural) PRONASAR Endline

House connection (Public Network) 29.1 11.6 51.7 20.0Yard tap (Public Network) 0.8 63.0 1.2 69.8Public standpipe 3.5 3.1 1.8 1.4Public or private well 10.7 6.2 5.9 6.5River, springs, etc. 53.2 14.4 32.2 2.2Other sources 2.7 1.6 7.2

15

Type of Sanitation Facility 2005 2014

National (rural) PRONASAR Baseline

National (rural) PRONASAR Endline

Piped Sewer System 6.2 15.8 11.9 19.0Septic Tank 23.5 30.0 -Pit Latrine 19.8 38.9 29.5 11.0Pour-flush latrine - 9.2 - 10.2Composting Latrine 0.3Ventilated dry pit latrine 45.7None 48.3 33.6 26.2 4.4Other 2.3 1.5 9.1Source: INEI (2012). Encuesta Nacional Demográfica y de Salud Familiar. 50. Increasing sustained use of water supply facilities. There was a marked increase in the percentage of communities with systems that are functioning without serious problems from 2005 to 2014 in intervention areas, showing increased sustainability. The percentage of localities with systems that are functioning without serious problems increased by 69 percent between 2005 and 2014 in intervention areas compared to 27 percent in non-intervention areas, showing increased sustainability of the WSS facilities as a result of PRONASAR. This is one of the key results and contributions of PRONASAR to the sector. In addition, systems provide continuous water services through household water connection for at least 12 hours per day in 73 percent of the communities. Furthermore, the endline study shows statistically significant results on the households’ use of improved water sources for consumption and cooking. Where PRONASAR financed new systems, the increase was much higher than where it financed rehabilitated systems: 61 percent and 8 percent increase in intervention areas, respectively, compared to 27 percent and 6.6 percent in non-intervention areas (p<1% for new systems; p<5% for rehabilitated systems). 51. Increasing sustained use of sanitation facilities. A smaller but also significant increase of 11 percentage points can be observed in the impact of the PRONASAR in the increased use of hygienic sanitation in its intervention areas compared to those that did not benefit from the Project (p<1%). There was a dramatic increase in the percentage of men, women, and children over three years of age who used latrines17 in the intervention areas compared to non-intervention areas. For both new and rehabilitated systems financed by the project, this percentage increase was 59 percent between 2005 and 2014, a significant difference from the non-intervention areas where the latrine usage rate only increased by 16 percent for new systems and 36 percent for new systems in the same time period (p<1%). 52. Strengthening local communities’ capacity to manage services. In many ways, the project helped strengthen local communities to manage, operate and maintain their water and sanitation service. For instance, there was a 38.1 percentage point increase in the percentage of JASS that are trained in managing, operating, and maintaining their water and sanitation services; a 32.6 percentage point increase in the percentage of JASS with a complete and fully-functioning equipment; and a 46.4 percentage point increase in the percentage of JASS trained in

17 These latrines are not necessarily defined as hygienic, which refers to absence of feces on the floor, seat or walls and absence of flies and bad odor.

16

the daily chlorination (both system and household). However, even though the JASSs were trained in this aspect, the percentage point increase (24.8 percent) of systems in which daily chlorination is done was lower. There was also dramatic reduction of 67.2 percentage points in the percentage of JASS that know the method of calculating family contributions to O&M costs. Even though this last indicator showed a negative reduction, it is clear that families in general are paying for their water services and pay around US$ 2-3 a month, around 10 times more than the average rural Peruvian family.18 53. Strengthening municipal - district and provincial - level capacity to plan and oversee water and sanitation services to rural communities in a sustainable and cost effective way. The key indicators related to capacity building of local governments show a negative effect. For instance, the percentage of municipalities that have incorporated and carry out their planning, supervision, monitoring, and evaluation functions in water and sanitation services in rural communities has decreased from 42.1 percent in 2005 to 31.58 in 2014. Another indicator that shows the effect (or lack thereof) of the project on district municipalities is the percentage of municipalities with at least one person who knows the management activities of the JASSs. The baseline figure was 15.79 percent and this did not increase in the endline, putting into question the long-term involvement of the local governments in the WSS services in the intervention areas. One of the key instruments for decentralization under PRONASAR was the ‘mesa de concertación’ outlined in the PAD, a consultation “space” for the prioritization of development projects in the districts. However, the endline shows that the percentage of districts with a ‘mesa de concertación’ or similar decreased from 47.37 percent to 42.11 percent. 54. Improving health and hygiene practices. The project had no impact on household health and hygiene practices compared to localities in non-intervention areas. The evaluation study shows no impact of the project on the percentage of children under five with diarrhea in the last 15 days and on the percentage of mothers with adequate hand washing practices compared to the control group, which decreased by 16.2 percentage points from baseline to endline for rehabilitated systems and 14.5 percentage points for new ones. The lack of health impact is not surprising finding given the current literature on impact evaluations of handwashing programs carried out by WSP under the Global Scaling Up Handwashing Project in Peru and elsewhere, which found no health impacts on children under five.19 The lack of impact on behavior change can possibly be attributed to the lack of intensity and frequency of the behavior change intervention. There is a need to further link the water and sanitation sector to the education and health sectors to maximize the effectiveness of interventions. The endline shows that there is already an increase in collaboration between the water and sanitation and education sectors: there was a 42.1 percent increase (from 10.53 percent in 20015 to 52.63 percent in 2014 in the percentage of district municipalities who collaborated with the Ministry of 18 Ministerio de Vivienda, Construcción y Saneamiento (2013). Programa Nacional de Saneamiento Rural. Plan de Mediano Plazo 2013-2016: February 13, 2013, p. 16. This medium-term plan used data from the 2011 baseline of social indicators and management of water and sanitation services in rural areas published by the Instituto de Estudios Peruanos. 19 See, for instance, Galiani, Sebastian, Gertler, P. and Orsola-Vidal, A. (2012). Promoting Handwashing Behavior in Peru: The Effect of Large-Scale Mass-Media and Community Level Interventions. World Bank Policy Research Working Paper Series Impact Evaluation Series No. 74, November 2012. However, it should be noted that the Global Scaling up Handwashing Project in general, helped change behavior in terms of increasing handwashing practices.

17

Education in terms of cash, manual labor or other activities related to water and sanitation, in the past year. 55. Improving sustainable management of WSS facilities in small towns through the establishment of Specialized Operators and training on operation and maintenance. This objective was not achieved and none of the five intermediate results targets were achieved. Almost three years after the small towns component of the project closed, the UGP carried out an assessment of the pilot project and found that none of the ten systems visited had operational revenues greater than or equal to 110 percent of operational expenditure, seven out of the ten systems provided continuous service for more than 16 hours a day and only one city out of 10 continued using micro-metering in billing its customers. 56. Supporting the National Sanitation Directorate and the Vice-Ministry of Sanitation in the normative and policy making role. This objective was partially achieved and the intermediate results were partially achieved. The strategy of a key policy document, the National Sanitation Plan (2006-2011), developed during President Alan Garcia’s term were similar to the strategy and policy proposed by PRONASAR towards the beginning of the last decade (2000). There was some progress in sector development with the creation of the National Sanitation Directorate and the Vice Ministry of Sanitation by the time the Project was approved in 2002 as well as the development of a ten-year strategic plan (2002-2011) adopting policies that aligned greatly with PRONASAR, such as community participation, private sector participation, and acceptance of an enhanced role of district municipalities for the provision of WSS. 57. For this reason, PRONASAR’s role in the evolution of the Peruvian water and sanitation sector should be considered as an important legacy. The whole premise of PRONASAR was ambitious and given the institutional context in which it was designed and implemented, it had made considerable progress in terms of introducing the concepts of demand-responsive approach, community participation in the decision making process, capacity building for operation and maintenance and enhance role of local governments. The concepts of sustainability, community participation and social mobilization have already been mainstreamed into key sector documents.20

58. It is important to point out that even though these concepts have been mainstreamed into key sector documents, there is much room for improvement in implementing them. The prevailing policy under the PNSR is now to favor a supply-driven infrastructure-focused approach, contradicting totally these core concepts introduced by the project. Government policy now promotes funding 100 percent of the work, regardless of funding source. The new policy also promoted the concept of “Baño Digno”, which expanded the menu of options for a basic sanitary unit and further increased the costs of sanitation provision since the “Baño Digno included a shower, toilet, sink, elevated tank, and septic tank. These policy changes triggered the

20 See “Plan Nacional de Saneamiento 2006 - 2015” DECRETO SUPREMO Nº 007-2006-VIVIENDA (http://www.vivienda.gob.pe/pnc/documentos/PMM/MARCO_NORMATIVO/DS_2006_007_VIVIENDA.pdf last accessed on June 18, 2014.) and the Medium-Term Plan 2013-216 (http://www.vivienda.gob.pe/direcciones/Documentos/RM-031-2013-VIVIENDA%20(Plan%20del%20PNSR).pdf last accessed on June 18, 2014) developed under Humala.

18

exceptional early closing of the project due to a lack of Project alignment with the new government policies.

3.3 Efficiency 59. Overall, the project’s efficiency is rated Modest. The average net present value and average economic rate of return at ICR are US$ -40,781 and 3 percent, respectively, showing low economic efficiency. Even though prices had gone up throughout the duration of the project and the additional financing was a solution to the cost overruns met by the project, the lower than expected economic efficiency resulted mainly from lower benefits than those originally estimated. 60. The measures of efficiency used at the appraisal stage for both the original loan and the additional financing were net present value (NPV) and the economic rate of return (ERR) for each sub-project. At the project’s original appraisal in 2002, a cost-benefit analysis was carried out for 11 potential sub-projects, which had an average rate of return of 71 percent and an average net present value of US$ 10,940. The original analysis did not attempt to calculate the overall rate of return for the entire project given the demand-driven nature of PRONASAR which gave the community multiple choices of capital investments. For the additional financing in 2010, a detailed cost-benefit analysis was undertaken and at that time, it was for a sample of 36 potentially eligible sub-projects (11 in the region of Apurimac, 3 in the region of Amazonas, and 22 in the region of Cusco) spanning the available technology choices for communities and using the same methodology as that employed in the appraisal of the Original Project. The size range of the systems analyzed was between 42 to 243 water connections. In spite of investment cost increases, each of these sub-projects is economically feasible, and they have an average net present value of US$161,894 and an average economic rate of return of 17 percent. This average rate of return is lower than the 71 percent average rate of return calculated at Project appraisal, and can be largely explained by the cost increases discussed in this paper. The calculation of an overall rate of return for the Program was not attempted, given that such calculation would be based on an assumed distribution of community choices of capital investments. Therefore, it was considered more important to establish that each of the possible choices would generate acceptable economic returns. 61. The endline study carried out in 2014 showed that the time savings, which was a key component in the calculation of Project benefits, was minimal, overall. The baseline showed that, for rehabilitated systems, 10 percent obtained water from rivers, springs, and lakes, in the intervention areas and while this number decreased to 2 percent, the number of minutes spent fetching water actually went up 17 percent, from 35 to 43 minutes. For new systems, there was a 61 percentage point decrease (63.9 percent to 3 percent from 2005-2014) in the percentage of households obtaining water from rivers, springs, and lakes and there was a 10 percent drop in the time spent fetching water. In addition, health savings, which were a significant factor in calculating the economic efficiency of the project at appraisal and under the AF, were not included in the calculations at ICR because the impact evaluation did not find any statistically significant improvement in health outcomes attributable to the project.

19

62. At completion, unit costs were also calculated to provide an idea of the project’s efficiency. The cost per beneficiary of the entire project, taking into account all four components, was US$ 484.21 The cost per beneficiary in rural areas (Component 1) is much higher than in small towns (Component 2), US$ 497 versus US$ 120, respectively, though this was to be expected due to the more dispersed nature of the rural intervention areas. Under Component 1, systems were built or rehabilitated in nine regions, 51 provinces, and 117 district municipalities.22 The cost does not compare as favorably to other similar rural water and sanitation programs in other countries: less than US$ 400 in Paraguay and less than US$ 200 in Haiti. Around 32% of the total cost per beneficiary in Component 1 comprised the technical assistance to municipalities and communities and capacity building at the national level. However, when one takes into account only the infrastructure cost, the cost drops dramatically to US$326, which is actually in line with the estimates elsewhere in Peru23. The fact that this cost includes a basic sanitation facility makes it more comparable with the rest of the country.

3.4 Justification of Overall Outcome Rating: Moderately Unsatisfactory 63. The ICR rates the overall outcome of the project as Moderately Unsatisfactory, given the significant shortcomings in the achievement of its objectives and its low efficiency. The targets were not fully achieved in terms of increased access and the improvement in hygiene practices (such as hand-washing at critical times) was not as significant as expected. However, it is important to point out that the larger objective of increasing the sustainable use of WSS facilities in rural areas and small towns was achieved, even though long-term sustainability remains to be seen.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development

64. The project targeted the rural poor who represented 60 percent of the rural population through the FONCODES poverty map of Peru. The poverty map classified Peru’s 1,821 districts by poverty level (extremely poor, very poor, poor, marginally acceptable and acceptable) using indicators such as percent of households without access to potable water and adequate sanitation, percent of households without access to health clinics, and level of inaccessibility by road. Even though the implementation agency changed for the rural WSS component from FONCODES to the UGP-PRONASAR in 2006, the project was implemented in some of the poorest departments in the country: Ayacucho, Huancavelica, Huanuco, and Pasco, with poverty rates above 50 percent a decade ago.

21 In calculating the cost per beneficiary, the author took into account the financing provided by the municipalities as well as the communities (unskilled labor participation). The latter was derived from author’s estimates based on available project documents. 22 The increased dispersion of the project was a response to government strategy and policies which required the project to include more locations. 23 The cost per capita estimates in rural concentrated areas in Peru, US$ 280 for rehabilitated systems and US$ 336 for new systems, were derived from an unpublished internal working document written by staff from WSP, COSUDE, and CARE-Peru. This paper from 2009, which focused on gaps in access to water and sanitation in Peru particularly in dispersed rural and native communities, proposed that the actual costs in rural dispersed areas would be more given the geographic dispersion of the communities served.

20

65. A few aspects of the project related to gender were expected to contribute to improving sustainability, health outcomes and hygiene behavior in the communities. However, the percentage of mothers with adequate hand-washing practices and of women in leadership positions within JASSs was not statistically significant.

66. Community participation throughout the project cycle was at the core of PRONASAR’s methodology. It has long been argued that strong community participation leads to more sustainable outcomes, particularly when communities are involved in project planning and implementation (choosing appropriate service levels and discussing both financial and labor contribution to the development of the WSS infrastructure). (b) Institutional Change/Strengthening

67. Institutional strengthening was a core element of the PRONASAR operations. As already mentioned in Section 3.2, the project contributed significantly to the evolution of the Peruvian water and sanitation sector in various ways. For instance, the continued dialogue on participation, sustainability and decentralization within the WSS sector could be partly attributed to the project’s core concepts.24 In addition, the creation of the PNSR, as a national program aimed at aligning policies and strategies and as the main vehicle to increase water and sanitation services in rural areas, has been largely inspired by the example of the PRONASAR.

(c) Other Unintended Outcomes and Impacts 68. No unintended outcomes and impacts were observed or measured.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 69. An endline was carried out for Component 1, which deals with rural WSS and the key findings have already been incorporated in the preceding sections. A more detailed summary of findings is presented in Annex 5.

ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 4. 70. The ICR assesses the risk to the development outcome as High given the current challenges of ensuring enough social support to the communities and JASS to ensure sustainability. The lack of firms and NGOs to perform the role of providing social support and supervision (Operadores Sociales Técnicos or OST and Operadores Supervisores or OS) particularly in the regions of Piura, Huancavelica and Cusco continued to be an issue towards the end of the project in 2013.25 Even though the endline data point to a positive significant impact in terms of sustainability (i.e. functional systems) of a sample of the sub-projects under PRONASAR, long term sustainability remains to be seen.

24 There is ongoing dialogue about these concepts, as evidenced by a recent request for technical assistance by the Government of Peru to WSP) and discussions with key sector specialists. 25 Ministerio de Vivienda Construcción y Saneamiento, Informe de Seguimiento Financiero (IFR), April 1-June 30, 2013.

21

71. While community participation was emphasized throughout the project, it is not clear how well the district municipalities will be involved in the post-construction phase. Government commitment to policies that encourage and support decentralization of water and sanitation functions and institutional arrangements to support such functions are of utmost importance to fully realize the development outcomes that PRONASAR had envisioned from the start. The endline survey of municipalities indicates that some work needs to be done to increase their involvement in facilitating the successful provision of water supply and sanitation services. Results have been encouraging - a 47 percent increase from 2005 to 2014 of the percentage of district municipalities with a complete list of active JASSs in their district and a 32 percent increase of district municipalities with information about coverage and quality of water and sanitation services. However, the percentage of municipalities who carry out their supervision functions decreased from 74 percent to 42 percent in the same time period. Although these district municipalities have been trained in management, operation and maintenance of WSS services, only 16 percent has at least one staff with knowledge of the main management activities of the JASS, the same percentage as it was in 2005. 72. Therefore, it would be important to help mitigate these risks by providing additional technical assistance to the Government of Peru, both at the national and regional/local levels in ensuring the sustainability of the newly rehabilitated and constructed WSS schemes, not just under PRONASAR but also under the National Rural Water and Sanitation Program and the investment programs funded by other donors.

ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5. 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 73. The performance of the Bank in ensuring quality at entry was Moderately Satisfactory. The project preparation was a lengthy process and the multi-disciplinary Bank team handled the political and institutional uncertainties of the sector at that time, adapting to the different changes in sector authorities but maintaining the important concepts of participation and social support throughout the project cycle, as well as institutional strengthening at the local level. The demand-responsive approach promoted by the Bank as a way to increase sustainability was timely and relevant in terms of the sector challenges at that time. In terms of strategic relevance, approach and the poverty, gender, and social development aspects, the project pushed the boundaries in the sector where supply-driven approach was still the norm (and continues to be in many places). The project aimed at improving the access in poorer areas in Peru and strengthening the capacity of local levels of government (district municipalities) and communities to ensure sustainability, a concept that is still as relevant today as it was then. 74. However, the ICR identified gaps in the project preparation process such as the overestimation of the capacity of the borrower, private sector firms, NGOs, and national consultants and consequently, the project outcomes, and the underestimation of costs and administrative and licensing regime battlements, as discussed in Section 2. The risk assessment

22

at entry could have been more realistic, given all the new concepts and innovations proposed that sought to transform the sector. Some risks were more substantial than anticipated such as the “integration of physical investments with training in O&M, hygiene education and long-term follow up is not done”, which was rated modest. Given the novelty of the approach and the scope and depth of intervention needed in thousands of communities that the project hoped to benefit, this should have been rated as substantial. A related risk which could have been identified then was related to the fact that there might not be enough capacity to implement the social and technical components of the project at the regional level, at scale, given that the pilot NGO experiences were localized and at a substantially small scale compared to what the project was trying to achieve. 75. Despite these gaps at entry, it is important highlight that the project was highly innovative when it was conceived and consisted of state-of-the-art features borrowed not just from experiences in Peru but also around the world in order to address the fundamental issue of sustainability. (b) Quality of Supervision Rating: Moderately Unsatisfactory 76. The quality of supervision was Moderately Unsatisfactory. On average, the Bank undertook three supervision missions per year and team members travelled to different parts of the country to supervise the implementation of the project. It also engaged the support of WSP staff to carry out some of the supervisory functions. The team was able to correctly detect and identify implementation problems and propose solutions accordingly. The Bank team also made key decisions to help move the project forward by restructuring and amending activities and targets, while keeping the same development objectives. However, it was not always possible to resolve the implementation problems due to the complexity of the project, the lack of flexibility, the fear that drastic changes would have even further delayed progress in implementation, and the different actors involved who were not always aligned to the project’s objectives. The project went through three different restructurings before the government eventually requested its early closure, yet the project components and methodology largely remained the same. Could the supervision team have missed these opportunities to restructure the project in a more meaningful and effective way to allow for the outcomes to be fully achieved? Could the Bank team have overestimated the capacity on the ground, particularly at the local level, to deliver on the results promised under PRONASAR? Given the complex and challenging nature of the project, it would have been helpful to provide additional, just-in-time technical assistance at all levels, to ensure that the social component and long-term capacity building would be addressed. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 77. The overall bank performance is rated Moderately Unsatisfactory. Even though the supervision was strong for the most part throughout the long life of the project, the Bank’s performance has much room for improvement. Continuous supervision and additional technical assistance on the ground and more flexibility could have helped improve the Bank’s performance. There were missed opportunities for reorienting the project in a deeper way,

23

especially given the length of time that it took and the underlying context within which it was operating. PRONASAR operated within a context of continuous sector transition. Policy and institutional reform should have been given more priority in order to ensure that the outcomes were achieved. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 78. The government performance is rated Moderately Satisfactory. A couple interlinked factors negatively affected the performance of the government such as the lack of timely resolution of implementation issues such as those related to long delays in appointment of key staff and budget transfer and the lack of policy and institutional support to help fully achieve all of the project objectives, such as reducing the bureaucratic process involved in implementing small WSS projects in rural areas. Furthermore, even though rural areas are now prioritized compared to when the project first began, there is still a predominant view of emphasizing infrastructure development (“hardware”) and not placing enough emphasis on the social aspects of infrastructure (“software”). The espousal of the demand-responsive approach by the government did not quite materialize after over a decade of experience with PRONASAR. However, the government tried the new and innovative concepts that PRONASAR promoted and while over time, the level of commitment changed depending on the leadership of the agencies and the political economic context, the request for additional financing showed the commitment to continue the project. For this reason, the ICR rates the government performance Moderately Satisfactory. (b) Implementing Agency - UGP PRONASAR Rating: Moderately Unsatisfactory 79. The performance of the implementing agency is rated Moderately Unsatisfactory mainly because of the following reasons: (i) implementation arrangements that relied too much on a large centralized team and not enough field presence; (ii) inadequate coordination and participation of stakeholders at the local level; and (iii) insufficient M&E to monitor progress and make mid-course corrections to improve results. The project relied on a large team based in Lima, which was not necessarily complemented with local support and presence to push the project forward. Relationship and coordination with local governments was an important aspect of the project as it aimed to strengthen their capacity in supervision, but supervision and monitoring was largely centralized. Furthermore, municipalities were not enabled to sign contracts, making them less accountable and ‘invested’ in the project. Though there were significant shortcomings, it is important to commend the work of key UGP staff who worked hard to push through different barriers to implementation. For instance, from 2009 to 2012, the UGP supported the demand from households and municipalities of including a shower in the definition of a basic sanitation unit, a relatively expensive option that lowered the economic viability of the project dramatically. In the end, the PRONASAR leadership succeeded in convincing the government to allow for such option. Another example is the way PRONASAR’s leadership maneuvered its way within the difficult policy and institutional environment which did not always align with the project’s objectives. To avoid competing for

24

the same municipalities to participate in the investment program, PRONASAR discussed and came into an agreement with the government about not intervening in the same places. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 80. The overall performance of the borrower is rated Moderately Unsatisfactory. The government did not show enough commitment to achieving the project’s development objectives, at times contradicting the core concepts introduced by PRONASAR, such as the demand-responsive approach and importance of strengthening the municipalities and the communities in the provision and management of WSS services. The readiness for implementation was also not evident throughout the life of the project. In the end, the government asked to close the project prematurely due to the lack of alignment of its strategy with other government programs, such as those contemplated by the PNSR. While the government had committed to funding and implementing pending activities, at the time of writing of this ICR, those works had not yet been completed.

LESSONS LEARNED 6. 81. Accelerating water and sanitation services requires a common strategy and the simplification of the country’s complex administrative processes. The PRONASAR co-financing strategy by the municipalities aimed at ensuring ownership and long term support of the JASS for the extension and rehabilitation the systems constructed by the project. However both cumbersome administrative procedures and supply of other state programs without co-financing, was a serious obstacle to the progress in project implementation. As an example, the emergence of strategies like "National Growth Strategy" affected the third call for bids since it expanded the project area, making it more expensive due to increased dispersion of sub-projects in order to meet the targeting policy of that district national strategy. Simplifying the National Public Investment system (Sistema Nacional de Inversión Pública – SNIP) to take into account smaller projects in dispersed rural areas would contribute significantly to infrastructure development for those without access. The two key priorities for the RWSS sector in the country should be to eliminate these administrative bottlenecks and more importantly, define a common strategy taking into account the lessons from PRONASAR and other projects. 82. A more decentralized approach to planning, execution and monitoring of projects needs to be emphasized to go to scale and ensure sustainability, supported by a strong center and continuous capacity building and institutional strengthening. It took the project 11 years to provide about less than a quarter million people with water supply service. The centralized planning, execution and monitoring approach used by PRONASAR implemented from Lima, using consultants and contractors mainly based in the capital, did not necessarily yield good performance in many areas of the project. Replicating the same approach would be rather expensive. It would probably have been more efficient and sustainable to build capacity at the Municipal District level to implement project activities and works. This lesson, however, may not be applicable to all regions because capacity at the various levels of the public sector varies across the country. A capacity assessment needs to be carried out to determine how much

25

to decentralized and what the central government can do to support institutional strengthening at the local levels. A good practice element of the small towns component (Component 2) was the creation of Technical Management Units in the municipalities, which has contributed significantly to improving the model. This unit served as a direct link between the local government and the operator; in places where these units were functioning, the specialized operator has been able to work well. Aside from the local government’s active role, it is also important to establish user committees to help with monitoring (Junta de Vigilancia). More research is needed on good practice cases in Component 1 to learn how good governance at the local levels can work to ensure sustainability of rural water supply schemes. 83. Municipal co-financing and community contribution requirements need to be better adapted to the local context. The policy of municipal co-financing and community contribution faced difficulties. Many municipalities did not have significant budgetary resources to participate in the project. The Bank did not accept the co-financing from any regional governments or other sources of funds. This led to redirect the proposed investment to other municipalities not included in PRONASAR. In addition, over time, rural communities became more and more reluctant to participate in the project due to the requirement to provide unskilled labor or local materials. Increased economic activity in rural areas has increased the opportunity cost of labor of the people, making it increasingly difficult for them to contribute in terms of labor. In some cases, it has slowed and even paralyzed the implementation of works not only because the people refused to participate, but because contractors, aware of this issue, either did not bid or withdrew from it, affecting overall project progress. There were also limits to community participation given community members’ work schedules which did not necessarily coincide with those established by the contractors. 84. Community participation in planning and operations underpins sustainability, but adequate financial incentives are necessary for continued participation. The endline shows that community participation in planning and operations underpins sustainability, as evidenced by the fact that the percentage of localities with systems that are functioning without serious problems increased by 69 percent between 2005 and 2014 in intervention areas compared to 27 percent in non-intervention areas. The demand-responsive approach, whereby communities make informed choices regarding their participation and service levels, is implementable. The project has shown that isolated communities, including the poorest - the project was implemented in some of the poorest departments in the country, with poverty rates above 50 percent - can demonstrate their demand and contribute to partial capital investment as well as manage the water supply system and fully or partially recover operation and maintenance costs. The endline also found that well-organized and trained people may assume the management, operation and maintenance, but to ensure sustainability they must be paid. In small towns, it was clear that extensive consultations with the communities, particularly the involvement of women, were instrumental in achieving more sustainable outcomes. For instance, the project found that women were more willing than men to pay an additional amount to the monthly contribution in order to cover the operations and maintenance of the water supply system, given the women’s primary responsibility in water collection and use. 85. It is important to think more holistically and involve other sectors in order to achieve the higher level impact of improved health benefits. A project alone cannot achieve

26

health benefits as evidenced by the fact the project has no impact on the percentage of children under five with diarrhea within the 15 days and on the percentage of mothers with adequate handwashing practices in the intervention area compared the no intervention area. Changing and sustaining behavior change requires much more than hygiene education and more intensive interventions at the level of the community; this lesson is actually consistent with recent impact evaluation studies carried out by WSP under the Global Scaling Up Handwashing Project in Peru and elsewhere, as already mentioned in Section 2.1. It is important to sustain handwashing activities and reinforce the messages after the project; doing this requires a multisectoral approach that includes not only the health and education sectors but also the private sector, to the extent possible. Continuous training of frontline workers and delivery of such training through local governments are key areas to focus on in the future in order to increase the changes of behavior to occur and be sustained.

27

COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING 7.AGENCIES/PARTNERS

(a) Borrower/Implementing Agencies (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

28

Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Millions)

Components Appraisal Estimate

Additional Financing Estimate

Actual/Latest Estimate

Percentage of

Appraisal 1. Water Supply and Sanitation in Rural Areas26

68.00 48.30 102.02 150%

2. Water Supply and Sanitation in Small Towns

4.00 0 6.6 165%

3. Capacity Building 2.00 0 1.1 55% 4. Project Management, Monitoring, Evaluation, Audits and Supervision

2.50 2.60 11.32 453%

Unallocated 3.00 0 0 0% Total Project Cost 79.5 50.9 121.04 152%

Front end fee 0.5 0 0.5 100% Total Financing Required 80.0 50.9 121.54 152%

(b) Financing (in USD Millions)

Source of Funds

Appraisal Estimate

Revised Estimate

Actual/Latest

Estimate

Percentage of Appraisal

Borrower 12.58 20.90 38.07 303% Canadian International Development Agency (CIDA)

5.00 - .53 11%

Communities 7.57 - 4.4627 59% International Bank for Reconstruction and Development

50.00 30.00 74.83 150%

Municipalities 4.85 - 2.79 58% Others28 .86

Total 80.00 50.9 121.54 152%

26 This includes the estimated amount of community contribution, as per the author’s calculations. 27 This figure was estimated from project documents which included investment cost data from the first and second call for bids. 28 Others refer to financing from the Regions and Ecuador, as per data provided to the World Bank by the PRONASAR team in June 2014.

29

Annex 2. Outputs by Component The project had four main components aimed at increasing the sustainable use of new and rehabilitated water supply and sanitation facilities in rural areas and small towns while emphasizing improvement in hygiene practices and training in operation and maintenance. There was a strong emphasis on a demand-responsive approach coupled with social support to local communities from planning to post-construction. The components along with their main achievements are described as follows: Component 1: Rural Water Supply, Sanitation, Training and Hygiene Education. Component 1 comprised of two sub-components, rehabilitation of existing systems and construction of new ones. It also included technical assistance to the establishment of new and strengthening of existing water user committees (JASS) to manage the systems, and hygiene education. An important part of the overall objective of PRONASAR is capacity building at the community level, a key project feature that was emphasized throughout the duration of the project by the Bank team. The outputs under this component were partially achieved, as shown in the table below. By the project’s closing on June 2013, there were 380 localities which benefitted from the water, sanitation, and hygiene interventions, out of the target of 489. Project documents (i.e. Aide Memoire, June 2013) mentioned that by the end of the project on June 30, 2013, around 211,323 people would have gained access to improved water sources (82 percent of revised targets) and 180,607 would have gained access to sanitation services (74 percent of revised targets). Overall, Component 2 was fully implemented in nine regions, 51 provinces, and 117 district municipalities, which means that systems were actually built or rehabilitated/expanded as opposed to just technical designs developed. The table below provides further detail on the key outputs of Component 1, such as the number of sub-projects, localities, beneficiaries, connections, and sanitation facilities. Completed Water and Sanitation Projects29

29 The number of beneficiaries in the table is an estimate by the PRONASAR team based on completed projects as of March 2013 and thus, does not exactly match the final numbers reported in the Aide Memoire. It is also a combined estimate for water and sanitation.

Department

Number of Projects Works Completed

Year(s)

Number of Benef.

Number of

Connections

Number of

Sanitation Facilities New Rehabilitated Total

Arequipa 11 13 24 2006-07 10,613 1,947 1,478

Ayacucho 17 95 112 2006-2009 47,281 11,094 8,078

Huancavelica 4 23 27 2008; 2010 12,429 2,631 2,117

Huanuco 7 26 33 2006-07/2009 12,823 2,300 2,211

Junín 12 56 68 2007-2009 44,555 6,470 6,191

Lima 4 43 47 2007-08; 2011 32,513 4,037 2,617

Pasco 4 11 15 2007-08; 2012 10,039 1,182 674

30

Source: UGP-PRONASAR In addition, 177 technical designs have been finalized (86% of target), out of which 35 have obtained approval (Resolución Jefetural) to move forward to the construction phase in the regions of Ancash, Apurimac, Cajamarca, Lambayeque, and San Martin. Only the design phase is covered by the Bank loan agreement in these five areas. Some of the challenges faced by the implementation agency were: Lack of operators with sufficient capacity to implement sub-projects in rural areas; Lack of operators to implement the social component in Piura, Huancavelica and Cusco Complex process of obtaining licenses and approval (CIRA); Modifications in the menu of options to incorporate the demand from the communities to

construct ‘complete bathrooms’, with a shower (Baños Completos) as opposed to simpler ones (Baños Comunes), particularly in Cusco and Piura. This process took time given that the economic viability of the projects were calculated without taking into account the showers which drove the prices up and the viability down. Different agencies had different rules and the UGP-PRONASAR worked hard to remove this impediment; and

Intense rains which affected construction of the WSS systems (interrupted road access for delivery of materials and inputs) in Huancavelica, Cusco, and Piura.

Component 2: Water Supply and Sanitation in Small Towns. Under this component the project provided both technical assistance to district municipalities interested in delegating service provision to private operators or autonomous entities (cooperatives, municipal companies etc.) and water supply system investment finance for district municipalities that have signed delegation contracts. Component 2 had been designed as a follow-up to a pilot project to be implemented by WSP with CIDA funds (about US$4 million) in 8 municipalities. In total, the project established strategic alliances with 30 localities situated in seven regions throughout the country. At the end of the Pilot Project, there were 16 municipalities which received financing to rehabilitate, improve, and/or expand their water and sanitation systems. The total investments in infrastructure amounted to US$ 2.9 and the estimated number of beneficiaries was 55,000 people. However, not all of these municipalities signed a contract with an operator, though other municipalities decided to go through the process of institutional change and reform even without financing from PRONASAR. The table below shows the different towns included under Component 2 and distinguishes between those which received financing (middle column titled “Con ejecución de obra”) and those which did not (third column). Furthremore, the table also shows which municipalities received financing and signed a delegated management contract with a specialized operator (Vice, Piura; Chuschi, Ayacucho; San Lorenzo, Junín; Río Negro, Junín; San Pedro de Cachora, Apurímac; Yuracyacu, San Martín; Pampa Cangallo, Ayacucho; and San Salvador, Cusco). Seven of the 16 municipalities

30 The number of localities was slightly higher, 380.

Piura 17 17 34 2011-12 23,238 4,064 3,878

Cusco 2 4 6 2013 2,638 574 574

TOTAL 78 288 36630 380 196,129 34,299 27,818

31

which received PRONASAR financing decided not to sign a contract with a specialized operator for various reasons ranging from security to local authorities’ refusing to sign it. However, an additional six municipalities decided to sign agreements with specialized operators even without the investment financing from PRONASAR, thus yielding a total of 15 small towns with a delegated management model and specialized operator in place. Small Towns in the PRONASAR Pilot Project31

The lower number of municipalities included in the Pilot Project was due to several factors that include the lack of political will to be part of a new project that did not receive total subsidy from the government; change in leadership at the municipal level with perspectives that did not align with the project; and interest only in the construction of WSS infrastructure and not the social components. Before the closing of the Project, the UGP carried out an assessment of the results of the Pilot Project and visited 10 of the 16 localities that had a Specialized Operator managing their water and sanitation services. The key performance indicators after three years since the Pilot Project’s closing are as follows: (i) only 7 of the 10 specialized operators have remained and the same number of operators provide continuous water supply service greater than 16 hours a day; (ii) none of the specialized operators are covering 110% of their operating costs; and (iii) only one small town continues billing according to household consumption as measured by meters.

31 The list of small towns under PRONASAR’s Component 2 comprises of those that received financing for rehabilitation, improvement and/or expansion of the water and sanitation system. For additional information, see the report on the pilot experience: Ministerio de Vivienda, Construcción y Saneamiento (2010). Proyecto Piloto de Agua y Saneamiento en Pequeñas Ciudades PROPECI: Un Modelo de Gestión Sostenible.

32

The model proposed under the Pilot Project was based on the interaction among four actors: the Technical Management Unit (Unidad Técnica de Gestión -UTG), the Citizen Monitoring Entities (Organismos de Vigilancia Ciudadana -OVC), Specialized Operator (Operador Especializado - OES) and the users. Despite the advances made in developing a new model for small towns WSS management, the tariff issue remains with the district mayors strongly influencing the process. As mentioned earlier, only one of the towns actually do micrometering. From the user perspective, they have accepted the management model and the role that the specialized operator plays. It is worth pointing out that while 93% of the users interviewed are satisfied with the services, only 54% attributed this to the specialized operator. However, 70% considers that the specialized operator should continue providing the service in the future. Component 3: Capacity Building. Under this component, consultants were hired to strengthen DGSR in its role as sector leader (e.g. by preparing policy and sector studies), train sector staff to assume their assigned functions, finance the training, orientation and accreditation of the project's implementing agencies (including municipalities, regional operators, NGOs, contractors, artisans, etc.), carry out pilot studies on source protection and the implementation of the environmental guidelines, etc. Studies carried out under this component contributed to sector norms and policies. For instance, consultancies about the design of the process and operations manual for investment programs shaped the organization of the Department of National Sanitation and Water for All Program. The three resolutions related to the sanitation policy for rural areas and small towns were adopted by MVCS were: (i) Investment Funds INVERSAN Ley Nº29061; (ii) Policy Incentives to Access Investment Financing in the Sanitation Sub-Sector RM. Nº681-2008-VIVIENDA; and (iv) Guidelines for Regulation of Sanitation Services in Small Towns Population Centers RM. Nº269-2009-VIVIENDA. However, in terms of policy, PRONASAR had limited impact in the end, given the new direction taken by the government at the time of Project closing, to focus on the supply-drive approach and less on the social component to help speed up infrastructure development. Component 4: Project Management, Monitoring, Evaluation and Supervision. This component financed the activities of the Project Management Unit (PMU), including equipment & software, contractual staff, external audits, the set-up and maintenance of financial management and project monitoring & evaluation (M&E) systems, vehicles for the FONCODES field offices, as well as special consultancies.

33

Annex 3. Economic and Financial Analysis The measures of efficiency used at the appraisal stage for both the original loan and the additional financing were net present value (NPV) and the economic rate of return (ERR) for each sub-project. A sample of 9 sub-projects covering 13 communities was selected for the economic analysis of the project for the ICR.32 These projects were selected from the list of communities where the impact evaluation was carried out in order to have as much actual data as possible instead of assumptions in the calculations. The average NPV was US$ -40,781 and the average ERR was 3%, a substantial difference from those calculated in 2010 for the Additional Financing and at the PAD stage. At the project’s original appraisal in 2002, a cost-benefit analysis was carried out for 11 potential sub-projects, which had an average rate of return of 71 percent and an average net present value of US$ 10,940. At the project’s original appraisal in 2002, a cost-benefit analysis was carried out for 11 potential sub-projects, which had an average rate of return of 71 percent and an average net present value of US$ 10,940. The original analysis did not attempt to calculate the overall rate of return for the entire project given the demand-driven nature of PRONASAR which gave the community multiple choices of capital investments. For the additional financing in 2010, a detailed cost-benefit analysis was undertaken and at that time, it was for a sample of 36 potentially eligible subprojects (11 in the region of Apurimac, 3 in the region of Amazonas, and 22 in the region of Cusco) spanning the available technology choices for communities and using the same methodology as that employed in the appraisal of the Original Project. The size range of the systems analyzed was between 42 to 243 connections. In spite of investment cost increases, each of these projects is economically feasible, and they have an average net present value of US$161,894 and an average economic rate of return of 17 percent. This average rate of return is lower than the 71 percent average rate of return calculated at Project appraisal, and can be largely explained by the cost increases discussed in this paper The two key drivers of this negative difference in efficiency between the two earlier calculations and at the ICR stage can be attributed to: (i) lack of health benefits of the project, as evidenced by the impact evaluation, which found that the impact on diarrhea for children under 5 was not found to be statistically significant; (ii) lack of savings on coping costs for inadequate water supply because more households were found to be boiling their water in 2014 (55%) than in the baseline in 2005 (0%). However, it should be noted that even though there was lack of savings on coping costs for water, it can also be taken as a good sign that people are now boiling their water before drinking, which means that community education campaigns might have been effective at instilling good habits to improve water supply at the household level. It is also not surprising to see more people boiling water since the data show that not all the JASSs were necessarily treating the water they were serving in an adequate and effective manner. Thus, the methodology for the economic analysis was slightly revised at ICR stage given these changes and also to take into account the on-site sanitation benefits, which were not included in the previous economic efficiency calculations, given the planned variety in household technology choices. In the end, over 90% of the households opted for a ventilated dry pit latrine, so this was taken into account in the efficiency analysis. 32 It was not possible to use the same sample from 2010 because the work in those communities had not been completed by the time of project closing and the expansion to the new areas became more limited.

34

List of Sub-Projects Included in the Economic Analysis of PRONASAR

Community - Department Beneficiaries Net Present Value

NPV (USD) Economic Rate of

Return ERR La Capilla-Buena Vista, La Pampa, Polobaya Chico, Uzuna, Rinconada-Bellavista - AREQUIPA

1,925 -65,075 7%

Totos - AYACUCHO 1,103 -42,666 2%Acllahuayin - HUANUCO 208 -3,115 8%Bellavista - HUANUCO 450 -12,495 5%Salvio - JUNIN 800 -35,272 2%El Tingo - JUNIN 305 -55,855 1%Apas - LIMA 250 -18,879 1%Numumia - LIMA 260 -4,169 9%Ternique - PIURA 265 -129,502 -11%

Average 618 - 40,781 3%

35

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty Lending

Supervision/ICR Christophe Prevost Sr. Water and Sanitation Specialist LCSWS TTL Lizmara Kirchner Sr. Water and Sanitation Specialist LCSWS

Ana Lucía Jimenez Nieto Financial Management Specialist LCSFM Financial Management

Jean-Jacques Verdeaux Sr. Procurement Specialist LCSPT Sandra Arzubiaga Communications Officer LCREA Blanca Lopez Alascio Consultant LCSWS Selene del Rocio La Vera Procurement Specialist LCSPT Procurement Omar Guido Concepcion Extended Term Consultant LCSDU Raul Tolmos Environmental Specialist LCSEN Pamela Sofia Duran Vinueza Team Assistant LCSWS

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY00 8.65 37,430.91

FY01 9.80 71,324.28 FY02 39.33 297,165.41 FY03 10.12 45,939.69 FY04 1.45 262.08 FY07 0 11.26 Total: 69.35 452,122.37

Supervision/ICR

FY03 14.75 71,201.82 FY04 23.29 86,338.58 FY05 24.90 104,397.78 FY06 34.07 135,201.52 FY07 44.72 162,689.99 FY08 30.11 109,836.44 FY09 35.59 170,287.76 FY10 29.32 117,795.47

36

FY11 23.51 88,516.78 FY12 26.71 135.931.39 FY13 24.90 99.032.14 FY14 9.68 38,367.55 Total: 321.55 1,084,633.69

37

Annex 5. Beneficiary Survey Results One of the positive aspects of the PRONASAR project is the design and implementation of an impact evaluation (IE) study which measured the impact of the project between 2005 (baseline) and 2014 (endline)33. The focus of the evaluation was on Component 1: Water Supply and Sanitation in Rural Areas, which consisted of two types of interventions: 1A: rehabilitation, expansion, and improvement of existing systems; and 1B: construction of new systems. In both cases, the intervention also included the financing of latrines or basic sanitation facilities; training in hygiene education, management, operation, and maintenance; and the strengthening of the capacities of the district municipalities in supervision and the JASSs. The main objective of the IE was to determine whether the project had achieved its intended impact and outcomes, asking the fundamental question – the counterfactual: What would have happened if the program/intervention had not occurred? The baseline carried out in 2005 covered a sample of 43 localities in eight departments included in the first and second call for bids for PRONASAR. The methodology included the following: household surveys (860 total households interviewed); JASS or similar committee surveys (43 local committees interviewed); and district municipality surveys (19 districts interviewed). Survey questionnaires were generated and applied in both the baseline and endline studies, which were quantitative in nature. Health, Hygiene, WSS Access and Sustainable Management – PRONASAR’s Impacts The tables below summarize the main findings of the impact evaluation in terms of the impact of the interventions on health and sanitation access in the intervention areas, showing the treatment effect for each indicator (difference between treatment and control groups or intervention and non-intervention areas as well as the difference between the two periods, 2005 and 2015). Each table shows the statistical significance per indicator (p-value), using a “difference of difference” methodology/statistical test to measure treatment effect, with the exception of Indicator 1.6, which deals with JASS. A simple statistical test was used (Fisher test, which looks at the difference between the periods between the intervention and non-intervention areas) given the small sample of localities with JASS (24 in the baseline and 30 in the endline). At the impact level, the IE found that in the medium term, there was no statistically significant impact for Indicators 1.1 and 1.2 below related to health and hygiene, for both treatments. However, there were positive, statistically significant results in Indicators 1.3, 1.4, and 1.5. For indicator 1.6, there was a positive result (18.8% treatment effect) for the rehabilitation system.

Indicator Type of Intervention New System Rehabilitated System

1.1 Percentage of children under five with diarrhea in the last 15 days

Treatment Effect -6.7% 3.3% P-value 0.583 0.645 Significance

33 The consulting firm based in Peru, SASE Consultores, carried out the studies in both years.

38

Average Treatment Effect34: -7.4% (1.21)

1.2 Percentage of mothers with adequate handwashing practices

Treatment Effect -16.7% -11.4% P-value 0.108 0.146 Significance Average Treatment Effect: -10.6% (1.28)

1.3 Percentage of the population using a hygienic sanitation service

Treatment Effect 10.8% 5.8% P-value 0.000 0.007 Significance *** *** Average Treatment Effect: 11.7% (3.26)***

1.4 Percentage of households with continuous access to safe water.

Treatment Effect 42.3% 2.9% P-value 0.000 0.534 Significance *** Average Treatment Effect: 12.1% (2.11)**

1.5 Percentage of households with access to a sanitation service

Treatment Effect 48.4% 10.0% P-value 0.000 0.135 Significance *** Average Treatment Effect: 17.6% (2.26)**

1.6 Percentage of localities who adequately manage, operate and maintain their water supply system

Treatment Effect N/A N/A P-value N/A N/A Significance N/A N/A Treatment Effect (Rehabilitation): 18.8%

* = p<10%; **=p<5%; ***=p<1% Indicators Related to PRONASAR’s Intended Outcomes (a) Increasing sustained use of water supply facilities. One very significant positive outcome

of PRONASAR is the increased sustainability of water supply systems which were rehabilitated. The treatment effect is 42%, which means that the difference between the intervention and non-intervention areas in terms of functionality of the systems was 42%. In 2005, only 15% of the intervention areas had systems that functioned without major problems while in 2014, this has increased dramatically to 69%. In contrast, the non-intervention areas only experienced a 27% increase between these two periods. A few other statistically significant results are included below to further show what PRONASAR has achieved in the last decade or so of implementation. The results are much better for the new systems compared to the rehabilitated systems in terms of the indicators used below, but that is to be expected because the key indicator to look at in terms of sustainability of water systems, as measured by the impact evaluation is the percentage of systems which are functioning better, without major issues. This was one of the main objectives of the project and perhaps one of the key contributions of PRONASAR moving forward.


Percentage of household who use a safe water

Treatment Effect 34.1% 1.7% P-value 0.001 0.675

34 The treatment effect was calculated by the consultants, SASE Consultores, in determining the impact of the project in the endline study.

39

source for consumption and cooking

Significance ***

Percentage of households who bring water from outside the home

Treatment Effect -34.1% -1.7% P-value 0.001 0.675 Significance ***

Average time it takes a person to bring water to the house

Treatment Effect 27.1% 1.79% P-value 0.046 0.8 Significance **

* = p<10%; **=p<5%; ***=p<1% In general, the degree of satisfaction with the quality of water supply is quite high among households in intervention areas, though this is also high in non-intervention communities (Intervention areas: from 64% to 91% and non-intervention areas from 84% to 88%). Satisfaction with their sanitation facilities increased, as well, but it was not statistically significant for both types of intervention, new construction and rehabilitation. In addition, families were also satisfied with the management by the JASS, but it was not statistically significant for rehabilitated systems (80%, compared to non-intervention areas, which was 76%). For new systems with newly formed JASS, the level of satisfaction was 42% among households. While this was statistically significant, it still seems a bit low.

(b) Increasing sustained use of sanitation facilities. A smaller but also significant increase of 11 percentage points can be observed in the impact of the PRONASAR in the increased use of hygienic sanitation in its intervention areas compared to those that did not benefit from the Project (p<1%). There was a dramatic increase in the percentage of men, women, and children over three years of age who used latrines35 in the intervention areas compared to non-intervention areas. For both new and rehabilitated systems financed by the project, this percentage increase was 59 percent between 2005 and 2014, a significant difference from the non-intervention areas where the latrine usage rate only increased by 16 percent for new systems and 36 percent for new systems in the same time period (p<1%).

(c) Strengthening local communities’ capacity to manage services. In many ways, the

project helped strengthen local communities to manage, operate and maintain their water and sanitation service. Below are select indicators that show this as well as some which show a negative effect such as in the percentage of JASS that know the method of calculating family contributions.

Indicator Treatment Effect Percentage of communities with a JASS 7.7% Percentage of JASS trained in managing, operating, and maintaining water and sanitation services

38.1%

Percentage of JASS operating and maintaining the systems with updated records of users showing monthly household payments

19.1%

Percentage of JASS that know the method of calculating family contributions to O&M costs

-67.2%

Percentage of JASS with an O&M manual 31.3% Percentage of JASS with a complete and fully-functioning equipment 32.6%

35 These latrines are not necessarily defined as hygienic, which refers to absence of feces on the floor, seat or walls and absence of flies and bad odor.

40

Percentage of JASS trained in the daily chlorination (both system and household)

46.4%

Percentage of JASS in which daily chlorination is done 24.8% Percentage of water systems in which disinfection is done in a continuous manner

26.2%

Percentage of JASS led by women 7% Percentage of JASS trained in self-assessment and implementation of hygiene education

13.8%

(d) Strengthening municipal - district and provincial - level capacity to plan and oversee

water and sanitation services to rural communities in a sustainable and cost effective way. The key indicators related to capacity building of local governments show a negative effect. Below are select indicators that show the impact of PRONASAR on local governments, which can be summarized as low. Indicator Baseline (2005) Endline (2014) Percentage of district municipalities that know their supervisory functions

10.53% 47.37%

Percentage of district municipalities that carries out his supervisory functions

73.68% 42.11%

Percentage of district municipalities with at least one person who knows the key O&M activities in rural water supply and sanitation

15.79% 26.32%

Percentage of district municipalities with at least one person who knows the principal management activities of the JASSs

15.79% 15.79%

(e) Improving health and hygiene practices. The project had no impact on household health

and hygiene practices compared to localities in non-intervention areas. The evaluation study shows no impact of the project on the percentage of children under five with diarrhea in the last 15 days and on the percentage of mothers with adequate handwashing practices compared to the control group. Furthermore, the percentage of latrines without flies, bad odor, and that are clean dropped from baseline to endline, showing a worsening of hygiene practices in the communities, as can be seen in the table below.


Percentage of latrines without flies

Treatment Effect -27.5% -32.2% P-value 0.034 0.000 Significance ** ***

Percentage of households who bring water from outside the home

Treatment Effect -47.7% -30.7% P-value 0.000 0.000 Significance *** ***

Percentage of latrines that are clean

Treatment Effect -57.8% -31.8% P-value 0.000 0.000 Significance *** ***

* = p<10%; **=p<5%; ***=p<1%

41

Annex 6. Stakeholder Workshop Report and Results N/A

42

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

43

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A

44

Annex 9. List of Supporting Documents Project documents: 1. Project Appraisal Document, Report No. 24173, August 6, 2002 2. Project Papers on Restructuring 3. Project Supervision Aide-Memoires 4. Project Implementation Status and Results Reports 5. Project Financial Reports and Procurement Plans Other background information: 1. COSUDE. Análisis de las fases de los proyectos del PRONASAR y la línea de tiempo de

implementación (First and Second Call for Bids), Undated and unpublished report.

2. INEI (2012). Encuesta Nacional Demográfica y de Salud Familiar.

3. Ministerio de Vivienda, Construcción y Saneamiento (2010). Proyecto Piloto de Agua Y Saneamiento en Pequeñas Ciudades PROPECI: Un Modelo de Gestión Sostenible.

4. Ministerio de Vivienda, Construcción y Saneamiento. Plan Nacional de Saneamiento 2006 –

2015: DECRETO SUPREMO Nº 007-2006-VIVIENDA.

5. Ministerio de Vivienda, Construcción y Saneamiento Plan Nacional de Saneamiento 2006 - 2015” DECRETO SUPREMO Nº 007-2006-VIVIENDA

6. Progress reports and presentations from the Project Management Unit (UGP) 7. SASE (2005). Servicio de Consultoría para el Levantamiento de Información de Base –

Componente 1 del Proyecto PRONASAR. Informe Final. Abril 2005. 8. SASE Consultores (2014). Análisis de Impacto y Levantamiento de Información en las

Localidades Consideradas en la Línea de Base – PRONASAR. Informe Borrador. Abril 2014.

45

MAPS Map 1: Map of PRONASAR’s Areas of Intervention, by Department

Document of The World Bankdocuments.worldbank.org/curated/en/... · Ministry of Transport,...

Documents

Transcript of Document of The World Bankdocuments.worldbank.org/curated/en/... · Ministry of Transport,...