World Bank Documentdocuments.worldbank.org/curated/en/778331512661086878/pdf/ICR-P... · FEHIDRO...

The World Bank Brazil São Paulo Water Recovery Project (P106703)

Document of

The World Bank

Report No: ICR00003902

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IBRD-78700)

ON A

LOAN

IN THE AMOUNT OF US$64.50 MILLION

EQUIVALENT

TO THE

GOVERNMENT OF THE STATE OF SÃO PAULO (GESP)

FOR THE

BRAZIL - SÃO PAULO WATER RECOVERY PROJECT - REAGUA

November 29, 2017

Water Global Practice

Latin America and Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized


CURRENCY EQUIVALENTS

Currency Unit = Real (R$)

R$ 1.75 = US$1.00

US$1.00 = R$ 0.57

(Project Board Approval – January 12, 2010)

R$ 3.26 = US$1.00

US$1.00 = R$ 0.31

(Project Closing Date – May 30, 2017)

FISCAL YEAR January 1 - December 31

ABBREVIATIONS AND ACRONYMS

ANA National Water Agency (Agência Nacional de Águas)

CBA Cost Benefit Analysis

CEF Caixa Econômica Federal

CFP Call for Proposal

CPS Country Partnership Strategy

EMP Environmental Management Plan

ESF Environmental Social Framework

FEHIDRO São Paulo State Water Resources Funds (Fundo Estadual de Recursos Hídricos)

GESP Government of the State of São Paulo

GPOBA Global Partnership on Output Based Aid

IBRD International Bank for Reconstruction and Development

ICR Implementation Completion and Results Report

IFR Interim Financial Report

ILI Infrastructure Leakage Index

IRR

M&E

Internal Rate of Return

Monitoring and Evaluation

MTR Mid-Term Review

MRSP Metropolitan Region of São Paulo

OBA Output-based Aid

OBD Output-based Disbursement

PAC Growth Acceleration Program (Programa de Aceleração do Crescimento)

PAD Project Appraisal Document

PDO Project Development Objective

PMU Project Management Unit

PPA Project and Performance Agreement

PRODES Water Basin Pollution Control Program (Programa de Despoluição de Bacias

Hidrográficas)

QER Quality Enhancement Review

SAAE Autonomous Water and Sanitation Service Provider (Serviço Autônomo de Água e

Esgoto)


SABESP State Water and Sanitation Autonomous Utility (Saneamento Básico Do Estado De São

Paulo)

SANASA Municipality of Campinas water and sanitation company (Sociedade de Abastecimento

de Água e Saneamento S.A.)

SNIS National Information System on Water Supply, Sanitation and Solid Waste

SSE State Secretariat for Water Supply, Sanitation and Energy

SSRH State Secretariat for Water, Sanitation, and Water Resources (Secretaria de Saneamento

e Recursos Hídricos)

VA Verification Agent

WRM Water Resources Management

WSS

WWTP

Water Supply and Sanitation

Wastewater Treatment Plant

Vice President: Jorge Familiar Calderon

Country Director: Martin Raiser

Senior Global Practice Director: Guang Zhe Chen

Practice Manager: Rita E. Cestti

Project Team Leaders: Oscar Alvarado / Juliana Garrido

ICR Team Leaders: Oscar Alvarado / Juliana Garrido


BRAZIL

SÃO PAULO WATER RECOVERY PROJECT (REAGUA)

CONTENTS

A. Basic Information.................................................................................................................... i

B. Key Dates ................................................................................................................................ i

C. Ratings Summary .................................................................................................................... i

D. Sector and Theme Codes ....................................................................................................... ii

E. Bank Staff ............................................................................................................................... ii

F. Results Framework Analysis ................................................................................................. iii

G. Ratings of Project Performance in ISRs ................................................................................ v

H. Restructuring ......................................................................................................................... vi

I. Disbursement Profile ............................................................................................................. vii

1. Project Context, Development Objectives and Design ........................................................... 1

2. Key Factors Affecting Implementation and Outcomes........................................................... 4

3. Assessment of Outcomes ...................................................................................................... 13

4. Assessment of Risk to Development Outcome .................................................................... 20

5. Assessment of Bank and Borrower Performance ................................................................. 21

6. Lessons Learned ................................................................................................................... 23

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................... 24

Annex 1. Project Costs and Financing ...................................................................................... 26

Annex 2. Outputs by Component ............................................................................................. 28

Annex 3. Economic and Financial Analysis ............................................................................. 35

Annex 4. Bank Lending and Implementation Support/Supervision Processes ......................... 38

Annex 5. Beneficiary Survey Results ....................................................................................... 40

Annex 6. Stakeholder Workshop Report and Results ............................................................... 41

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ................................. 42

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................................... 44

Annex 9. List of Supporting Documents .................................................................................. 45

Annex 10. Maps .................................................................................................................... 46


i

DATASHEET

A. BASIC INFORMATION

Country: Brazil Project Name:

BR Sao Paulo Water

Recovery Project -

REAGUA

Project ID: P106703 L/C/TF Number(s): IBRD-78700

ICR Date: 10/08/2017 ICR Type: Core ICR

Financing Instrument: SIL Borrower: STATE OF SAO PAULO

Original Total

Commitment: USD 64.50 million Disbursed Amount: USD 37.74 million

Revised Amount: USD 64.50 million

Environmental Category: B

Implementing Agencies:

State Secretary for Water Resources and Sanitation (SSRH)

Cofinanciers and Other External Partners: N/A

B. KEY DATES

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 09/18/2008 Effectiveness: 12/27/2010 12/08/2010

Appraisal: 04/22/2009 Restructuring(s): 11/28/2015

Approval: 05/04/2010 Mid-term Review: 09/17/2012 10/28/2013

Closing: 11/30/2015 05/30/2017

C. RATINGS SUMMARY

C.1 Performance Rating by ICR

Outcomes: Moderately Unsatisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Unsatisfactory

Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Unsatisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing

Agency/Agencies: Moderately Unsatisfactory

Overall Bank

Performance: Moderately Unsatisfactory

Overall Borrower

Performance:

Moderately Unsatisfactory


ii

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments (if

any) Rating

Potential Problem Project at

any time (Yes/No): No Quality at Entry (QEA): None

Problem Project at any time

(Yes/No): Yes

Quality of Supervision

(QSA): None

DO rating before

Closing/Inactive status: Moderately Unsatisfactory

D. SECTOR AND THEME CODES

Original Actual

Major Sector/Sector

Public Administration

Public administration - Water, sanitation and flood

protection 4 4

Water, Sanitation and Waste Management

Water Supply 31 31

Sanitation 65 65

Major Theme/Theme/Sub Theme

Environment and Natural Resource Management

Environmental Health and Pollution Management 66 66

Air quality management 22 22

Soil Pollution 22 22

Water Pollution 22 22

Water Resource Management 33 33

Water Institutions, Policies and Reform 33 33

E. BANK STAFF

Positions At ICR At Approval

Regional Vice President: Jorge Familiar Calderon Pamela Cox

Country Director: Martin Raiser Makhtar Diop

Practice Manager: Rita E. Cestti Guang Zhe Chen

Task Team Leader(s): Oscar E. Alvarado

Juliana Menezes Garrido Carlos E. Velez

ICR Team Leader (s): Oscar E. Alvarado

Juliana Menezes Garrido

ICR Primary Author: Paula Pini


iii

F. RESULTS FRAMEWORK ANLYSIS Project Development Objectives (from Project Appraisal Document)

The Project Development Objective is to increase clean water availability in the critical watersheds in the State of Sao

Paulo.

Revised Project Development Objectives (as approved by original approving authority)

Not Applicable.

(a) PDO Indicator(s)

Indicator Baseline Value Original Target

Value (from

approval

documents)

Formally Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Cubic meters of recovered water (cubic meters (m3), Custom)

Value quantitative or

qualitative -- 30,106,311 47,966,660 47,127,718

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017

Comments

(incl. % achievement)

Target was almost achieved reaching 94 percent. Target was revised in the November 2015

restructuring to reflect expected outcomes by ongoing subprojects (components 1 and 2) and

contributions from the new component 4.

Indicator 2: Direct Project beneficiaries (number)


qualitative 0 -- 289,686 97,373

Date achieved 12/31/2009 12/31/2014 11/27/2015 05/30/2017

Comments


Target was not achieved reaching 11 percent. Indicator was added in the November 2015

restructuring. It was designed to reflect the achievements of component 2 and 4 added in the

November 2015 restructuring.

Indicator 3: Female beneficiaries (percentage, custom, supplement)


qualitative 51% -- 51% 51%

Date achieved 12/31/2009 12/31/2014 11/27/2015 05/30/2017

Comments


Target was fully achieved reaching 100 percent. Indicator was added in the November 2015

restructuring.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value Original Target

Value (from

approval

documents)

Formally Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Volume of recovered water per year from reduction in water losses (m3).


qualitative -- 9,275,156 30,572,331 43,713,975

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017


iv

Comments


Target was surpassed reaching 142 percent. The wording and target were revised in the

November 2015 restructuring. The original wording was: Volume of m3 of recovered

water per year from reduction in water losses.

Indicator 2: Cubic meters of reduced water consumption per year in selected public schools.

Value quantitative

or

qualitative

-- 98,915 147,338 335,080

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017

Comments

(incl. %

achievement)

Target was surpassed reaching 227 percent. The wording and target were revised in the

November 2015 restructuring. The original wording was: Volume of reduced water

consumption per year in selected public schools.

Indicator 3: Volume of m3 of wastewater reused per year in selected activities.

Value quantitative

or

qualitative

-- 938,727 Dropped

Date achieved 12/31/2009 12/31/2014 11/27/2015

Comments

(incl. %

achievement)

Dropped. This indicator was dropped in the November 2015 restructuring given that no

contract was signed to implement reuse activities under the Project.

Indicator 4: Number of new active connections to the collection network of participating services

provider.

Value quantitative

or

qualitative

0 4,748 2,346 330

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017

Comments

(incl. %

achievement)

Target was not achieved reaching 14 percent. The target for this indicator was revised

during the November 2015 restructuring to a smaller number to reflect the actual number

of contracts signed associated with activities that would contribute to the indicator.

Indicator 5: Volume of wastewater transported to and treated in a wastewater treatment plant.

Value quantitative

or

qualitative

-- 19,793,513 13,172,267 3,078,633

Date achieved 12/31/2009 12/31/2014 11/27/2015 05/30/2017

Comments

(incl. %

achievement)

Target was not achieved reaching 23 percent. Wording and target revised during the

Novermber 2015 restructuring. The original wording was: Volume of water that is

returned to selected watersheds in conformity with environmental standards, as measured

by the increase in population equivalent that is served by wastewater treatment plant.

Indicator 6: Impact evaluation study ready.

Value quantitative

or

qualitative

-- Completed Dropped --

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017


v

Comments

(incl. %

achievement)

This indicator was dropped during the November 2015 restructuring given technical

difficulties (lack of similar Projects to compare to) that impedes carrying out the study. A

results evaluation study was carried out as part of the ICR preparation.

Indicator 7: Number of water utilities that the Project is supporting.

Value quantitative

or

qualitative

-- 8 8 8

Date achieved 12/31/2014 12/31/2014 11/27/2015

Comments

(incl. %

achievement)

Target was fully achieved reaching 100 percent. The original indicator was changed in the

November 2015 restructuring. The original wording was: Increasing number of service

providers receiving technical assistance, capacity building and training.

Indicator 8: Wastewater volume transported and treated in the treatment plant and returned to the basin

in accordance with environmental legislation. (m3)

Value quantitative

or

qualitative

-- -- 3,513,600 0

Date achieved 12/31/2009 12/31/2014 11/27/2015 05/30/2017

Comments

(incl. %

achievement)

Target was not achieved. New indicator included as part of the November 2015

restructuring associated with the inclusion of component 4.

Indicator 9: Volume of reduced water consumption per year in housing units of social interest.

Value quantitative

or

qualitative

-- -- 700,800 0

Date achieved 12/31/2009 12/31/2014 11/30/2015 05/30/2017

Comments

(incl. %

achievement)

Target was not achieved. New indicator included as part of the November 2015

restructuring associated with the inclusion of component 4.

Indicator 10: Number of people in urban areas provided with access to improved sanitation under the

Project.

Value quantitative

or

qualitative

0 20,000 Dropped --

Date achieved 12/31/2009 11/30/2015 11/30/2015 05/30/2017

Comments

(incl. %

achievement)

Dropped. This indicator was dropped as part of the November 2015 restructuring as the

number of beneficiaries was already included as a PDO level indicator.

G. RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR

Archived DO IP

Actual Disbursements

(USD millions)

1 06/08/2010 Satisfactory Satisfactory 0.00

2 11/30/2010 Satisfactory Satisfactory 0.00

3 06/25/2011 Satisfactory Moderately

Satisfactory 0.00


vi


Satisfactory 0.00


Satisfactory 0.25

6 07/10/2013 Moderately Satisfactory Moderately

Unsatisfactory 1.22

7 02/24/2014 Moderately Satisfactory Moderately

Unsatisfactory 3.27

8 10/20/2014 Moderately Unsatisfactory Moderately

Unsatisfactory 6.75


Unsatisfactory 16.67









H. RESTRUCTURING (IF ANY)

Restructuring

Date(s)

Board

Approved

PDO

Change

ISR Ratings at

Restructuring

Amount

Disbursed at

Restructuring

in USD

Millions

Reason for Restructuring & Key

Changes Made DO IP

11/28/2015 MU MU 25.66

The restructuring was to: (i)

extend the Project closing date

by 18 months to 05/30/2017; (ii)

introduce a new component 4;

(iii) incorporate changes to the

implementing agency,

safeguards, implementation

schedule, reallocation of funds;

and (iv) modify the results

framework, disbursement and

cost estimates.


vii

I. DISBURSEMENT PROFILE


1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. The challenge of water scarcity in the State of São Paulo1. With 95 percent of its 41 million

people living in urban areas, the State of São Paulo is one of the world’s most urbanized areas and

emblematic of the urban challenges facing Brazil. At Project appraisal, the Metropolitan Region of São

Paulo (MRSP), with its 20 million inhabitants, was the seventh largest city in the world. Among its most

pressing problems, water scarcity and environmental degradation were extremely critical constraints on

economic growth and competitiveness, as well as on social and environmental sustainability. MRSP’s per

capita water availability was so low that it was comparable to that prevailing in the driest areas of the

Brazilian Northeast.

2. The problem of water quality. The State of São Paulo has not only faced the problem of critical

water scarcity, but also suffers from the deterioration of its water quality. At the time of Project

preparation, although 49 percent of collected wastewater received treatment, there was still approximately

15.7 million people living in the State with wastewater collection but no treatment. Dumping untreated

wastewater directly into the water bodies further exacerbated the already critical problem of water

scarcity.

3. Other dimensions of water scarcity. The relatively high level of water losses and the lack of

treated wastewater reuse compounded the problem of water scarcity. Water losses can be physical (or

real) through leakages, commercial (or apparent), through theft or metering inaccuracies, or unbilled

authorized consumption. According to the National Information System on Water Supply, Sanitation and

Solid Waste (SNIS) statistics for 2007, of the 477 municipalities in São Paulo reporting to SNIS (out of

645 municipalities in the State), 16 percent had real water losses above 50 percent, 44 percent had real

water losses above 30 percent, and only 14 percent had real water losses below 20 percent. This lack of

homogeneity reflected huge variations between utilities, with real water losses averaging 28 percent for

SABESP, 58 percent for the autonomous service provider (autarquia) in Guarulhos, and 24 percent for

SANASA2 in Campinas.

4. The poor are the most vulnerable to the effects of low water availability. The scarcity of

potable water and the deficit of wastewater collection and treatment were more common in poor

neighborhoods, and particularly severe in slums. This was the case of São Paulo, where about 54 percent

of the poor lived in areas of critical water deficit and 59 percent lacked proper sanitation.

5. State interest in results-based operations. In an example of the evolving and dynamic

relationship between the Bank and the Government of State of Sao Paulo (GESP) in the water sector, the

State Secretariat for Water Supply, Sanitation and Energy (SSE)3 expressed interest in executing part of

1 Brazil: Inputs for a Strategy for Cities: A Contribution with a Focus on Cities and Municipalities, World Bank

Report No. 35749 (June 2006). 2 Sociedade de Abastecimento de Água e Saneamento S.A (Municipality of Campinas water and sanitation

company) 3 Later on named State Secretariat for Water, Sanitation, and Water Resources (Secretaria de Saneamento e

Recursos Hídricos, SSRH).


2

its programmatic commitment through a pilot Project using an output-based financing mechanism. SSE’s

objective was to satisfy water supply and sanitation (WSS) investment needs, while providing

transparency in the management of resources, better application of funds, and improved service quality

within a reasonable timeframe. In response to GESP’s request, the proposed Project designed an Output-

Based Disbursement (OBD) financing mechanism to encourage investment in activities aimed at

improving the efficiency of WSS assets. Under the results-based scheme, Project funds would be

disbursed to incumbent providers of water services (state or municipal-owned) against agreed and

independently verified outputs.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

6. The Project Development Objective (PDO) in the loan agreement is stated as follows: “To

increase clean water availability in the critical watersheds.” The PDO in the Project Appraisal Document

(PAD) also mentions “…in the State of São Paulo.” In addition, the PAD defines “clean water” as

additional potable water available for consumption or additional treated wastewater discharged to a water

body in conformity with environmental legislation. The Project had one PDO indicator: The number of

cubic meters of clean water made available.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and

reasons/justification

7. The PDO was not revised. The PDO indicator target was changed during the November 2015

restructuring from 30,106,311 cubic meters (m3) to 47,966,660 m3. The increase reflected the expected

outcomes from the ongoing activities under both Components 1 and 2, and contributions from the newly

included Component 4. In addition, the November 2015 restructuring introduced two new PDO

indicators: (i) Direct Project Beneficiaries, with a target of 289,000 people; and (ii) Female Beneficiaries,

with a target of 51 percent of the Direct Project Beneficiaries.

1.4 Main Beneficiaries

8. The Project beneficiaries were identified in the PAD4 (page 76 and paragraph 10), as the water

and sanitation service providers located within the selected five critical watersheds5 of the State of São

Paulo. Furthermore, only three types of service providers were eligible for financing: autonomous

services providers, municipal companies, and state/regional companies. The November 2015 restructuring

estimated that the Project aimed to directly benefit approximately 289,000 people; out of which 51

percent were expected to be female.

1.5 Original Components (as approved)

9. The Project comprised three original components.

10. Component One: Increase Water Availability (cost at appraisal, US$30.68 million). This

4 Report No. 53620-BR.

5 The five critical watersheds were: Piracicaba/Capivari/Jundiaí; Alto Tietê; Sapucaí Mirim/Grande; Mogi-Guaçu;

and Sorocaba/Médio Tietê. These were the five most critical watersheds in the State, with water demand/availability

ratio of above 80 percent.


3

component was designed to finance activities aimed at increasing the quantity of available water by

controlling and reducing real water losses, promoting the rational use of water in public schools and

reusing treated wastewater. It was 100 percent OBD financed, with payments to the service providers tied

to the delivery of clearly specified outputs with a final payment made upon reaching agreed targets and

sustainability conditions. It included three sub-components: (i) Control and reduction of real water losses

(US$22.88 million); (ii) Rational use of water (US$3.05 million); and (iii) Reuse of treated wastewater

(US$4.75 million).

11. Component Two: Enhance Water Quality (cost at appraisal, US$64.80 million). This

component was designed to finance activities aimed at improving the quality of water by: (i) building new

wastewater connections to new or existing networks; (ii) installing pipes and pumps to transport

wastewater to new or existing treatment plants; and (iii) constructing new or upgrading existing

wastewater treatment plants (WWTPs). It was 100 percent OBD financed. It included three sub-

components: (i) Wastewater collection networks (US$4.00 million); (ii) Wastewater transport systems

(US$27.10 million); and (iii) Upgrading existing and constructing new WWTPs (US$33.70 million).

12. Component Three: Project Management and Development of Institutional and Technical

Capabilities (cost at appraisal, US$8.16 million). This component was designed to finance activities to

increase the efficiency of the State’s WSS sector. It included: (i) the carrying out of an impact evaluation

study on the output-based financing scheme financed by the loan vis-à-vis the traditional input-based

financing approach, specifically on impact of the output-based approach on the construction time period

and costs and sustainability; and (ii) the provision of technical assistance, capacity building, and training

to water and sanitation service providers. In addition, it aimed at providing support for project

management, supervision and auditing, including provision of experts for the strengthening of the Project

Management Unit (PMU) in the management of the Project and in the verification of outputs and the

Project and performance indicators under the Project and Performance Agreement (PPA). This

component was to disburse based on inputs.

1.6 Revised Components

13. During the November 2015 restructuring, the components descriptions were slightly revised to

reflect the activities that were dropped; and included activities in response to the severe drought faced by

the State that affected the MRSP water supply services dramatically:

14. Component One: Increase Water Availability (revised cost, US$31.67 million). The reuse of

treated wastewater sub-component was dropped. The budget of the other two components was adjusted as

follows: (i) Control and reduction of real water losses (US$29.47 million); and (ii) Rational use of water

(US$2.20 million). This component continued to disburse based on results.

15. Component Two: Enhance Water Quality (revised cost, US$27.68 million). The budget of the

sub-components was adjusted as follows: (i) Wastewater collection network (S$1.12 million); (ii)

Wastewater transport system (US$6.90 million); and (iii) Upgrading existing and building new WWTPs

(US$19.66 million). This component continued to disburse based on results.

16. Component Three: Project Management and Development of Institutional and Technical

Capacity (revised cost, US$5.75 million). The following activities were dropped: (i) impact evaluation

studies; and (ii) technical assistance to municipalities; and municipal and regional service providers. The

total cost amount increased for the following activities: (i) project management; (ii) financial audit; and

(iii) Independent Verification Agent. This component continued to disburse based on inputs.


4

17. Component Four: Emergency Works for Water Supply Sources (estimated cost, US$30.65

million). This component was added to help minimize the (2013 – 2015) drought’s impact on the MRSP

water production sources6. The component included: (i) the execution of environmental sanitation works

to minimize the impact of the crisis that undermined the two main water sources for the MRSP (the

Cantareira and the Alto Tietê systems), including upgrading or construction of wastewater collection

networks, transport system, and/or treatment plants; and (ii) a water demand management program,

including the implementation of measures to increase the rational use of water in low-income housing

complexes in key neighborhoods of the MRSP, located preferably in the catchment area of key water

production systems. This component was input-based financed, rather than output-based.

1.7 Other significant changes

18. Other changes introduced during the November 15 restructuring included: (i) 18-month extension

of the original closing date from November 30, 2015 to May 30, 2017; (ii) changes in the implementation

schedule; (iii) modifications in the results framework, including indicators and targets; and (iv)

disbursement estimates.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

19. The financing mechanism. Project preparation started7 in late 2008, following the 2007 request

submitted by GESP to the Federal Government and the Bank. The São Paulo Water and Sanitation and

Water Resources Secretariat (SSRH)8 was the executing agency. The Project was appraised 9 months

later. During this short period, the Project concept and design evolved significantly: from a broader to a

narrower focus on selected water and sanitation activities and areas/basins for interventions; and, from a

general to a tailored results-based payment approach, fine-tuned to the technical specifications of the

activities financed, as well as to adjust to the Bank financing requirements. The Project preparation was

supported by a US$180,000 GPOBA grant9 signed in early 2008. The Project was the first Bank-financed

operation in the water supply and sanitation sector to implement a comprehensive results-based scheme.

Between the Bank and GESP, this scheme was an OBD, while between the GESP and the beneficiaries

(WSS providers), it was an Output-Based Aid (OBA), therefore, a grant.

6 Extremely low rainfall over the past two years before the 2015 Restructuring led to a severe water supply crisis in

the state of São Paulo and in the MRSP in particular. The reservoir system that supplies the metropolis was at risk of

depletion, with potential dire social and economic consequences. Despite institutional gaps in water resources

management at higher levels of government, SABESP managed to avoid the collapse of the system. Nevertheless,

risks remained and there was uncertainty about the immediate future and recovery of the accumulation level of the

reservoirs. Of particular concern was the Cantareira reservoir system, which supplied to about half of the

metropolitan 20 million people but by the 2015 Restructuring its supply capacity was reduced to only 25% of the

metropolitan inhabitants. 7 The Project Concept Review Meeting (PCN) took place on September 18, 2008, the Decision Meeting was held 9

months later, on May 07, 2009. 8 During Project preparation and early implementation, the SSRH was named SSE.

9 GPOBA grant (P114151): Design of innovative OBD schemes for water supply and sanitation Projects in two

Brazilian states (Pernambuco and São Paulo).


5

20. Lessons incorporated. The Project assimilated lessons from: (i) the Bank supported

OBA approach, in particular from the Mexico Decentralization Infrastructure Reform and

Development Project solutions with respect to unit costs definition; (ii) the Brazilian Water

Basin Pollution Control (PRODES) Program, a federal program sponsored by the National Water

Agency (ANA), which had produced significant experience with results-based financing

mechanisms that partially subsidized construction of WWTPs; (iii) the SABESP investment

programs in reducing water losses, as well as from10 addressing rational water use in public

facilities through performance-based contracts; and (iv) the findings from the International Water

Association’s Water Loss Task Force.

21. Testing OBD effectiveness. The PAD11

highlights that the Project envisaged to properly test in a

wider context the effectiveness of the results-based mechanisms in terms of: (i) encouraging investments

in activities aimed at increasing water quantity and improving water quality in the most seriously

compromised watersheds in the State of São Paulo; (ii) promoting transparency and efficiency; (iii)

improving access to services by the poor; (iv) demonstrating potential for scale up and replication; (v)

providing solutions to the difficulty faced by municipalities and utilities in putting up funds in advance to

finance Projects; (vi) supporting weaker municipalities and service providers so they would be capable of

competing for the funds and implementing the activities; (vii) ensuring a better application of funds and

improved services within a reasonable timeframe; and (viii) shifting performance risk from GESP to

selected service providers.

22. Key inputs that shaped the Project design. Early during Project preparation, the SSRH

launched the Projects’ first Call for Proposals (CFP)12

. The Project results-based financing specifications

were still under development, but it was already defined that the Project-targeted territory would

encompass the five most critical watersheds13

in the São Paulo state. The SSRH received 165 proposals

with a total value of US$259.38 million14

. The significant response indicated demand for the Project

results-based approach, and provided guidance on the distribution of the Project funds amongst its

components.

23. Pioneering legal and technical tools, as well as decision chain. The Projects’ innovative nature

required the development of several new technical and legal tools consisting of: (i) the CFP procedures

aimed at the participants selection; (ii) the set of six normative presenting in detail the methodology and

unit costs; (iii) the PPA model, which set forth the obligations in the implementation of the activities and

constituting the legal contract15

to be signed by the service providers and the SSRH; (iv) the Verification

Agent (VA) procedures to verify that the agreed outputs have been produced in accordance the signed

PPA; (v) the Implementation and Targets Plan providing the baseline information; and (vi) the overall

detailed procedures for the preparation, implementation, and monitoring of each activity.

10 SABESP’s Programa Uso Rational da Água (PURA) intervenes in public facilities such as schools, health centers

and hospitals, as well as prisons. 11

Project PAD dated March 26, 2010. See paragraphs 11 and 15 in pages 5 and 6 respectively. 12

The Project first Call for Proposals was launched in November 27, 2008. 13

The five most critical watersheds encompassed: 40,000 km2; 30M people; and 191 municipalities. 14

REAGUA Project - QER Meeting Minutes, dated March 30, 2009. Page 2. 15

PPA/contract as designated by the SSRH: Contract for Concession of a Financing Stimulus to the Water Recovery

Activity. REAGUA Project Operational Manual, Vol. 1, page 23.


6

24. Readiness of required implementation tools: By the completion of Project preparation

activities, the degree of readiness of the required implementation tools was uneven. The most advanced in

preparation was the set of normative. This was essentially conceived and developed in the few months of

the preparation phase, benefiting from ANA’s PRODES and SABESP’s performance-based contracts

tested implementation tools. However, the other several technical and legal required instruments were left

in very preliminary stages. The development of the PPA — the legal contractual document — proved to

be one of the most difficult to achieve. It generated broad, long and complex discussions, triggering

numerous unexpected and challenging technical and legal issues. One of those referred to the legal

implications associated with GESP providing a grant to SABESP, given that GESP held 51 percent of

SABESP’s shares, while the remaining was held by the private sector. During the preparation phase, no

conclusion was reached regarding the results-based PPA contract template. In addition, the development

of other critical technical tools and procedures pertinent to Project implementation, such as the

specification for the VA activity, was left to be resolved at the implementation phase of the Project. The

lack of readiness of these key technical tools significantly undermined and delayed the Project

implementation.

25. Implementation arrangements. At appraisal, the Project aimed to operate through PPA

contracts between the State and the operators, which were selected by a demand and merit approach. The

SSRH staff were expected to ensure Project oversight, while two private consulting companies were

expected to be selected, one to support the overall Project implementation activities and, the other, to

carry out the VA attributions. The arrangements were consistent with those prevailing in most of the

lending operations in the country.

26. Other relevant elements. Other elements that impacting on quality at entry related: (i) The

reference unit price: The definition of the reference unit price was one of the most complex activities

undertaken during preparation. It involved a detailed study tn the construction costs of water and

sanitation infrastructure practiced both in the State of São Paulo and at national level, as well as

developing a cost model for the typical activity under each subcomponent. These were used in calculating

and establishing the payments that would be received when complying with the implementation and

sustainability conditions; (ii) The two-phased payment: The results-based approach considered two

payment phases (implementation and sustainability). The payments during implementation sought to help

the provider to overcome, at least partially, the lack of funds for executing the activity. They were

disbursed against partial results or execution stages achieved. The sustainability payments were disbursed,

after the subprojects were operational, against achievement of agreed targets and sustainability conditions.

Partial compliance resulted in partial payments. The ratio for control of water losses activities was 60/40,

while it was 70/30 for the other subcomponents; (iii) The total cost: The Project was not informed on the

expected or actual costs of the activities it supported. Payments were calculated assuming counterpart

funds were necessary and were expected to be significantly larger than the Project financial incentive,

especially for the construction of WWTPs; and (iv) The financing: The activities supported by financing

were also eligible under the Project, excluding those supported by a grant, since the Project also provided

grants.

2.2 Implementation

27. Slow implementation takeoff. The Project faced severe delays over the first two- years (2011

and 2012) to get actual implementation off the ground. The following reasons can be ascribed to the slow

takeoff:


7

(a) Lack of an approved PPA template. The final version of the PPA fell under the GESP

juridical department responsibility, beyond the SSRH institutional scope and control.

(b) Delays in contracting both the management and the Verification Agent (VA) teams. It was

only accomplished by the end of the Projects’ second implementation year.

(c) Fine-tuning the operational procedures. Once the two consulting companies were finally on

board, it became evident that the overall operational procedures, as well as the VA protocol,

needed further detailing, updating and a reality check.

(d) Learning-by-doing. Both the Project management and the VA teams underwent a learning-

by-doing process, developing while also implementing and revising the measuring and

evaluation tools.

(e) Tailored technical assistance. It became increasingly evident that technical support was

needed. Two Bank consultants that played a fundamental role in the preparation phase

provided intense tailored clarification and training.

28. Substantial demand drop. During the Project life cycle, three CFPs were launched to attract

demand in meeting the Project financing availability. The first CFP resulted in a demand almost three

times the Project total funds; the two others generated enough demand to commit all the remaining

available funds. However, as the process evolved (i.e., selection, signing of contract, and implementation),

the drop in participants and funds commitment was substantial16

. The reasons for the drop included:

(a) Given the long period of 4 years between the first CFP launch (early in the Project

preparation) and the subsequent selection and contract signing, many providers declined from

participating in the Project.

(b) Many of the proposals from the first CFP had inconsistencies with the results-based approach

since the set of normative was not ready, thus were not disclosed.

(c) Lack of upfront own or other financing, in disagreement with the CFP requirements.

(d) Provider declined to participate considering that committing to the Project was too complex

and not financially rewarding.

(e) PPA template limitations, such as no reference unit price adjustments during the contract

duration and no retroactive financing (several proposals from the first CFP were already

under implementation when signing a contract with the Project was proposed).

(f) Provider obtained a Federal grant. which would offer better financial incentives and then

became ineligible.

(g) Long delays faced by the SSRH in obtaining internal clearances, in particular from the legal

department, to allow the signing of contracts, triggered more dropouts.

(h) No payment for results could be made until the VA had been hired17

and was performing the

evaluations activities, thus severely affecting the proposals under the first CFP, launched 4

years earlier.

(i) Proposal implementation timeline no longer compatible with the Project implementation

period.

(j) Proposal was already under implementation when there were changes in the designs, thus the

proposal submitted to the Project was no longer valid.

16 First CFP:165 proposals received, 19 contracts signed, and 11 implemented. Second CFP: 26 proposals received,

14 contracts signed, and 7 implemented. Third CFP: 10 proposals received, 9 contracts signed, and 8 implemented. 17

The VA contract was signed in August 2012, as well as contract with the consulting company to perform the

Project management activities. The gap period between the first Call for Proposal and the VA contract signing was

approximately 4 years.


8

(k) Proposal submitted by a municipality in which the water and sanitation services were

delivered by a private service provider, or the selection process to concede the concession to

a private company was ongoing were not eligible.

29. Issues that significantly changed Project commitments and disbursement projections. At

each CFP, the results achieved in terms of number and costs of the proposals submitted led to the

conclusion that the Project funds were fully committed. However, as the processing progressed, several

downward adjustments were necessary. Finally, by Project closure just about 42 percent of the total

Project funds were disbursed. The main reasons included the following:

(a) Impact of the Brazilian currency depreciation. Significant appreciation of the dollar vis-à-vis

the Real (43 percent increase from April 2009 to May 2017) increased uncommitted and

undisbursed loan proceeds.

(b) Water and financial crisis. The unprecedented drought from 2014 to 2016 in São Paulo

significantly affected investments prioritization by the service providers particularly

regarding those contracts that were meant to carry out sanitation investments. Most of these

investments were put on hold while all efforts were focused on activities to increase water

supply. Also, given the severe drought affecting the state of São Paulo, water losses in places

where there was rationing (about 6 cases), caused difficulties in establishing the baseline and

results measurements, brought about by the intermittent supply of the water service.

(c) New Component 4 added during restructuring. In the November 2015 restructuring, the

added component addressing emergency works for water supply was expected to absorb the

uncommitted loan proceeds. However, its implementation was not initiated given GESP fiscal

and financial crisis that hit the State in early 201impacting on counterpart funds availability.

30. Mid-Term Review (MTR). The MTR mission took place in September 2013. Despite the low

disbursements, the PPA commitments at that time were evaluated as sufficient to achieve the PDO and

able to absorb all remaining Project funds by the Project closing date. In addition, the operational process

was analyzed and some opportunities for improvement were identified, such as revising the PPAs to

reduce the time of the last payments and increasing the incentive payments. However, these were deemed

to only bring marginal acceleration and were not pursued given the expected difficulties with amending

these contracts in Sao Paulo. However, after the MTR, several simultaneous and unexpected unfavorable

events took place, such as the eventual dropping of various contracts mostly due to the financial crisis

and/or legal issues (the three largest contracts valued at a combined US$26 million failed), as well as the

dollar appreciation peak of about 80 percent in the following 2 years. These again, led to a commitment

lag, which was at the time addressed by the third CFP. In retrospect, the MTR may have missed the

opportunity to improve the PPA’s financial rewards. Improvements on this front would have accelerated

Project implementation by reducing the requirement of counterpart funds and likely preventing some

drop-outs.

31. The Project restructuring. The Project restructuring and the original closing date coincided. At

restructuring, the total loan proceeds disbursed and total estimated commitments were 25 and 58 percent,

respectively18

. The restructuring responded to GESP request to help in addressing the then ongoing

18 REAGUA Project ISRs # 9 and # 10.


9

drought crisis in the State of São Paulo19

and among other changes, reallocated funds from other

components to finance complex activities under the new Component 4. In retrospect, the planned

timeframe (18 months) to implement the new component was over-optimistic.

32. Technical assistance, capacity building and training. While the Project was supposed to

provide technical assistance, capacity building and training to support the service providers in the design,

operation, and maintenance of programs and facilities associated with the implementation of Components

1 and 2, these activities were not implemented. Based on ongoing implementation learning, the

government and Bank team agreed that tailored assistance would provide greater benefits. The two Bank

consultants who were pivotal in developing the project results-based approach delivered it. Finally, it was

also foreseen to finance an impact evaluation study on the OBD financing scheme vis-à-vis the traditional

input-based financing approach. This study was canceled in the November 2015 restructuring given the

difficulties associated with the methodology.

33. Project success in implementing non-intensive capital investments activities. In summary, the

Project results-based approach was very successful in financing non-intensive capital investment

activities (such as the control of water losses, and rational water use). In contrast, it was less successful in

supporting capital-intensive activities (such as waste water treatment facilities). Regarding the control of

water losses and rational water use (in particular those selected through the third CFP), the Project results-

based approach was able to: (i) influence the technical solutions; (ii) provide a meaningful financial

incentive; and, (iii) contribute to a reasonable implementation timeframe. With respect to the sewerage

investments, the Project results-based approach did not achieve the expected results. Table 2 provides key

information on the contracts implemented.

34. Selected providers. The Project supported 23 results-based contracts, which were implemented

by 8 providers20

. Four21

of those responded to 95 percent of the total Project funds allocated to the results-

based financing. Some of these providers were rated amongst the best performing ones in the State of São

Paulo22

.

35. Results of testing OBA effectiveness. Project implementation generated a wealth of information

on the effectiveness of the results-based approach tested in a very wide range of conditions. The most

relevant are summarized below:

(a) Demonstrating potential for scale up and replication. The results indicate that the results-

based approach was an appropriate mechanism to support activities that do not require

intensive capital investments. It was successful in shifting the focus from implementation to

the results, promoting changes in the providers’ management culture and promoting the

adoption of new technologies. This also gave autonomy and flexibility to the provider’s

technical teams involved, resulting in greater agility and efficiency in using the financial

incentive offered by the Project. However, the results-based approach did not promote

19 The proposed 2015 restructuring of the REAGUA Project, along with the restructuring of the Mananciais Program,

also carried out in 2015, allowed the Bank to provide critical and timely technical and financial support to GESP and

SABESP in facing unprecedented water supply crisis affecting the MRSP. 20

SABESP; SANASA, Campinas; SAAE, Guarulhos; SAAE, Indaiatuba; CODEN, Nova Odessa; SAAE, Santo

André; SAAE, Sorocaba; and DAEV, Valinhos. 21

SABESP, 50 percent; SAAE Indaiatuba, Guarulhos, and SANASA, 15 percent each. 22

Ranking of WSS in Brazil (Ranking do Saneamento no Brasil, 2015); Instituto Trata Brasil and GO Associados.


10

changes in the water and sanitation capital intensive investments. Their designs, execution

procedures, timetable23

, and costs remained unchanged. Moreover, it was emphasized that all

sanitation investments financed by the Project received simultaneous support from input-

based financing, since no utility committed to such investments otherwise.

(b) Providing solutions to address the difficulty faced by utilities in putting up funds in advance

to finance sub-projects. Greater support through the implementation phase, coupled with

higher reference unit costs, could have triggered more favorable implementation of the

activities under the Project24

.

(c) Shifting performance risk from GESP to selected service operator. According to statements

from the mid-size utilities representatives, the payment for results-based approach actually

induced the transfer of risks to the technical team in charge of implementing the intervention,

generating anxiety and defensive attitudes. The technical team saw advantages in the results-

based approach and took the initiative to sell it to the company, but faced doubts particularly

from the financial department, which claimed that the risk of not accessing the full incentive

support was high. In large-size utilities, such dialogue concerning the results-based financial

support was not under debated. Committing to the Project was a high-level decision aligned

with government initiatives.

36. Risks. There were two risks not considered during Project preparation. Underestimated: (i) the

high dependency on traditional input-based financing for sanitation investments; and (ii) the unforeseen

execution timeline for these investments in order to comply with regulations applying to technical,

institution and environmental rules.

37. The VA contract designs. The VA contract was the first of such contracts in Brazil. Its

implementation successfully met the expected results, providing sound indications that the contract as

conceived was appropriate and constituted a solid contribution from the Project for future similar

operations. Nevertheless, in order to fully understand their tasks, as well as the Project results-based

approach, the VA team required significant training provided by the Bank consultants. Still, the VA team

undertook all activities under its contract, providing the required information for issuing the Project

results-based payments. No complaint against the VA was registered and no amendment to the VA

contract was required. Moreover, the providers supported by the Project were soon able to clearly

distinguish the roles performed by the VA and those undertaken by the Project management team.

38. Low disbursement (51 percent) but Project achieved 98 percent of the PDO indicator target

value. At Project closure, the target value of the PDO indicator “cubic meters of recovered water” was 98

percent achieved, although only 51 percent of the Project total funds were disbursed. Moreover, 92

percent of the PDO indicator target value achievement resulted from the implementation of just one sub-

23 Information provided by Project stakeholders that also participated in the ANA’s PRODES Program, the

implementation timetable of the WWTPs that received incentive from PRODES ranged from 10 to 16 years.

However, as the PRODES financial incentive was deposited in a special escrow account which benefited from

interest rate returns, when it was allowed to be withdrawn (by the agreed results verification), the amount in the

account had increased at least accordingly with inflation. 24

The activities addressing reduction and control of real water losses were also supported by different input-based

financing such as from the São Paulo State’s water resources funds (FEHIDRO) and the Institutional Development

financing lines financed by the Federal Bank (CEF). They had the support of own source revenues and/or labor

resources. Interventions implemented by SABESP were, in some cases, complemented by the company’s large

losses control performance-based program.


11

component — control of water losses. The Third CFP, which targeted only water losses, focused on sub-

projects with ongoing water losses programs, which required modest investments but yielded high

volumes of recovered water.

39. The strong depreciation of the Brazilian currency to the dollar. The Brazilian currency

experienced a 43 percent depreciation against the dollar during the Project disbursement period but

peaked to nearly 80 percent depreciation from September 2013 to December 2015. While this

significantly increased the undisbursed loan proceeds, it did not affect the implementation of the signed

contracts and the achievement of the targets associated with them.

40. Uneven Project achievements. Component 1 largely exceeded in achieving the associated

indicators’ targeted values, almost entirely supported by the subcomponent “water losses control”. The

latter exceeded its target by 143 percent that had been increased at the 2015 restructuring, which was, in

turn, 333 percent larger than the target set at appraisal. However, this remarkable achievement was

supported by an increase of just 33 percent of funds (from the appraisal estimates to the actuals at the

Project closure). Component 2 contribution to the achievement of the PDO level indicator was minimal

— only 6 percent, while Component 4 did not contribute since it was not implemented.

41. Final and verified results larger than Providers’ estimates. The implementation of the results-

based approach and subsequent verification process revealed that the providers supported by the Project

made a downward estimate of the volume of water to be recovered in order to avoid eventual Project

financial penalties and criticism from other departments.

42. Realism in Project design. With the benefit of hindsight, the Project could have been able to

achieve its targets with a more realistic design and restructuring.

43. The control of water losses approach constitutes an exceptional contribution to the sector. Nevertheless, despite of the shortcomings mentioned above, the Project results-based approach addressing

control of water losses achieved remarkable success acknowledged by all the Project stakeholders. They

stated that the approach, beyond achieving the results pursued, established a practical and innovative

solution to one of the most complex and difficult challenges in the water industry worldwide.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

44. The M&E design. The M&E system was well defined. It was able to translate the Project

objectives and components into just a single simple indicator – cubic meters of water recovered. In

addition, the information provided under the “use of Project outcome information” column of the PAD,

Annex 3, constituted a clear guidance regarding the underlined message conveyed by the PDO and

Intermediate Outcome indicators. The intermediate results indicators were aligned with the Project

components and the systematic collection of data and periodic results tracking was an integral part of

Project implementation. The PMU capacity to undertake the M&E system requirements was appropriate

and was further strengthened by the VA procedures.

45. The M&E implementation and utilization. Implementation was straightforward given the

simplification allowed by the adoption of a single PDO indicator. Also, the combined contributions from

the management firm and the VA contributed to a smooth follow-up of the Project goals throughout

implementation, including timely and meaningful Project implementation reports. A study on the impact

evaluation part of the Project design was dropped in the November 2015 restructuring given the

difficulties associated with technical specifications embedded in the study, including finding a control


12

group. The study findings were expected to provide significant inputs to evaluating the Project against not

only its objective, but also against the numerous assumptions raised in the PAD. In the restructuring, two

new PDO indicator were added. In all, the M&E system was utilized as a tool to register the PDO and

Intermediate targets values and to guide managerial and operational decisions, in particular the inclusion

of two additional subproject selection processes and the new Component 4.

2.4 Safeguards and Fiduciary Compliance

Safeguards

46. Safeguards compliance. During implementation, the safeguards performance evolved from

Moderately Unsatisfactory, to Moderately Satisfactory, and finally to Satisfactory. The Project was an

Environmental Category B. It triggered the following safeguard policies: OP 4.01 Environmental

Assessment; OP 4.04 Natural Environment; OP 4.11 Physical Cultural Resources; and OP 4.12

Involuntary Resettlement. An Environmental Safeguards Framework defining criteria and procedures the

selected water and sanitation providers needed to observe when developing and implementing the

activities. In addition, a Resettlement Management Framework was prepared, although the activities

triggering the Resettlement policy were not eligible for participation in the Project. Both frameworks

were incorporated in the Project operational manual. The environmental procedures in place were

reviewed and few adjustments were made. In the November 2015 restructuring, Component 4 was added

to the Project. The detailed designs available for the activities incorporated under the new component

allowed for a detailed screening concerning the safeguards triggered. All designs complied with the

safeguard policies, including the resettlement policy. Nevertheless, as indicated before, none of the

activities under the new component were implemented.

47. OP 4.01 Environmental Assessment. In agreement with the Environmental Framework, the

activities selected for financing by the Project presented, as part of the mandatory documentation, an

environmental screening sheet. During the activity implementation, the procedures described in the

Construction Manual included in the Project operational manual were followed.

48. OP 4.04 Natural Habitat. This OP was triggered because there were environmental protected

and conservation areas within the five headwaters targeted by the Project. The activities financed by the

Project were screened to ascertain if any impact on these areas could occur. It was confirmed that none of

the activities the Project financed had impacts on these areas.

49. OP 4.11 Physical Cultural Resources. The Project was not expected to affect areas of historical

and cultural relevance. Nevertheless, the Environmental Framework incorporated chance-finding

procedures.

50. OP 4.12 Involuntary Resettlement. The activities triggering the Resettlement Policy were not

eligible for Project financing. Nevertheless, a Resettlement Framework was prepared and incorporated in

the mandatory documentation the selected activity had to comply with.

Financial Management

51. During implementation, financial management was rated as Satisfactory or Moderately

Satisfactory. Initially, there were delays in adjusting the State’s Administrative System to generate

Interim Financial Reports (IFRs) and the need to strengthen the internal control arrangements further.

These aspects improved during implementation. The PMU and the Management firm’s experience were


13

also important factors to ensure acceptable financial management arrangements throughout Project

implementation. Agreed action plans were generally implemented, but were not sufficient, to improve

disbursement, especially during the last years of the Project, as a result of the devaluation of the Brazilian

Real since 2015 and fiscal constraints. The financial management risk rating was considered Moderate

throughout the Project’s life. Audit reports were generally received on time (with the exception of the

2014 and 2016 reports). All audit reports expressed unqualified/unmodified audit opinions (with the

exception of the 2015 report, as the IFRs did not reconcile the opening balances). Although some IFRs

were received with delays, they were all considered acceptable. There were no instances of ineligible

expenditures identified.

Procurement

52. During implementation, procurement was rated Satisfactory or Moderately Satisfactory.

Procurement under the project was implemented at two levels. The PMU established within the SSRH

was responsible for procurement under Component 3, while water and sanitation service providers,

selected by the SSRH through the procedures informed by the launched CFP, were responsible for

procurement under Components 1 and 2 (following applicable Federal Law 8,666/93 with procedures

acceptable to the Bank). However, during Project preparation, both the Bank and the government teams

faced severe difficulties in agreeing upon appropriate legal and technical documents that were also

acceptable by eligible providers identified at that phase. Although it happened almost 10 years ago, the

teams involved still cannot forget the numerous and very difficult issues that had to be resolved to agree

on the final version of these documents. During Project implementation, the quality of the procurement

processes varied: while in some cases, the processes were well prepared and quickly implemented, in

other cases, they were very confused and slow. Clearly the quality of the procurement processes depended

on technical and political issues. The critical drought the State of São Paulo faced during Project

implementation contributed to the mixed performance. The Project team lacked previous experience with

Bank's procurement to select consulting services. This was a major issue at the start of Project

implementation (specially for Component 3). Gradually, it evolved to difficulties associated with

government administrative aspects and quality of technical documents. For the activities implemented

under Components 1 and 2, the selected providers had solid experience in addressing the national

procurement legislation requirements, and no remarkable issues had to be surmounted to increase the

provider’s capacity to comply with the procedures acceptable to the Bank. Despite these minor issues,

from a procurement point of view, the quality of the procedures followed by the Project was acceptable.

No major issue such as a mis-procurement or ineligible expenditure was declared or found.

2.5 Post-completion Operation/Next Phase

53. The Project results-based approach underpinned sustainability. Sustainability of the activities

under the subcomponent ‘control of water losses’ was verified by the VA, who confirmed that the agreed

targets and sustainability conditions had been met triggering the authorization for the last payment tranche.

Payments under the other two subcomponents addressing increasing water quantity as well as the three

subcomponents addressing water quality were scheduled as follow: 70 percent of total contract amount

paid during implementation, and 30 percent paid once compliance with sustainability targets was met.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (Pre- and Post-restructuring)


14

54. The relevance of objective is rated Substantial. The objective of Project was consistent with

the development priorities and circumstances at the time of Project preparation and implementation. The

FY08-FY11 Country Partnership Strategy (CPS), dated May 1, 2008, laid out a program of continued

support to Brazil through four pillars of engagement: equity, sustainability, competitiveness, and sound

macroeconomic management. The CPS asserted that Brazil continued to falter in the area of

environmental sustainability, and that water scarcity and environmental degradation were urgent problems

hindering the country’s sustainable growth. The FY12-FY15 CPS 25

, valid by Project closure, was also

structured around four strategic objectives: (i) increase the efficiency of public and private investments;

(ii) improve quality and expand provision of public services for low-income households; (iii) promote

regional economic development through strategic investments and policies; and (iv) and improve

sustainable natural resources management and climate resilience. The third and fourth strategic objectives

were highly relevant to the Project’s rational and objectives. The results areas under these strategic

objectives were also closely aligned with the programmatic engagement with the State of São Paulo;

namely, improved policy coordination at territorial level, expanded access to improved basic sanitation,

integrated water resources management, and improved environmental management.

55. In addition, the Program objective’s relevance to the current situation of the country still remains

high. It is consistent with FY18-FY23 Country Partnership Framework (CPF).26

The third priority area

identified in the CPF focuses on inclusive and sustainable development with the objective, among others,

to increase urban resilience and provide more sustainable and inclusive urban services. Promoting the

improvement of the quality of urban infrastructure, improving the efficiency of service delivery, and

building resilience of populations against the variability of water supply are among the key activities

proposed. In conclusion, the Project objective is still closely aligned with the CPS in place by the Project

closure and the CPF valid for the coming years. Also, the SSRH during a workshop carried out by

Project closure indicated that it intends to continue to apply the Project’s results-based approach as a

financing method for future sector programs. This was also confirmed in the client’s letter included in

Annex 7.

56. The relevance of design and implementation is rated Modest. The Project design and

implementation proved to be relevant, as well as timely, particularly in adapting innovative tools to

address water recovery through several activities including water loss and wastewater treatment. The

Project developed and tested these tools, which constitute a major contribution to the water sector

worldwide. Additional positive aspects include: (i) the choice of priority basins that were prone to

drought; (ii) the incorporation of lessons learned; and (iii) the focus on both implementation and

sustainability aspects. As discussed earlier in Section 2, there were some shortcomings during

implementation including: (i) limited appropriateness of the results-based approach for supporting capital

intensive activities such as large sewerage system investments; (ii) limited state of readiness; and, (iii) the

ambitious timeframe.

3.2 Achievement of Project Development Objectives

57. The Project’s PDO remained unchanged over the life of the Project. However, the PDO indicator

target was amended during the November 2015 restructuring and new PDO indicators were added. Still,

25 CPS report number 89498 - BR, FY2012—2015.In 2016, the World Bank launched the Brazil Systematic Country

Diagnostic (SCD) to inform the preparation of a new Country Partnership Framework (CPF). 26

In June 2017, the Board approved the Brazil’s CPF for FY2018-2023 —report number 113259-BR.


15

as the PDO indicator target value was not reduced but increased, a split evaluation was not considered

relevant.

58. PDO objective: To increase clean water availability in the State of São Paulo’s critical

watersheds. Rating: Modest

59. Achieving the PDO Objective — Pre-restructuring. Prior to the 2015 restructuring, the

achievement of the PDO Objective was measured by a single PDO-level indicator: Cubic meters of

recovered water. The original appraisal target was 30,1106,311 m3, which was expected to be achieved

through the implementation of the six subcomponents included under Components 1 and 2. By the fifth

year of implementation, the Project had achieved approximately 90 percent27

of the target. Though, this

achievement was met through the implementation of just one of the six-subcomponents, the

subcomponent that addressed control and reduction of real water losses, which responded for 97 percent

of the PDO-level indicator achievement. The implementation of the activities under the other

subcomponents contributed to only 3 percent.

60. Achieving the PDO Objectives — Post-restructuring. In the 2015 restructuring, the original

PDO-level indicator (cubic meters of water recovered) was retained, although its target was increased by

159 percent to 47,966,660 m3. Also, two new PDO-level indicators were added: direct project

beneficiaries, targeting 289,686 people; and female beneficiaries, targeting 51percent of direct project

beneficiaries.

(a) Regarding the original PDO-level indicator, the Project helped to recover 47,127,718 m3

against the revised target of 47,966,660 m3. The revised target was expected to be achieved

through the implementation of the five remaining28

subcomponents under Components 1 and

2, although the expected weight given to the subcomponent that addressed control and

reduction of real water losses increased from 30 percent to 63 percent. Nevertheless, by the

Project closure, once again this subcomponent largely responded to achieving the PDO-level

indicator by 96 percent.

(b) Regarding the direct Project beneficiaries, the Project directly benefited 97,373 people

against a target of 289,686 people (33 percent). The low achievement is explained by the

difficulties faced by the components tied29

to this PDO-level indicator.

(c) Regarding female beneficiaries, 51 percent of the direct Project beneficiaries were female.

Thus, the target was fully achieved.

61. With respect to the ‘clean’ water aspect of the PDO, the Project achieved 96 percent of the

revised target since the additional water recovered is also potable. In addition, Component 2 regarding

‘enhancing wastewater quality’ was also supposed to contribute to the ‘clean’ aspect. There is a strong

likelihood of achieving the results associated with Component 2 on ‘Enhancing Water Quality’. The

27 By the Project original closing date, November 30, 2015, the achievement of the PDO-level objective “Volume of

recovery water” target (30,106,311 m3) was 90 percent. The implementation of the activities under subcomponent

“Control and reduction of real water losses” had produced 26,569,790 m3 of recovered water, thus responding for 97

percent of the achievement concerning the PDO-level indicator. 28

The subcomponent on Reuse of Treated Wastewater was dropped during the November 2015 restructuring given

the lack of signed contracts under this subcomponent. 29

In the November 2015 restructuring, when the PDO-level indicator “Direct project beneficiaries” was added, it

was defined that the target would be achieved through the implementation of the activities under Component 2 and 4.


16

implementation of six subprojects under Component 2 continued after the Project closure, with five

having active contracts with the State. The five ongoing subprojects expect to result in 4.4 million m3 of

water recovered per year and are considered likely to be completed between 2018 and 2019. The

remaining subproject without a contract so far, a big WWTP in Campinas, is expected to result in 3.9

million m3 of water recovered per year, but its date of conclusion is uncertain at this point. Assuming that

these 6 subprojects are completed, the total result for Component 2 would reach 86 percent of the overall

goal in enhancing water quality.

3.3 Efficiency

62. The efficiency of the Project is rated Substantial. The ex-post economic analysis carried out,

which was based on a sample of subprojects, indicates that the Project had a positive return and positive

impacts. Nonetheless, the most positive results derived from the high performance of activities under

Component 1, Subcomponent 1.1 on Control and Reduction of Water Losses, which was responsible for

93 percent achievement of the PDO goal. Results for Subcomponent 1.2 on Rational Use of Water was

modest. The results for activities under Component 2 on Enhance Water Quality were also modest and

were heavily influenced by the opportunity cost of avoiding government-imposed fines for non-action.

63. At appraisal, a Cost-Benefit Analysis (CBA) was carried out using a sample of proposed and or

similar subprojects in representation of each subcomponent. A total of six cases were analyzed. The CBA

analysis conducted for the above set resulted in positive returns and an overall economic Internal Rate of

Return (IRR) of 65 percent. None of the initially evaluated subprojects eventually participated in the

Project.

64. Summarizing Project results at closure. A total of 23 subprojects were implemented and their

results verified (refer to Table 1 in Annex 2). In terms of results, subcomponent 1.1 outperformed the

other subcomponents with a 93 percent contribution to the PDO results indicator while receiving only 37

percent of the total Project payments.

65. The ex-post analysis of the Project followed the appraisal evaluation concept, and comprised the

analysis of a new set of representative sample subprojects for Components 1 and 2 that had been carried

out to full completion (including sustainability period) as shown in Table 1 below.

Table 1: Sample Subprojects Analyzed

Brief Description of activities Verified water

recovered

results (m3/year)

% of total

Project

Component

Component 1 Water losses

SANASA First and

Second Batches for

Campinas

Infrastructure replacement,

sectorization, pressure management,

active leakage control in Campinas

3,578,490 8

Component 1 Rational

Use of Water in SANASA

Environmental education, equipment

and works carried out in 200 schools.

240,244 70

Component 2 Wastewater

Transport and Treatment

Boituva

Wastewater transport and treatment

serving a target of about 25 percent of

the city

409,968 13

Total Verified Results 4,228,702


17

66. Costs for Component 1 were approximate, particularly for the SANASA rational water use

subproject, as several SANASA personnel were significantly involved throughout the subproject timeline

and their salaries were not accounted for. The amount of additional water recovered under Component 1

through water losses and rational use of water subprojects was valued at marginal cost (US$1.8/m3)

..

Benefits for wastewater transport and treatment in Boituva under Component 2 included: (i) the volume

of BOD and COD pollution loads removed; (ii) the beneficiaries value of improved sanitation; and (iii)

the burden of disease avoided. Other environmental benefits, like nitrogen and phosphorus removal were

not included. In addition, an opportunity cost, derived from fines to be imposed by the Public Ministry for

non-action for curving pollution, was used as a proxy for minimum benefits.

67. Table 2 below shows the results of the analysis. Since none of the initially evaluated subprojects

eventually participated in the Project, the economic IRRs at appraisal and completion are not comparable.


18

Table 2: Cost-Benefit Analysis Results

Subcomponent Benefits (NPV) Costs (NPV)

Results

IRR

(%)

B/C at 10%

Discount

Rate

Water Losses

SANASA

Life period 10 years

Measured as water

recovered at marginal

cost

Investment costs plus

coordination costs

63 5.6

Efficient Use of

water

SANASA


Measured as water


cost

Investment costs plus

SANASA coordination

and operational costs

14 1.37

Waste Water Boituva


Pollution Loads

removed

Beneficiaries valuation

Burden disease avoided

Opportunity cost

Actual investment costs 22 2.49

Wastewater with

opportunity cost

alone

Opportunity cost Actual investment

costs

16

68. The high return of the SANASA water losses subprojects is consistent with the sector experience

with these types of activities, and particularly, in this case the zones intervened had reasonably good

infrastructure and therefore getting the results required measures such as pressure management, leakage

control, and other activities that required moderate investments. Additional benefits not quantified derive

from the fact that there was a learning experience with new methodologies, and the establishment of

improved leakage control programs to the recipients, which are expected to increase and become

permanent.

69. Results of the water school programs were also positive. The analysis of benefits focused on the

water recovery aspect of the subproject and excluded those related to the environmental education part.

The training of students in efficient use of water represented a very successful program. Furthermore, the

program is currently expanding within Campinas and is expected to bring important benefits associated

with a long term rational use of water by the students and their families. In the case of SANASA,

efficiency was impacted by significant amount of internal manpower used, both for the establishment and

coordination and implementation of the Program. Future programs should benefit from this initial

learning curve and be more efficient.

70. The case of Boituva, costs included investments as well as operation and maintenance. An

additional analysis included benefits associated with the opportunity cost derived from high fines the

municipality would have been subjected to given a decree of the Public Ministry requesting the operators

to treat their pollution loads. By adding this benefit, the economic IRR increased from 16 percent to 22

percent. While the benefits were conservatively estimated, the relatively modest returns are not atypical in

these cases of WWTPs serving relatively small cities.

71. The Project was extended by 18 months to help respond to GESP request to help in addressing

the then ongoing and unprecedently drought crisis in the State of São Paulo. Despite all efforts from the

Bank and PMU teams, Component 4 was not implemented as expected due to fiscal and legal issues. The


19

extension period was important, though, to finalize ongoing Project activities and measure the

sustainability of completed subprojects.

72. The original Project cost was estimated at US$107.49 million. During restructuring, the Project

cost was reduced to US$95.91million, mainly due to: (i) the depreciation of the Real, which was BRL 2.0

per US$ in 2011 and BRL 4.0 per US$ in 2015, doubling the undisbursed loan portion; and (ii) the

cancellation of six contracts (out of 40 contracts) mostly due to the reduced financial capacity of many

operators unable to bear the Project’s financially demanding results-based disbursement mechanism.

73. At time of Project restructuring, good results of the water loss subprojects resulted in the increase

of the cost of Component 1 (even with the cancellation of the subcomponent related to water reuse) to

US$31.67 million (additional US$1 million). On the other hand, the cost of Component 2 faced a big

reduction due to number of delays in implementation, as: (i) many of the initial subprojects were dropped

from the Project as they were financed by other Federal programs; (ii) these investments take longer to be

implemented and to achieve results; and (iii) difficult financial conditions of the service providers have

impacted their capacity to invest. Resources allocated to Component 2 (US$37.12 million) were in

general, reallocated to Component 4 (US$30.65 million). The cost of Component 3 was also increased by

US$2.89 million due to the need to extend the contracts of the management firm and VA to monitor

Project implementation during the additional 18 months. Contingency funds in the amount of US$3.21

million were cancelled.

74. At time of Project closure, however, total Project cost went further down to US$47.40 million (a

reduction of US$48.51million) due to the limited progress on Component 2 (uncommitted funds

US$19.35 million) and nonexecution of Component 4 (US$30.65 million). On the positive note,

Component 1 progressed more than expected (additional US$1.33 million).

3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory

Table 3. Evaluation Summary

Dimension Rating

Relevance

Relevance of objectives Substantial

Relevance of design and implementation Modest

Efficacy

PDO: Increase the water availability in the

State of São Paulo’s critical watersheds.

Modest

Efficiency Substantial

Rating Moderately Unsatisfactory

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

75. The Project has almost entirely achieved its PDO target value, which was the recovery of

approximately 45 million m3 of water. This total water amount of water recovered corresponds to the

consumption of around 800,000 people per year. Thus, its contribution to social development is relevant,

given the restrictions in water availability faced by MRSP. The impact on poverty was not measured

given the Project focus on water recovery. Moreover, to benefit low-income population, it was concluded

that the results-based scheme must include technical specifications and reference unit prices appropriate

to design solutions suitable to the low-income areas.


20

(b) Institutional Change/Strengthening

76. The Project has significantly contributed to knowledge creation by testing a methodology to

finance the water and sanitation sector. It was highly successful in developing a methodology to address

water losses, as the financial incentive offered was attractive to the market and, moreover, induced the

selected water and sanitation providers to adopt a technical methodology leading to sound and sustainable

solutions. As such, it contributed to disseminate a practical and innovative solution to one of the most

complex and difficult challenges in the water industry worldwide.

(c) Other Unintended Outcomes and Impacts (positive or negative)

77. None.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

78. A Stakeholder Workshop, part of the Project ICR preparation, was held in May 2017. In total, 25

participants attended the workshop, including representatives from SSRH, SABESP and SANASA. The

strengths and weaknesses of Project preparation and implementation and key recommendations identified

by the participants are presented below.

79. Project preparation and implementation. The strengths identified by the participants were: (i)

adopting the results-based approach; (ii) focusing on the priority basins; (iii) requiring a sustainability

period; (iv) flexibility given to providers to select solutions leading to agreed results; and (v) the

payments to providers did not require a heavy bureaucracy. The weaknesses in turn included: (i) reference

unit prices not well calibrated; (ii) long delays in selecting and signing contracts with providers; (iii)

difficulties faced in formulating the set of normative; (iv) huge technical differences between activities

under Components 1 and 2, creating difficulties to be handled under the management arrangement; (v)

risks associated with the results-based approach transferred to the technical team; (v) the value of the

reference unit prices was too low and did not include adjustment to inflation; and (vi) it was difficult to

understand the Project operational procedures.

80. Recommendations. Main recommendations for improving performance identified by the

workshop participants included: (i) developing the results-based set of normative was an intense learning

process; (ii) applying the normative addressing control of water losses required prior training; (iii) the

normative addressing sewerage systems needed simplification; (iv) the normative addressing rational

water use was quite appropriate; (v) the water crisis resulting from the drought boosted the control of

losses activities and reduced the investments in sewerage; (vi) the Project implementation generated an

important database that can be used to improve the set of normative; (vii) getting the Project

implementation started was a very difficult task because many indispensable elements were not ready,

such as the contract template, the VA and the management team; and (viii) the less efficient service

providers required prior and more intense capacity building.

4. Assessment of Risk to Development Outcome Rating: Moderate

81. The risk to development outcomes is rated Moderate taking into account that the subcomponent

“Control of water losses” responds to almost 96 percent of the achieved PDO indicator target value. The

adopted methodology for the implementation of this subcomponent was widely acknowledged by the


21

Project stakeholders as a major success. Moreover, the Project results-based approach included a

sustainability period to ensure the results achieved were sustainable. In addition, some of the water and

sanitation providers that implemented the subcomponent activities have been ranked amongst the most

efficient in the State of São Paulo.30

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory

82. The Project was well anchored strategically; its preparation was supported by quality technical

work. The Project incorporated the lessons learned from pervious interventions. The Project’s objective

was well aligned with the Government’s sector priorities. The Project envisaged to properly test in a

wider context the effectiveness of the results-based mechanisms. However, the innovative results-based

approach the Project adopted required numerous new key tools indispensable for implementation. During

preparation, one of these tools (the set of normative) was developed, consisting of an exceptional

technical contribution to the sector. Other key tools, such as the contract template including its legal and

technical foundations and required clearances, and the VA and the Project management team, were only

available at the end of the second year of Project implementation. As described in Section 2, there was

limited readiness for implementation when the Project was approved. Also, the Project results-based

approach was based on various technical assumptions. Some of them could be questioned even with the

information available during preparation, such as the feasibility of implementing large investments on

sewerage systems supported by results-based financing and within the Project timeframe. The Project

implementation timeframe (4 years) as presented in the PAD including the sustainability period was

unrealistic. Furthermore, the complexity of the results-based approach was not perceived during

preparation. This was a recurrent issue raised by stakeholders during the Project-closing workshop.

(b) Quality of Supervision Rating: Moderately Unsatisfactory

83. The challenges facing supervision and how these were or were not addressed also emerged from

Section 2, as in the previous paragraph. The lack of readiness for implementation was most likely the

major issue the supervision team faced, and had no option but to follow all the mandatory and complex

technical, juridical and bureaucratic steps in order to have the needed implementation tools ready to start

Project implementation. The first two-implementation years were mostly dedicated to get the Project off

the ground. From concept to the first disbursement, it took the Project almost 50 months, largely longer

that the average 30 months faced by Projects in Brazil. During this phase, the supervision team provided

significant support to the client implementation team, by financing two Bank consultants, who had a

major role in developing the results-based approach during preparation. Their support to the client

implementation team was critical in overcoming technical difficulties emerging as the legal and

contractual aspects were being addressed. In addition, the Bank team held regular supervision missions

and constant additional teleconferences. There was candor in the ISR reporting and the fiduciary aspects

were managed well by the team.

30Ranking of WSS in Brazil (Ranking do Saneamento no Brasil, 2015); Instituto Trata Brasil and GO Associados.


22

84. The supervision team was less successful in achieving solutions for key aspects that delayed

Project implementation, such as the lack of an inflation correction mechanism in the contract template,

and ensuring that the Project funds were fully committed. During preparation (the first CFP) and three

times during implementation (second and third CFPs as well as in the November 2015 restructuring), the

Project funds were deemed fully committed. The Projections were not correct as the results show – by

Project closure only 58.52 percent of the loan proceeds were disbursed. Nevertheless, it must be noted

that events beyond the Project control such as the drought, alternative competing financing mechanisms,

financial constraints by providers, and fiscal limitation at the State level, were largely responsible for the

Projections failure. Also, the decision made of including an input-based financed component at

restructuring, carried out late in Project implementation, was seemingly not based on sound realism given

the difficulties inherent to the implementation of the investments targeted. Moreover, the reduction of the

loan financing percentage also made during the November 2015 restructuring increased implementation

problems. Nevertheless, a very successful decision was made during implementation: launching the third

CFP tailored for control of water losses activities. The providers engaged had been trained in previous

contracts, the activities were properly designed and implementation was carried out in a timely and

efficient manner.

85. Component 4 was not implemented due to the lack of counterpart funding by the Government.

This was a result of the financial crisis that the country and particularly the state of Sao Paulo was

experiencing. While the Bank team considered partial cancellation of the loan, there was a collective

agreement to see if the financial challenges could be surmounted.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory

86. The overall Bank performance is rated Moderately Unsatisfactory

5.2 Borrower Performance

(a) Government Performance Rating: Moderately Unsatisfactory

87. The Government initiative in asking for and supporting an innovative results-based Project was

noteworthy, as well as the support given during preparation when top quality technical knowledge

nationally available was brought to contribute in conceiving the Project results-based approach. Despite

the initial weaknesses with the counterparts, the government subsequently provided the support needed

for Project implementation. The support provided during the first two years of implementation was

particularly noteworthy. During this period, the Project-financed support was not yet available. The

government team responsible for the Project implementation was basically just one person, who managed

to resolve the highly complex issues that the Project preparation had left unresolved. The most significant

shortcoming by the government was the lack of counterpart funds for implementing Component 4 that

was added to the Project in the November 2015 restructuring. Despite the delay in complying with the

legal covenants: (i) Schedule II, Section B, Paragraph 1; and (ii) Schedule II, Section B, Paragraph 2,

overall, the covenants were largely complied with for the duration of Project implementation.

(b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory

88. The implementing agency played an outstanding role in getting the Project off the ground into

actual implementation. It included three different teams: the SSRH own team; a consulting company hired

to carry out the Project management activities; and another consulting company hired to carry out the

verification activities tied to the Projects’ results-based approach. Although the consulting companies


23

were on board by the end of the Projects’ second implementation year, both teams still had to get deeply

involved in concluding and correcting numerous details in the Project implementation tools. Their

dedication in moving the Project forward, as well as technical capacity was decisive. Nevertheless, they

went through quite a long learning curve and they acknowledged getting to fully understand the Project

along its implementation. They also acknowledged that there were aspects tied to the results-based

approach that were misunderstood during implementation, which required a painful redressing process.

Moreover, the Projections on funds disbursements developed by the Project management team were

recurrently rejected by reality.

(c) Justification of Rating for Overall Borrower Performance Rating: Moderately

Unsatisfactory

89. The overall Borrower performance is rated Moderately Unsatisfactory.

6. Lessons Learned

90. Results-based approach requires substantial effort during preparation. The formulation of

the Projects’ innovative results-based approach required a substantial effort in preparing the set of

normative in order to make them consistent and operational. Innovative financing schemes might trigger

legal aspects the legislation in place does not contemplate, requiring long and complex juridical approval

processes. Therefore, the preparation of a results-based operation must include the full development of

the set of legal and operative normative prior to project implementation

91. Results-based approach needs superior management and verification capacity as well as

technical assistance support to potential beneficiaries. The success of a WSS Project implemented

through results-based financing requires more complex preparation procedures and a superior

management and verification capacity. In addition, more than in a conventional Project, its success

depends on the management capacity of well-structured service providers. The implementation of a

results-based operation requires the support of a technical assistance component to assist the providers in

preparing proposals that will ensure consistency throughout the execution of the activity. The type of

technical assistance needed, depends on its technical, financial and managerial capacity. The Terms of

Reference, as well as the selection of documents to hire the VA must also be fully developed during the

preparation phase. The VA must be available to verify the baseline data that is required for signing the

results-based contract with the provider.

92. Results-based scheme to target the poor need suitable solutions. In order to benefit low-

income population, the results-based scheme must include technical specifications and reference unit

prices appropriate to the design solutions suitable to the targeted neighborhoods.

93. Simplification of verification tools and set of normative: According to statements from the

Project team and providers involved, the verification tools, as well as the set of normative needed to be

simplified. Also, the procedures undertaken by the VA needed to be provided with absolute clarity.

94. Timeframe is essential for results-based schemes to be attractive to providers. The results-

based proposal selection and contract signing must be fast. The long process discourages the operators

interested in participating and causes the proposals to become outdated. In order to speed up the

implementation of results-based activities, the selection process needs to prioritize proposals that

demonstrate more advanced implementation status from the engineering perspective, as well as regarding

compliance with the environmental legislation. It should also contemplate the availability of land, when


24

required, and seek the participation of well-structured providers complying with a defined management

capacity threshold. In addition, the results-based contract must include mechanisms adjusting the

reference unit prices to inflation, particularly, for those activities requiring long implementation periods.

The absence of rules that preserve the contract value might result in breaking the supporting financial

balance and its feasibility.

95. Complexity or long implementation interventions and sustainability conditions need to be

revisited in case replication of the Project is envisioned. Activities requiring long implementation and

complex compliance with the environmental legislation, such as the construction of WWTPs, are most

likely not good candidates for financing operations limited to a five-year implementation period because

the sustainability time might exceed the implementation period. In order to ensure solid sustainability

conditions, the sustainability period considered under the Project needed to be revised, in particular for

those activities that depended on deep cultural and institutional changes, as it is the case for the WWTPs.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies

96. Annex 7 provides the comments by the São Paulo State Governor on the draft ICRR. In general,

the Borrower appreciates the final assessment of the Project and acknowledges to have no disagreement

with the challenges and shortcomings in Project design and implementation identified in the report.

Considering the pilot nature of the operation and the external factors that impacted its implementation

such as the unprecedented water crises in the state and the economic crises, the Borrower feels that the

report has underestimated some of the ratings.

97. It is the Government’s view that the situation was not as severe as stated in the report to justify

the Moderately Unsatisfactory ratings in several aspects of the Project. Key aspects to take into account

include:

(a) The innovative design which impacted on overall Project costs.

(b) The several changes to the Project scope and the preparation of technical and normative

documents to guide contracting and implementation, which took considerable time and

impacted the time for effective implementation of the Project.

(c) The water crisis of unprecedented proportions that posed great risks to the São Paulo

Metropolitan Region, considering its significant urban and economic importance.

(d) The introduction of the exceptional operational measures in the supply system and the

physical interventions designed and executed extremely quickly to increase water production

and flexibility to transfer water between different production systems led to the postponement

of investments in sewage systems, which, strongly affected SABESP's participation in the

Project.

(e) The intensity of the economic crisis, with an impact on the taxes collected by the public

sector, hindered progress under Component 4, which was the main focus of the November

2015 restructuring.

(f) The appreciation of the US dollar and the absence of inflation correction measures in the

amounts paid by the Project to the providers under the results-based contracts impacted

negatively on Project implementation and disbursement.


25

98. The Borrower also acknowledges the excellent support provided by the Bank’s management and

technical supervision team. The letter highlights the openness of the supervision team to institutional and

technical innovations, the caliber and dedication of Bank staff, and the understanding and collaboration

shown during the water crisis faced by the state.

99. The Bank team has addressed the issues raised by the Borrower, and has provided clarifications

on the overall evaluation and ratings.

(b) Cofinanciers Not applicable.

(c) Other partners and stakeholders Not applicable.


26

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal

Estimate (USDM)

Disbursed by

11/30/15

restructuring

Restructuring

Estimate (USDM)

Actual (USDM)

Percentage

of Appraisal

Component 1: Increase water

availability 30.68 19.09 31.67 33.00 107

1.1: Controlling and reducing real

water losses 22.88 18.35 29.47 30.80 134

1.2: Rationalizing the use of

water 3.05 0.74 2.20 2.20 72

1.3: Reusing treated wastewater 4.75 — — — —

Component 2: Enhance water

quality 64.8 3.88 27.68 8.33 13

2.1: Building wastewater

collection networks 4.00 0.10 1.12 0.73 18

2.2: Building wastewater

transport systems 27.10 2.04 6.90 5.22 19

2.3: Upgrading/building new

wastewater treatment plants 33.70 1.74 19.66 2.38 7

Component 3: Develop

institutional and technical

capabilities

318.64 4.31 5.75 5.91 68

Component 4: Emergency

works for water supply — — 30.65 — —

Total Cost 104.12 27.28 95.75 47.24 45

Physical Contingencies 0.41 — — — —

Price Contingencies 2.80 — — — —

Total Project Costs 107.33 27.28 95.75 47.24 45

Front-end fee IBRD 0.16 0.16 0.16 0.16 100

Total Financing Required 107.49 27.44 95.91 47.40 45

(b) Financing

Source of Funds Appraisal

Estimate (USD M)

Disbursed by

11/30/2015

restructuring (USD M)

Restructuring

estimate (USD M)

Actual (USD M)

Percentage

of Appraisal

Borrower 42.99 4.84 31.41 9.66 22

International Bank for

Reconstruction and Development 64.49 22.60 64.49 37.74 58

31 Numbers in the PAD are distinct. On pages related to the project components description, the total cost for

Component 3 is presented as US$8.16 million. On Annex 5 of the PAD, Component 3 is presented with a total cost

of US$8.64 million.


27


28

Annex 2. Outputs by Component

1. Summarizing Project results at closure. A total of 23 subprojects had been carried out to conclusion and

with their results verified. In terms of results, subcomponent 1.1 on water losses, outperformed the other

subcomponents with a 93 percent contribution to the PDO results indicator.

Table 1: Results per Component

Components # of

Subproj

ect

Verified Results

(m3/year)

% Results

Sustainability

perspective

Reagua

Payments

($R mill)

Component 1

1.1 Water losses

1.2 Efficient Use in

schools

13

4

43,713,974.63

335,080.39

93

1

Estimated to remain

constant throughout

project lifetime

98.30

6.44

Total Comp. 1 17 44,049,055.00 94 104.74

Component 2

All subcomponents*

6

3,078,663.00

6

Will increase as

ongoing projects

culminate and design

capacity constructed

facilities is reached

23.96

Total 23 47,127,718.00 100 128.70

* At Project closure 8 projects were ongoing with expected results of 8,336,471 m3/year

Table 2: Results per Activities

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

Component 1: Increase water availability

Subcomponent 1.1: Control and reduction of real water losses

1 CODEN - Nova

Odessa

Infrastructure

replacement,

sectorisation,

pressure management,

active leakage

control

2.01 2.01 May/2012

May/2017 417,633 1,703,794 Closed

Closing 5 months extension

to December

2016. Additional 5 months

extension to May

2017.

1 SAAE -

Indaiatuba 10.86 10.86

May/2012

Aug/2016 2,514,253 2,997,089 Closed

Closing 10

months extension

to June 2016. Additional 3

months closing

extension to September 2016.

1

SABESP -

Campo Limpo/Varzea

Paulista

10.59 10.59 May/2012 Sep/2015

2,439,111 3,179,313 Closed

1 SABESP –

Franca 2.58 2.52

May/2012

Dec/2016 572,221 543,538 Closed


to April 2017.


29

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

1 SANASA -

Campinas 2.24 2.24

Dec/2011

Dec/2015 533,671 872,699 Closed


to November

2015.

3 SAAE -

Guarulhos

Sectorisation, pressure

management,

assets and leaks control

20.36 20.36 Jul/2014

Dec/2015 3,927,303 5,524,190 Closed

No amendment to

contract.

However, the payment was

delayed for 18

months until fiscal constraints

blocking the

provider financing were

cleared.

3 SABESP –

Botucatu 2.71 2.71

Jul/2014

Dec/2015 475,188 1,216,227 Closed

3

SABESP -

Francisco

Mourato

4.68 4.68 Jul/2014 Dec/2015

1,004,208 2,280,644 Closed

3 SABESP –

Itaquacetuba 10.87 10.87

Jul/2014

Dec/2015 2,216,376 2,167,087 Closed Paiement in 2016

3 SABESP –

Osasco 5.03 5.03

Jul/2014

Dec/2015 1,078,045 11,550,143 Closed Paiment in 2016

3 SABESP - São Bernardo do

Campo

Sectorisation,

pressure management,

assets and leaks

contro

11.22 11.22 Jul/2014

Dec/2015 2,405,047 8,241,641 Closed

3 SABESP – Tatuí 1.32 1.32 Jul/2014 Dec/2015

283,546 731,810 Closed

3 SANASA - Campinas

13.86 13.86 Jul/2014 Dec/2016

2,589,568 2,705,799 Closed

Subtotal subcomponent 1.1 98.33 98.27 20,456,170 43,713,975

Subcomponent 1.2 Rational water use

1 SANASA

Environmental

education, equipment and

works carried

out in 100 schools.

2.39 2.35 Dec/2011

Dec/2016 73,802 143,957 Closed


to August 2016.

Additional

closing 5 months

extension to

December 2016.

2 SANASA

Environmental education,

equipment and

works carried out in 100

schools.

2.48 2.39 Dec/2013

Dec/2016 53,722 96,287 Closed

Closing 12

months extension

to November 2016. Additional

6 months

extension to May 2017.


30

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

2 SAAE -

Indaiatuba


equipment and


schools.

1.13 1.12 Dec/2013

Aug/2016 19,814 32,163 Closed

Closing 6 months extension to June

2016. Additional

6 months extension to

December 2016.

2 SAAE -

Guarulhos


equipment and


schools.

0.69 0.58 Dec/2013

Aug/2017 35,708 62,673 Ongoing


to December

2017 to allow verifying

sustainability

targets, once fiscal constraints

blocking the

provider to receive payments

have been

cleared. The remaining

payments will be

100 percent counter-part

funds.

Subtotal subcomponent 1.2 6.69 6.44 183,046 335,080

Total Component 1 105.04 104.71 20,639,216 44,049,055

Component 2: Enhance quality of water

Contracts listed below includes activities under Subcomponent 2.1 Wastewater collection network; Subcomponent 2.2 Wastewater transport

system; Subcomponent 2.3 Upgrading existing and constructing new wastewater treatment plants

Component 2: Enhance

quality of water

Subcomponent 2.1 Wastewater collection network

Subcomponent 2.2 Wastewater transport system

Subcomponent 2.3 Upgrading existing and constructing new wastewater treatment plants

1 SABESP –

Conchas

Subcomponent

2.1: Construction

works creating

89 new active connections

0.13 0.13

Jun/12

Dec/2014

89 new

active

connections


Closed

Subcomponent

2.3 Build a wastewater

treatment plant

2.42 2.42 757,21 772,632

1 SABESP - Alumínio

Subcomponent

2.2: Wastewater pumping station,

forced

wastewater pipe and gravity

wastewater pipe

1.51 0.85 May/2012 May/2017

Treatment plant results accepted.

Closed

(operational)

Change in designs and closing 22

months closing

extension. Additional change

in design and

sustainability targets exclusion.


31

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

Subcomponent

2.3: Wastewater

treatment plan

built.

1.74 1.22 581,587 -

Closing 22 months extension to May

2017 and design

change modifying the treatment

solution. Original

contract amount was R$5,25M.

Additional

amendment

eliminated

sustainability

phase.

1 SABESP –

Boituva

Subcomponent

2.2: Gravity wastewater pipe

0.28 0.28

Jun/12

Dec/2016

Treatment plant results

accepted.

Closed

Closing 16 months

extension to Nov

2016. Additional extension to June

2017.

Subcomponent

2.3: Wastewater

treatment plan built.

1.16 1.16 448,704 409,968


to Nov 2016.

Additional 6 months extension

to June 2017.

1 SABESP -

Anhembi

Subcomponent:

2.2: Wastewater

pumping station,

forced

wastewater and

gravity

wastewater pipe lines

0.56 0.56

Jun/12

Dec/2016

Treatment plant results

accepted.

Closed


2017.but

concluded in December 2016

Subcomponent

2.3 Wastewater treatment plan

built.

0.46 0.46 142,502 279,269

Closing 20 months

extension to June

2017.

1 SABESP –

Araçariguama

Subcomponent

2.1: Construction works creating

102 new active

connections

0.16 0.00

Jun/12

Jun/17

102 new active

connections

-

Closed

(WWTP not yet

operational)


2017. Change

contract scope to comply with

closing date.

Subcomponent

2.2: Wastewater

pumping station

and forced wastewater pipe

0.31 0.00

Closing 24 months

extension to June 2017.

Scope reduction

excluding sewerage

transportation.

Subcomponent

2.3: Wastewater treatment plan

built.

5.17 2.00 722,477 -


2017. Additional

amendment reduces scope;

eliminate

sustainability and equipment.


32

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

1 SABESP –

Sarapuí

Subcomponent

2.1: Construction

works creating 0

(zero) new active connections

0.00 0.00

Jun/12 Jun/17

- -

Closed

(work not

operational)


2015, and

exclusion of 178 new connections.

Additional

amendment to close contract.

Subcomponent

2.2: Wastewater

pumping station, forced

wastewater pipe

and gravity wastewater pipe

4.45 2.48

Closed (not

operational)

Closing 24 months

extension and

scope reduction. Additional scope

change, including

eliminating monitoring

sustainability

targets and pumping stations.

Subcomponent

2.3: Wastewater

treatment plan built.

1.03 0.72 350,957

-

Closed

(WWTP not

yet operational)


to June 2017. Additional

amendment to

reduce scope eliminating the

sustainability

phase and

equipment install.

1 DAEV – Valinhos

Subcomponent

2.1: Construction

works creating


0.03 0.02

Jan/14 Aug/2017

18 new

active

connections


Closed

(works

operational)

Closing 20 months extension to May

2017. Additional

amendment to reduce contract

scope and cancel

sustainability targets.

Subcomponent

2.2: Gravity

wastewater pipe

0.16 0.11 110,108 - Closed (operational)

Closing 20 months

extension to May

2017. Second amendment to

reduce scope and

eliminate sustainability

phase.


33

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

2

SAAE -

Indaiatuba

(Região Norte)

Subcomponent

2.1: Construction

works creating 89 new active

connections

2.40 1.65

Dec/2013 Dec/2017

1250 new

active connections

822 new

active connections

Ongoing (works

already

operational)

Closing 24 months

extension to June

2017. Additional extension to

December 2017

and new connections

reductions to 1250.

Achievement of

sustainability

targets will be

verified counting on counterpart

funds exclusively.

Subcomponent

2.2: Build 3 wastewater

pumping station,

forced wastewater pipe

and gravity

wastewater pipe

4.00 2.37 410,884 - Ongoing

(operational)

Closing 18

months extension

to June 2017. Additional 6

months extension

to December 2017 and scope

reduction.

Sustainability targets will be

verified with

counterpart funds

support.

2 SAAE - Indaiatuba

(Buruzinho)

Subcomponent

2.2: Build 1 wastewater

pumping station,

forced wastewater pipe

and gravity

wastewater pipe

9.26 6.35 Dec/2013

May/2017 2,177,280 636,396 Closed


to December 2016.

Additional extension to May

2017.

2 SAAE - Sorocaba

(Bom Jesus)

Subcomponent


165 new active

connections

0.36 0.32

Dec/2013

May/2017

188 new active

connections

165 new active

connections

Closed

Closing 18 months extension to

September 2017.

Additional extension to

December 2017

Subcomponent

2.2: Gravity

wastewater pipe

0.02 0.02 46,656 46,656 Closed


to September

2016. Additional 13 months

extension.

2 SAAE - Sorocaba

(sorocabanos)

Subcomponent


262 new active

connections

0.50 0.10 Dec/2013

May/2017

262 new active

connections

71 new active

connections Closed

Closing 11 months extension to

September 2016.

Additional 6 months closing

extension.


34

Ca

ll f

or P

ro

po

sal

Service

Provider –

Municipality

Summary

contract scope

Contract

amount

(R$

millions)

Amount

paid

(R$

millions)

Contract’s

start and

end dates

Expected

results

(m3 water

recovered

per year)

Verified

results

(m³/ year)

Contract

status

Amendments to

contracts

Subcomponent

2.2: Gravity wastewater pipe

0.80 0.76 933,742 933,742 Closed

Closing 10 months

extension to September 2016.

Additional 18

months extension to December 2017.

Total Component 2 36.91 23.98 6.68 3.07

Total (Components 1 and 2) 141.95 128.69


35

Annex 3. Economic and Financial Analysis

1. During Project preparation, an ex-ante Cost-Benefit Analysis (CBA) was carried out mostly using a

sample of subprojects from an initial list of potential candidates that responded to a call for proposals. The

subprojects chosen represented the project subcomponents and included: two cases for subcomponent 1.1, Control

and Reduction of Water Losses; for subcomponent 1.2 – Rational use Water a sample of 348 schools for a similar

and at the time ongoing efficiency Program by SABESP; subcomponent 1.3 –Reuse of Treated Wastewater an

industrial activity which was using reuse; for Component 2 - Wastewater Collection and Treatment and two of the

potential candidate operators. A cost Benefit analysis was conducted for the above set together with a financial

and distributive analysis. All the individual cases turned out positive results with an overall an IRR of 65 percent

for the Project.

2. None of the initially evaluated projects eventually participated in the Project. Nonetheless, to approximate

the Project efficiency a similar approach was used at project completion now analyzing a new set of

representative sample projects that had been carried out to full completion (including sustainability period. Table

1 below depicts the results of the Project at closure, indicating that 23 subprojects had been carried out to

conclusion and with their results verified. In terms of results, subcomponent 1.1 – water losses, outperformed the

other subcomponents with a 93 percent contribution to the PDO results indicator.

Table 1: Results per Component

Components # of

Subproj

Verified

Results

(M3/yr)

% Results

Sustainability

perspective

REAGUA

Payments

($R mill)

Component 1

1.3 Water losses

1.4 Efficient Use in

schools

13

4

43,713,974.63

335,080.39

93

1

Estimated to remain

constant through project

lifetime

98.33

6.44

Total Comp. 1 17 44,049,055 94 104.74

Component 2

All subcomponents*

6

3,078,663

6

Will increase as

ongoing projects

culminate and design

capacity constructed

facilities is reached

23.96

Total 23 47,127,718 100 128.7

3. Table 1 above shows that the results were heavily influenced by the water losses subcomponent which

received 37 percent of the total Project payments. The table also suggests that the overall result was significant. A

quick assessment estimating one year benefits of subcomponent 1.1 at closure (valued at marginal cost of water

US$1.8/M3 would yield estimated benefits of US$78.7 million for just the first year, or almost twice as much as

the total project disbursements.

4. The ex-post analysis of the Project followed the appraisal evaluation concept, and comprised a Cost

Benefit Analysis of a sample of concluded subprojects that were chosen to represent Project Components 1 and 2

as shown in Table 2 below.


36

Table 2: Sample Subprojects Analyzed

Subcomponent Brief Description of activities Verified water

recovered

results

(M3/year)

% of total

Project

Component

Component 1 Water losses

SANASA First and

Second Batches for

Campinas

Infrastructure replacement,

sectorization, pressure management,

active leakage control in Campinas

3,578,490 8

Component 1 Rational

Use of Water in SANASA

Environmental education, equipment

and works carried out in 200 schools.

240,244 70

Component 2 Wastewater

Transport and treatment

Boituva

wastewater transport and treatment

serving a target of about 25% of the

city

409,968 13

Total Verified Results 4,228,702

Results of the CBA

5. Table 3 presents the results for the three analyzed subprojects.

Table 3: Cost-Benefit Analysis Results Benefits (NPV) Costs (NPV) Results

Subcomponent IRR% B/C at 10%

discount

Water Losses

SANASA

Life period 10yrs

Measured as water


price

Investment costs +

coordination costs

63 5.6

Efficient Use of water

SANASA


Measured as water


price

Investment costs +

SANASA coordination

and operational Costs

14 1.4

Waste Water

Boituva


Pollution Loads removed

Beneficiaries valuation

Burden disease avoided

Opportunity Cost*

Actual investment costs 22 2.4

Waste Water with

opportunity cost alone

Opportunity Cost Actual Investment

Costs

16

6. Component 1: Water Losses and Rational Use of Water in Schools in Campinas. The CBA analysis

for the two subprojects for Component 1 used different periods of analysis: water losses - 10 years, Rational use

of water – 20 years, and typical life time of these interventions. Costs for Component 1 were approximate,

particularly for the Campinas Rational Use subproject, as several SANASA personnel were significantly involved

part time. As for the benefits, for component 1 both water losses and the amount of additional water recovered

was monetized at marginal cost (US$1.8/m3).

7. The high returns of the SANASA water losses subproject is consistent with the sector experience with

these types of activities. Moreover, in this case getting the results required no major infrastructure investments

and only measures such as pressure management, leakage control, and other activities that require only moderate

investments were carried out. Additional benefits not quantified derive from the fact that there was a learning


37

experience and the establishment of a formal and improved leakage control Program which is expected to

continue indefinitely.

8. Results of the water school program analyzed was modest but positive. The analysis focused on the water

recovery aspect of the subproject and excluded the environment education part. The training of students in

efficient use of water, represented a very elaborate program, that is expanding within the Campinas and is

expected to bring important long-term benefits associated with a rational use of water by the students and their

families in the future. In the case of SANASA, though, given its innovative nature a costly preparation phase and

efficiency was impacted by a significant amount of internal manpower used, both for the establishment,

coordination and implementation of the Program. Future programs should benefit from this learned curve and be

more efficient.

9. Component 2: Enhanced Water Quality Wastewater. The CBA was carried out for a 25 years’ period.

Benefits that were calculated and monetized include: (i) volume of BOD and COD pollution loads removed; (ii)

beneficiaries value of improved sanitation; and (iii) burden of disease avoided. Other environmental benefits, like

Nitrogen and Phosphorus removal were not included. In addition, an opportunity cost, derived from defined fines

to be imposed by the Public Ministry for non-action for curving pollution, was used as a proxy for minimum

benefits. Table 3 above shows the results of the analysis.

10. For the case of Boituva, costs included investment, as well as operation and maintenance. An additional

analysis included an opportunity cost, reflecting fines the municipality was subject to, by a decree of the Public

Ministry, requesting the treatment of pollution loads. This was also included in the original analysis carried out by

the Project. Adding this as a benefit to the IRR becomes positive and a similar situation was found with the case

of Conchas, another concluded subproject. While the benefits were conservatively estimated, the low returns are

not atypical in these cases with WWTP serving relatively small cities.


38

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty

Lending

Susana Amaral Financial Management Specialist GGODR Financial

Management

Martin P. Gambrill Lead Water and Sanitation Spec GWADR Water and Sanitation

Juliana Menezes Garrido Sr Water & Sanitation Spec. GWADR Water and Sanitation

Karina de Souza Marcelino Program Assistant LCC5C Operations Support

Jose Alexandre Monteiro Fortes Environmental Spec. Consultant LCSUW-HIS Environmental

Specialist

Pery Nazareth Water and Sanitation Engineer, consultant GWA04 Water and Sanitation

Engineer

Airton Sampaio Water Loss Consultant GWA04 Water Loss Engineer

Mauricio Ludovice Wastewater treatment Consultant GWA04 Wastewater

Treatment Engineer

Luz Maria Gonzalez Financial Economic Spec. Consultant GWA04 Economic analysis

Luiz Tineo Procurement Specialist GPOBA Procurement

Julia Tierney Junior Professional Associate GWA04 Analyst

Marta Elena Molares-Halberg Lead Counsel LEGES Lawyer

Luis R. Prada Villalobos Senior Procurement Specialist GGODR Procurement

Maria Angelica Sotomayor Araujo Lead Specialist GSURR

Carlos E. Velez Lead Economist LCSWS - HIS Task Team Leader

Adriana M.G.M. Weisman Operations Officer OPSPQ Operations

Supervision/ICR

Sinue Aliram Procurement Specialist GGODR Procurement

Etel Bereslawski Procurement Specialist GGO08 Procurement

Susana Amaral Financial Management Specialist GGODR Financial

Management

Lizmara Kirchner Sr Water & Sanitation Spec. GWADR Former co-Task

Team Leader

Soraya Melgaço Social Specialist consultant GSU01 Social Specialist

Jose Alexandre Monteiro Fortes Environmental Spec., consultant LCSUW-HIS Environmental

Specialist

Menahem Libhaber Water and Sanitation Engineer, consultant GEN04 Water and Sanitation

Pery Nazareth Water and Sanitation Engineer, consultant GWA04 Water and Sanitation

Airton Sampaio Water Loss Consultant GWA04 Water Loss

Oscar Alvarado Sr Water & Sanitation Spec. GWA04 Task Team Leader

Perla Virginia Castillo Miranda E T Temporary LCSUW-HIS ACS

Juliana Menezes Garrido Sr Water & Sanitation Spec. GWADR Current co-Task

Team Leader


39

Catarina Isabel Portelo Senior Counsel LEGLE Lawyer

Julia Tierney Junior Professional Associate LCSUW-HIS Analyst

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel

and consultant costs)

Lending

FY08 4.63 56,602.05

FY09 41.46 246,533.06

FY10 13.24 89,585.00

Total: 59.33 392,720.11

Supervision/ICR

FY10 1.25 5,055.48

FY11 18.6 126,861.17

FY12 16.66 96,881.52

FY13 6.36 56925.29

FY14 6.71 91,604.44

FY15 12.33 82,290.44

FY16 19.68 99,535.03

FY17 14.44 130,131.51

FY18 1.35 19,981.12

Total: 97.38 709,266.00


40

Annex 5. Beneficiary Survey Results

Not applicable.


41

Annex 6. Stakeholder Workshop Report and Results

1. A Stakeholder Workshop was held in May 2017, as part of the Implementation Completion Report

preparation. In total, 25 participants attended the workshop, including representatives from SABESP and

SANASA. The workshop main findings are presented below. The Workshop Report is available in the Project

files.

2. Project preparation and implementation (a) Strengths: (i) adopting the results-based approach; (ii)

focusing on the priority basins; (iii) requiring a sustainability period; (iv) flexibility given to providers to select

solutions leading to agreed results; (v) the payments to providers did not require heavy bureaucracy; (b)

Weakness: (i) Reference unit prices were not well calibrated; (ii) long delays in selecting and signing contracts

with providers; (iii) Difficulties faced in formulating the set of normative; (iv) huge technical differences between

activities under component 1 and 2, creating difficulties to be handle under the management arrangement; (v)

risks associated with the results-based approach transferred to technical team; (v) the value of the reference unit

prices was too low and did not include adjustment to inflation; and (vi) it was difficult to understand the Project

operational procedures;

3. Main findings: (i) developing the results-based set of normative was an intense learning process; (ii)

applying the normative addressing control of water losses required prior training; (iii) the normative addressing

sewerage systems needed to be simplified; (iv) the normative addressing rational water use was quite appropriate;

(v) the water crisis resulting from the drought boosted the control of losses activities and reduced the investments

in sewerage; (vi) the Project implementation generated an important database that can be used to improve the set

of normative; (vii) getting Project implementation started was a very difficult task because many indispensable

elements were not ready, such as the contract template, the verification agent and the management team; and (viii)

the less efficient service providers needed prior and more intense capacity.


42

Annex 7. Summary of Borrower's Comments on ICR

This Annex includes the comments by the Borrower on the draft Implementation Completion and Result Report.

It is a translation of the letter sent to the Country Director. The original letter is included in the Project files.

Comments by the State of São Paulo Governor on the Implementation Completion and Results Report of the Loan

7662-BR of the Reagua Program

Mr. Director of the World Bank:

Our technical team has examined the Final Report on the Reágua Project, object of Loan Agreement 7662-BR,

sent to this Secretariat by the World Bank. This last formal assessment of the Program is very significant, in that it

must lay out, realistically, the lessons derived from the set of actions carried out under the Project. These include

concepts, technical aspects of greater significance and the influences of externalities that have greatly interfered

with the program's execution. We hope that a balanced assessment will lay the groundwork for Reágua to inspire

similar initiatives in the future, after it has undergone the necessary adjustments. To us, this is not just desirable -

it is essential.

In general, we have no significant disagreements with the points presented in the Report, which accurately and

precisely depicts the characteristics and conditions of the Program's design and implementation. It duly mentions

the challenges faced by the program since its very inception, such as the enormous efforts required to conciliate

Reágua's innovative ideas with the limitations set forth by law; the restrictive conditions imposed by a water crisis

that has afflicted the State of São Paulo - especially its main metropolitan region - since late 2013; the virtual

impossibility of securing counterpart funds from the State Government since the first quarter of 2016, as a result

of the Brazilian macroeconomic crisis and its effects on public finances. The Report is also correct in noting the

significant success of Reágua's Component 1, involving actions to control losses and promote the rational use of

water in public schools. It is correct as well concerning the Component 2, focused on sewage, subpar performance.

As the report aptly states, the water crisis prompted sanitation companies to relocate investment funds to

emergency interventions meant to ensure water supply.

Our assessment, however, is that the situation was not as severe as stated in the report, which rates several Project

aspects as "moderately unsatisfactory". As we broadly agree with the general content of the arguments, however,

the following comments will refer to the significant difficulties faced by Reágua due to the Project's very nature

and the externalities affecting execution.

The Project faced the costs associated with innovation- the report mentions that this was the Bank's inaugural

operation in the water and sanitation sector based on a payment-for-results scheme. Reágua's scope was changed

several times since the Project was first conceived; there was a long-time gap between the time when the final

scope was set and the preparation of technical and normative documents to guide contracting and implementation;

as mentioned earlier, these new and positive ideas had to be expressed in a judicially-sound format, which took

considerable time. This Secretariat and the World Bank worked tirelessly to find adequate solutions - which they

eventually did; nevertheless, the effective time for implementing Reágua was shortened considerably.

As widely reported, we faced a water crisis of unprecedented proportions that posed great risks to the São Paulo

Metropolitan Region, considering its significant urban and economic importance. These risks were only

circumvented by the organized and rigorous actions carried out by Sabesp, which included financial measures

meant to reduce consumption, exceptional operational measures in the supply system and physical interventions

designed and executed extremely quickly to increase water production and afford greater flexibility to the

structures used to catch and transfer flows between different production systems. The allocation of funds to


43

interventions in water systems - including the timely agreement with the World Bank to change the scope of the

Mananciais Program and enable interconnection works between the Billings and Taiaçupeba dams, the expansion

of the Alto da Boa Vista Treatment Station and interventions to control losses in distribution networks - led, in

turn, to the postponement of investments in sewage systems, which, in turn, strongly affected Sabesp's

participation in the Reágua Project. A similar problem delayed the work schedule of the company in charge of

sanitation services in Campinas (Senasa), at its Boa Vista Treatment Station.

Finally, the intensity of the economic crisis, with an impact on the taxes collected by the public sector, hindered

the development of Reágua's Component 4, the object of the contract restructuring process carried out in late 2015.

This restructuring, another testament of the strong relationship between the State Government and the World

Bank - I am referring here to its Director in Brasilia and the technical staff involved in managing the Program -

was meant to take advantage of the funds made available under the Loan Agreement due to the cancellation of

Component 2 contracts, the appreciation of the US Dollar exchange rate, and the absence of inflation correction

measures in the amounts paid by Reágua to the providers under the results based contracts.

Together, these events and challenges greatly affected the Project's disbursements - however, given the nature of

the payment-for-results model, the Projects’ total budget was not expected to be fully disbursed. In our

assessment, the amounts disbursed would certainly have been very different under less-dramatic circumstances,

with better performances in Component 2 and, above all, Component 4. A better rating would seem to be in order,

considering these restrictive events.

We have brief observations about two points that, inter alia, will require attention if similar programs are

implemented in the future:

First, actions to control physical losses in water systems and in wastewater treatment investments are

quite particular and have very distinct needs when it comes to resource mobilization and contract terms,

making it difficult to include them all under the same project. Nevertheless, both actions deserve support

and are fully compatible with the payment-for-results system.

Second, regarding measures to control water losses: Reágua was developed in partnership with the best

operators in São Paulo, which contributed to its success in this area. Future projects must include strong

action to support operators as they advance towards higher management and operational standards.

We would like to take this opportunity to thank the World Bank, its management and technical team in charge of

supervising the Reágua Program, and underscore their openness to institutional and technical innovations, the

caliber and dedication of its professionals, and the understanding and firm cooperation shown by the staff at

decisive junctures in the water crisis we had to face.

Yours faithfully.


44

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

Not applicable.


45

Annex 9. List of Supporting Documents

1. Project Appraisal Document (PAD), dated March 26, 2010.

2. Guidelines for Reviewing World Bank Implementation Completion and Results Report, Independent

Evaluation Group (IEG), November 12, 2013

3. Implementation Completion and Results Report, Guidelines, OPCS, updated on July 22, 2014

4. Stakeholders Workshop Report, May 2017

5. Implementation Supervision Reports (numbers 1 – 13)

6. Bank Supervision Aide Memoires

7. REAGUA Project – Concept Review Minutes

8. REAGUA Project – Quality Enhancement Review (QER) Minutes

9. REAGUA Project – Decision Meeting Review Minutes

10. REAGUA Project – November 2015 Restructuring Paper

11. REAGUA Project – Loan Agreement, dated September 27, 2010

12. REAGUA Project – Amendment to the Loan Agreement, dated November 30, 2015

13. REAGUA Project – Set of Normative supporting the Results-Based Approach

14. REAGUA Project – Progress Reports from Implementing Agency

15. Brazil: Inputs for a Strategy for Cities: A Contribution with a Focus on Cities and Municipalities, World

Bank Report No. 35749 (June 2006).


46

Annex 10. Maps

World Bank Documentdocuments.worldbank.org/curated/en/778331512661086878/pdf/ICR-P... · FEHIDRO...

Documents

Transcript of World Bank Documentdocuments.worldbank.org/curated/en/778331512661086878/pdf/ICR-P... · FEHIDRO...