INTERNATIONAL DEVELOPMENT ASSOCIATION Public … · 2020. 6. 2. · 06 12-Jun-2015 Moderately...

FOR OFFICIAL USE ONLY

Report No: ICR00004947

INTERNATIONAL DEVELOPMENT ASSOCIATION

IMPLEMENTATION COMPLETION AND RESULTS REPORT

ON A

CREDIT IN THE AMOUNT OF SDR 131.5 MILLION

(USD 200 MILLION EQUIVALENT)

WITH A GRANT

FROM THE AUSTRALIA DEPARTMENT OF FOREIGN AFFAIRS AND TRADE IN THE AMOUNT OF USD 6.04 MILLION

TO

THE SOCIALIST REPUBLIC OF VIETNAM

FOR THE

RESULTS-BASED RURAL WATER SUPPLY AND SANITATION UNDER THE NATIONAL TARGET PROGRAM

PROGRAM-FOR-RESULTS

January 29, 2020

Water Global Practice

East Asia And Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective July 31, 2019)

Currency Unit = VND

VDN 23212.09 = US$ 1

US$ 0.727052 = SDR 1

FISCAL YEAR

July 1 - June 30

Regional Vice President: Victoria Kwakwa

Country Director: Ousmane Dione

Regional Director: Benoit Bosquet

Practice Manager: Sudipto Sarkar

Task Team Leader(s): Lilian Pena Pereira Weiss, Hung Sy Pham

ICR Main Contributor: Alek Cannan

ABBREVIATIONS AND ACRONYMS

AusAID Australian Agency for International Development

CRR Comprehensive Results Report

CWS Community-Wide Sanitation

CORWSP Coordination Office for Rural Water Supply and Sanitation Program

DARD Department of Agriculture and Rural Development

DLI Disbursement-linked Indicators

DoH Department of Health

FSM Fecal Sludge Management

FWSC Functioning Water Supply Connections

GoV Government of Vietnam

GPOBA Global Program for Output Based Aid

ICR Implementation Completion Report

IEC Information, Education and Communication

IPF Investment Project Financing

IVA Independent Verification Agent

M&E Monitoring and Evaluation

MARD Ministry of Agriculture and Rural Development

MoF Ministry of Finance

MoH Ministry of Health

MPI Ministry of Planning and Investment

MTR Mid Term Review

NCERWASS National Centre for Rural Water Supply and Sanitation

NTP/NTP2/NTP3 National Target Program/ National Target Program for Rural Water Supply and Sanitation/Phase 2 / Phase 3

NRD National Target Program on New Rural Development for 2010-2020

O&M Operations and Maintenance

PAD Project Appraisal Document

PAP Program Action Plan

PCERWASS Provincial Centre(s) for Water Supply and Sanitation

PDO Program Development Objective

PIU Project Implementation Unit

PforR Program for Results

POM Program Operations Manual

PPC Provincial People’s Committee

RRD Red River Delta

RWSS Rural Water Supply and Sanitation

RWSSP Rural Water Supply and Sanitation Project

SAV State Audit of Vietnam

UTA Technical Assistance

VBSP Vietnam Bank for Social Policies

VIHEMA Vietnam Health Environment Management Agency

VND Vietnamese Dong

WASH Water Access, Sanitation, and Hygiene

WSS Water Supply and Sanitation

TABLE OF CONTENTS

DATA SHEET .......................................................................................................................... 1

I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES .................................................... 5

A. CONTEXT AT APPRAISAL AND THEORY OF CHANGE ......................................................................... 5

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ....................................................................... 9

II. OUTCOME .................................................................................................................... 11

A. RELEVANCE ..................................................................................................................................... 11

B. ACHIEVEMENT OF PDOs (EFFICACY) ............................................................................................... 12

C. JUSTIFICATION OF OVERALL OUTCOME RATING ............................................................................ 20

D. OTHER OUTCOMES AND IMPACTS ................................................................................................. 21

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 22

A. KEY FACTORS DURING PREPARATION ............................................................................................ 22

B. KEY FACTORS DURING IMPLEMENTATION ..................................................................................... 23

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25

A. QUALITY OF MONITORING AND EVALUATION ............................................................................... 25

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ........................................................... 25

C. BANK PERFORMANCE ..................................................................................................................... 26

D. RISK TO DEVELOPMENT OUTCOME ............................................................................................... 27

V. LESSONS AND RECOMMENDATIONS ............................................................................. 27

. 29

ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN ...................................................................................................................... 30

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 44

ANNEX 3. PROGRAM EXPENDITURE SUMMARY ................................................................... 46

ANNEX 4. BORROWER’S COMMENTS ................................................................................... 47

ANNEX 5. SUMMARY OF TECHNICAL ASSISTANCE IMPLEMENTATION AND COMPLETION REPORT ............................................................................................................................... 53

The World Bank Results-Based Rural Water Supply and Sanitation Under the National Target Program (P127435)

Page 1 of 61

Click here to enter text.

DATA SHEET

BASIC INFORMATION

Product Information

Program ID Program Name Financing Instrument

P127435

Results-Based Rural Water Supply and

Sanitation Under the National Target

Program

Program-for-Results Financing

Country IPF Component

Vietnam No

Organizations

Borrower Implementing Agency

State Bank of Vietnam

PIU under VIHEMA, NCERWASS, Rural Water Supply and

Sanitation Partnership Coordination Unit, State Audit of

Vietnam – Specialized Audit Department No. V,

Coordination Office for Rural Water Supply and

Sanitation Program (CORWSP)

Program Development Objective (PDO) Original PDO

Increase sustained access to water supply and sanitation services and improve sector planning, monitoring and evaluation in the Participating Provinces of the National Target Program for Rural Water Supply and Sanitation.


Page 2 of 61

FINANCING

FINANCE_T BL

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Administered Financing IDA-51760

200,000,000 200,000,000 170,739,102

TF-13061

6,042,000 4,787,901 4,945,001

Total 206,042,000 204,787,901 175,684,103

Non-World Bank Administered Financing

Borrower/Recipient 60,000,000 27,000,000 27,000,000

Total 60,000,000 27,000,000 27,000,000

Total Program Cost 266,042,000 231,787,901 202,684,103

KEY DATES

FIN_TABLE_DATA

Program Approval Effectiveness MTR Review Original Closing Actual Closing

P127435 01-Nov-2012 24-Apr-2013 15-Sep-2015 31-Jul-2018 31-Jul-2019

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

16-Jun-2016 87.52 Change in Results Framework Change in Loan Closing Date(s) Reallocation between and/or Change in DLI Change in Disbursements Arrangements Change in Implementation Schedule Other Change(s)

KEY RATINGS

Outcome Bank Performance M&E Quality

Highly Satisfactory Highly Satisfactory Substantial


Page 3 of 61

RATINGS OF PROGRAM PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 23-Jan-2013 Satisfactory Satisfactory 0

02 15-Sep-2013 Satisfactory Satisfactory 18.01

03 26-Mar-2014 Satisfactory Satisfactory 18.45

04 12-Oct-2014 Satisfactory Moderately Satisfactory 50.71

05 25-Dec-2014 Satisfactory Moderately Satisfactory 57.55

06 12-Jun-2015 Moderately Satisfactory Moderately Satisfactory 58.62

07 03-Dec-2015 Moderately Satisfactory Moderately Satisfactory 86.88

08 18-May-2016 Moderately Satisfactory Moderately Satisfactory 87.52

09 02-Nov-2016 Satisfactory Satisfactory 124.55

10 01-May-2017 Satisfactory Satisfactory 124.45

11 01-Aug-2017 Satisfactory Satisfactory 124.45

12 24-Jan-2018 Satisfactory Satisfactory 145.90

13 25-Jul-2018 Satisfactory Moderately Satisfactory 146.28

14 29-Jan-2019 Satisfactory Satisfactory 173.76

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Water, Sanitation and Waste Management 100

Sanitation 50

Water Supply 50

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%)


Page 4 of 61

Urban and Rural Development 100

Rural Development 100

Rural Infrastructure and service delivery 100

ADM STAFF

Role At Approval At ICR

Regional Vice President: Pamela Cox Victoria Kwakwa

Country Director: Victoria Kwakwa Ousmane Dione

Director: John A. Roome Benoit Bosquet

Practice Manager: Jennifer J. Sara Sudipto Sarkar

Task Team Leader(s): Parameswaran Iyer, Hoa Thi Hoang

Lilian Pena Pereira Weiss, Hung Sy Pham

ICR Contributing Author: Aleksander Mack Cannan


Page 5 of 61

I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL AND THEORY OF CHANGE

Context 1. Vietnam has had an impressive record of inclusive economic growth over the past two decades, with annual gross domestic product growth averaging about 6.4 percent over this period.1 Furthermore, growth has gradually been more inclusive (economically and in broader welfare terms). Vietnam has also achieved impressive poverty reduction. Forty million people escaped poverty over the last thirty years.2 Notwithstanding these achievements, development challenges persist. Poverty gains are fragile, with remaining poverty concentrated in rural areas and among ethnic minorities. Most of the poor—9 out of 10—live in rural areas and access to basic services in rural areas is very low.3 Ethnic minorities make up only 15 percent of the population, yet account for 60 percent of the poor.4 At the same time, natural assets are unsustainably exploited, institutions and planning remain weak, needed investments for infrastructure exceed resources available, and mobilization of private financing is challenging. Public debt is also a concern.

2. In response to these challenges, the Government of Vietnam (GoV) shifted priorities to address sustainability, quality, and equity of growth. Vietnam’s Five-Year Socio-Economic Development Plan (2011-2015)5 prioritized reducing rural poverty and increasing access to basic services. An example of GoV commitment is the National Target Program6 on New Rural Development for 2010-2020 (NTP/NRD), which directs investment into rural areas for physical and social infrastructure with 19 development indicators in different sectors, including rural water supply and sanitation (RWSS).

3. Access to safe water supply and sanitation (WSS) services in rural Vietnam has been a significant challenge. For water supply, it was estimated in 2010 that around 75 percent of the rural population had access to ‘improved’ water, however access to ‘clean’ water, defined by national water quality standards, was estimated to be only 35 percent.7 Where surface-water treatment and distribution networks were used, there were often significant management challenges, including unsustainable operations and inadequate attention to institutional arrangements and cost recovery. A study8 by the Ministry of Agriculture and Rural Development (MARD) showed that, because of weak management, poor planning, and design inconsistent with demand, 25 percent of rural water schemes in Vietnam on average, and up to 71 percent in certain provinces, had inefficient operations or stopped working altogether. Countrywide, only 33 percent of schemes function sustainably. Meanwhile, rural access to hygienic and properly maintained sanitation facilities was estimated to be 51 percent in 20109; and many of the remaining unserved are those in hard-to-reach areas who lack financial resources to afford hygienic latrines.10

4. The GoV has been proactive in responding to the challenge of increasing access to and ensuring the sustainability of rural water supply and sanitation (RWSS) services. The National Rural Clean Water Supply and Sanitation Strategy to 2020 (RWSS National Strategy), developed in 2000 and updated in 2011, set the overall vision and goals for the sector. The key principles of the strategy include community participation, sustainability and cost recovery;

1 World Bank analysis, cited in: “Vietnam: Toward a Safe, Clean, and Resilient Water System”, IBRD/World Bank (2019). 2 Data source: General Statistics Office of Vietnam, with World Bank staff analysis. Cited in: Bank Vietnam CPF FY18–FY22 (Report No. 111771-VN). 3 ibid. 4 ibid. 5 http://www.chinhphu.vn/portal/page/portal/English/strategies/strategiesdetails?categoryId=30&articleId=10057591. 6 From 2000 to 2015 the GoV had the sector-specific National Target Programs (NTP1, NTP2 and NTP3) focused on RWSS. In 2015 these evolved

into the broader NTP/NRD, encompassing other rural development sectors, in addition to RWSS. 7 National Target Program on Rural Water and Sanitation 2012-2015 document, approved by Prime Minister’s Decision 366 (2010). 8 Results and Lessons Learnt: Target Program Budget Support to NTP on RWSS 2011-2015. MARD, RWSS Partnership Coordination Unit, 2015. 9 National Target Program on Rural Water and Sanitation 2012-2015 document, approved by Prime Minister’s Decision 366 (2010). 10 Hygienic latrines ensure total isolation of human feces and are capable of eliminating pathogens, without creating foul smells or

environmental contamination (Vietnam National Technical Standards on Latrines (QCVN 01:2011/BYT; Circular no. 27/2011/TT-BYT; 2011).


Page 6 of 61

with emphasis also on poor communities, ethnic minority groups, and remote areas. To implement and achieve the objectives of the RWSS National Strategy, the GoV in 2000 established the National Target Program for Rural Water Supply and Sanitation (NTP).11 The NTP operates across all 63 provinces in Vietnam and is implemented as a series of five-year phases. The NTP has made considerable progress over the past 20 years. During the NTP Phase 2 (NTP2; 2006-2010) for instance, hygienic water services were upgraded for an estimated 5.4 million rural people and sanitation services for 6.8 million rural people. Total expenditures were approximately US$ 1 billion.

5. In 2011, the GoV designed the third phase of the NTP (NTP3), to be implemented starting in 2012. An assessment of the NTP identified key challenges to be resolved, including: (i) insufficient attention to household and institutional sanitation (sanitation coverage was lower than for water supply); (ii) lack of financial sustainability of water supply systems (due to low demand for piped water, low household connection rates, and unwillingness to pay for piped water services); (iii) weaknesses in the budget planning and expenditure frameworks; (iv) investment resources were spread too thin; (v) system design and cost inefficiencies; and (vi) governance weaknesses (especially fiduciary and environmental/social practices). NTP3 in particular sought to improve the sustainability of water systems and the quality of the water produced and distributed. NTP3 also emphasized full cost recovery in operations and maintenance (O&M). To further push the sanitation agenda, NTP3 oriented institutions to enhance promotion, education, and marketing for sanitation and hygiene. NTP3 was structured around four interventions: i) rural domestic water supply and environment; ii) rural sanitation; iii) capacity building and monitoring and evaluation (M&E); and iv) Information, Education and Communication (IEC). By its closure, the NTP3 had provided hygienic water to 6.1 million people, clean water to 5.1 million people, household latrines to 2.1 million households, and sanitation facilities in 2,940 schools and 740 health clinics.12

Rationale for PforR Support, and Program Scope and Boundaries 6. The World Bank has supported the RWSS sector in Vietnam for over 20 years. In 2000, the Bank assisted with the development of the RWSS National Strategy. In 2005, the Bank supported financing and implementation of the Red River Delta Rural Water Supply and Sanitation Project (RRD-RWSSP) (P077287), which successfully tested and demonstrated (in four provinces of the RRD) improved approaches for competitive procurement, community participation, and demand-driven system design that led to more sustainable water service provision, improved cost recovery, and more accountable enterprise management arrangements. However, long-term sustainability of the WSS system O&M remained a challenge at the end of the RRD-RWSSP. The Bank also helped the GoV to successfully test an output-based approach to rural water supply, supported by the Global Program for Output Based Aid (GPOBA) (P104528).

7. This program, the Results-Based Rural Water Supply and Sanitation Under the National Target Program (hereafter referred to as the Program or the PforR), was designed to support the NTP3 in eight geographically clustered provinces of the RRD: Bac Ninh, Ha Nam, Ha Noi, Hung Yen, Phu Tho, Quang Ninh, Thanh Hoa, and Vinh Phuc, covering a rural population of about 11.86 million (2012).13 In these provinces, a new approach for NTP3 was tested through the introduction of results-based planning and financing and by strengthening institutional mechanisms. NTP3 was an excellent candidate for the Bank’s new Program-for-Results (PforR) lending instrument, which allows client countries to improve design and implementation of their own programs while directly linking results with Bank financing. The lessons under the previous RRD-RWSSP Bank project, combined with the limitations and challenges identified from the NTP1 and NTP2 experience were the main drivers of the GoV decision to pursue a targeted, results-based approach to rural water supply and sanitation under NTP3. This Program aimed to progressively transform the NTP into a more focused and

11 Decision number: 104/2000/QĐ-TTg, on August 25, 2000. See footnote 6. The NTP-NRD RWSS component included the World Bank PforR;

and the government roles and responsibilities, and the funds flow and financial management arrangements, remained unchanged from the NTP3. In this ICR document, these programs are referred to as NTP, NRD, or NTP/NRD.

12 Data consolidated from MARD NTP3 reports. 13 Data source: Vietnam General Statistics Office (GSO) website, www.gso.gov.vn


Page 7 of 61

efficient system for delivering investments in sustainable RWSS systems. This was the first water-sector PforR for the Bank, and the first PforR in Vietnam.

Theory of Change 8. This Program responded to the development challenges in Vietnam where rural areas lagged in access to WSS services, where previous WSS investments were not sustainble, and where provincial implementing agencies lacked capacity to plan investments and manage WSS systems. The Program outcomes therefore were grouped into three results areas: increase sustained access to water supply services; increase sustained access to sanitation services; and improve RWSS sector planning, monitoring and evaluation. Figure 1 shows the Program Theory of Change.

9. Program results were achieved by supporting the activities and outputs of the four intervention areas of the NTP/NRD. For rural water supply access: construction of commune-level water supply schemes (treatment plants, distribution networks, and household connections). For rural sanitation access: construction of hygienic latrines in commune households, schools, and health centers. These WSS physical investments used best practice conditions to ensure sustainability. The Program also supported IEC and behavior change campaigns (BCC) to disseminate information and communicate benefits that raised the awareness and modified the behaviors of rural residents, so they connected to, used, and maintained the RWSS systems. The Program also supported capacity building activities that strengthened the planning skills of provincial agency staff, improved systems for monitoring and evaluation, and to produced higher quality planning documents. As a PforR, the Program also included a Program Action Plan (PAP), activities for technical assistance, implementation support, and the results-based verification and disbursement process (see below).

10. Key assumptions underlying the theory of change were: i) rural residents, once they were informed and educated, would be willing to connect to centralized water supply systems; ii) the agencies implementing the NTP/NRD had

PDO / Outcomes Activities Outputs

NTP / NRD RWSS program

Program DLIs

PforR Support

Implementation Support: planning; supervision; M&E; expertise; and reporting

PAP: transparency;

procurement; FM; land acquisition; ethnic minorities

Technical Assistance (TA): capacity bldg.; plans; M&E; technical; governance

Results Verification & Disbursement annual audit, verification, reporting, and results-based performance payments

IEC/BCC Awareness

Aware Community Members

Capacity Building

Improved RWSS Sector Planning and Monitoring and Evaluation (M&E)

DLI VI

M&E Information Systems

Increased Capacity of Staff/Agencies

5-Year and Annual Investment Plans

Annual Program Reports

Annual Verification Reports

Disbursement of Program Funds

18 Other Sectors

PDO Part (C)

Rural Water Supply

Rural Sanitation

Water Supply

Schemes + Networks

Hygienic Latrines (schools, HCs, HHs)

Increased Access to Water Supply Services

DLI III

Increased Access to Sanitation Services

DLI II

Community-Wide and Sustainable Sanitation

DLI IV, DLI V

Sustainability of Water Supply Schemes

DLI I

PDO Part (A)

PDO Part (B)

Parallel AusAID TA

Figure 1 - Program Theory of Change Schematic


Page 8 of 61

sufficient capacity to execute the RWSS investments at the speed and scale required; and iii) contractors (in the public sector or private sector) had sufficient capacity to carry out the WSS construction to the standards required.

Brief Overview of Program Design 11. All of the elements shown in Figure 1 were designed to work together to achieve the Program results.

Results-Based Financing of NTP/NRD Activity Areas: The core activity under the Program was to provide results-based financing (annually verified) to support the budget for investments and results in the four core rural water and sanitation interventions of NTP/NRD (listed above and shown in the vertical grey box of Figure 1).

PforR-Specific Support Activities: To support the unique PforR requirements, the Program included the following:

o Program Action Plan (PAP): A set of required actions, based on international good practice, to facilitate effective implementation and to enhance Vietnam’s country systems to comply with key fiduciary and social development principles. The PAP was co-developed by the Bank and the GoV. The PAP included specific actions in five areas: 1) transparency and citizen engagement; 2) procurement; 3) financial management and auditing; 4) land acquisition; and 5) ethnic minorities.

o Implementation Support: To facilitate both Program implementation and compliance with the new PforR processes, the Bank provided Program supervision, capacity building, technical expertise, and communication support to the implementing agencies at the national and provincial levels.

o Results Verification and Funds Disbursement: Under the PforR instrument framework, the Bank and the GoV developed a robust verification protocol and baseline survey. Each year, the implementing agencies collected and reported achievements for the six14 clearly defined Disbursement Linked Indicators (DLIs), mentioned below. The Program's independent verification agent annually conducted paper and physical audits of the achievements, and annual disbursements were made against the verified results.

Parallel Technical Assistance, Supported by AusAID: The PforR was supported by a parallel Technical Assistance (TA) program financed through an AUD 8 million recipient-executed AusAID trust fund, administered by the World Bank (TF 13061). The TA was implemented by the GoV with the objective to provide capacity building support to be directly utilized by the provincial- and national-level implementing agencies in conjunction with the PforR. The TA was designed based on technical and management capacity gaps identified in the program preparation assessments. The TA had six components, strengthening capacity in: A) planning, monitoring, and evaluation (including transparency and social accountability); B) sustainability of water supply systems and services; C) promotion and provision of sustainable sanitation services; D) governance (i.e. fiduciary, environmental, and social practices); E) verification of program results and external audits; F) improving uptake and sustainability of sanitation services; and G) communication and sharing of lessons and best practices. Annex 5 includes a summary and assessment of the TA.

12. Key Program Implementing Agencies: The main ministries and government agencies responsible for Program implementation include15: (a) the Ministry of Agriculture and Rural Development (MARD) was the lead agency16, with its National Centre for Rural Water Supply and Sanitation (NCERWASS) performing supervision, monitoring and evaluation, and inter-agency coordination for the water supply-related PforR implementation. MARD/NCERWASS also coordinated consolidation of DLI results, preparation of the annual Comprehensive Results Reports (CRR), and provided capacity building and technical support; (b) the Provincial Centre(s) for Rural Water Supply and Sanitation (PCERWASS) were the

14 The PAD originally had three DLI’s. At MTR, the DLI’s were revised. See the discussion on Program restructuring below. 15 Support agencies: Ministry of Planning and Investment, Ministry of Finance, State Bank of Vietnam, Rural Water Supply and Sanitation Partnership. 16 MARD leadership was through the General Department of Water Resources/Standing Office of the RWSS NTP.


Page 9 of 61

province level sub-divisions of NCERWASS responsible for implementation in the eight provinces; (c) the Vietnam Health Environment Management Agency (VIHEMA), an agency under the Ministry of Health17 (MoH), coordinated select sanitation activities and investments and organized the IEC activities for sanitation; and (d) the State Audit of Vietnam (SAV) was the Independent Verification Agent (IVA) for the PforR, conducting the annual results verification.

Program Development Objectives (PDOs) 13. The PDO was to: “Increase sustained access to water supply and sanitation services and improve sector planning, monitoring and evaluation in the participating provinces of the National Target Program for Rural Water Supply and Sanitation.”

Key Expected Outcomes and Outcome Indicators 14. The Program originally included three PDO/outcome Indicators (PI):

PI-1 - Number of people with water supply connections from Sustainable Water Systems PI-2 - Number of people with access to Commune-Wide Sanitation PI-3 - Number of Provincial Annual Plans and Program Reports from Participating Provinces

15. The Program originally included four Intermediate Indicators (II).

II-1 - Number of new Functioning Water Supply Connections II-2 - Number of newly constructed Improved Household Sanitary latrines II-3 - New water and sanitation facilities in schools and health centers constructed and in use II-4 - Program Reports from MARD disclosed to public

Program Results Areas and DLIs 16. The Program Originally had three DLIs, two of which had linked sub-DLIs. The inter-linkage was designed to incentivize balanced achievement of both water supply and sanitation targets.18 The original DLIs were as follows:

DLI 1: o DLI 1.1 - Number of new Functioning Water Supply Connections o DLI 1.2 - Number of newly constructed Improved Household Sanitary latrines

DLI 2: o DLI 2.1 - Number of people with water supply connections from Sustainable Water Systems o DLI 2.2 -Number of people with access to Commune-Wide Sanitation

DLI 3: o DLI 3.1 - Number of Provincial Annual Plans approved by participating provinces o DLI 3.2 - Number of program reports disclosed to the public

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 17. At the mid-term review (MTR) in June 2016, the Program underwent a Level 2 restructuring. Changes were relatively minor and positively impacted achievement of Program objectives. No changes were made to the Program scope, the supported activities, nor the theory of change. The changes are briefly summarized here.

Revised PDOs Outcome Targets, Result Areas, and DLIs 18. The PDO did not change during the project. It remained as written above.

19. The PDO Indicators, Intermediate Indicators, and DLIs were revised to be as follows:

PDO Indicators (revised) PI-1 - Number of people with water supply connections from Sustainable Water Systems PI-2 - Number of people with access to Commune-Wide Sanitation PI-3 - Number of Provincial Annual Plans and Program Reports from Participating Provinces

17 MoH also has the mandate for defining and checking drinking water and domestic water use quality standards. 18 In the original disbursement methodology, disbursements for linked DLIs were based on the lower percentage achieved of the linked DLIs.


Page 10 of 61

PI-4 - [added at MTR] Direct project beneficiaries19 (number), of which female20 (percentage)

Intermediate Results Indicators (revised) II-1 - Number of new Functioning Water Supply Connections (FWSC) II-2 - Number of newly constructed Improved Household Sanitary latrines (HH latrines) II-3 - New water and sanitation facilities in schools and health centers constructed and in use II-4 - [added at MTR] Number of communes where schools & health clinics have maintained hygienic status II-5 - [renumbered at MTR] Program Reports from MARD disclosed to public

Disbursement Linked Indicators (revised – split and renumbered at MTR) DLI I: Number of New Functioning Water Supply Connections (FWSC) DLI II: Number of newly constructed Improved Household Sanitary Latrines (HH latrines) DLI III: Number of people with water supply connections from Sustainable Water Systems DLI IV: Number of people with access to Commune-Wide Sanitation DLI V: [added at MTR] Number of communes where schools and health clinics have maintained hygienic status DLI VI: Number of Provincial Annual Plans approved by participating provinces; and program reports publicly disclosed

Other Changes The loan closing date and implementation schedule was extended by 12 months (to July 31, 2019). The DLI matrix and Bank disbursement arrangements were modified to reflect changes to DLIs.

Rationale for Changes and their Implication for the Original Theory of Change Changes to the PDO Indicators and Intermediate Indicators:

o PDO Indicator PI-4 was added as a corporate indicator. o Intermediate Indicator II-4 was added to address concerns about sustainability of school and health center

sanitation facilities, incentivizing these institutions to provide adequate maintenance. Changes to the DLIs:

o The original, linked sub-DLIs were split into independent DLIs: DLI 1 was split into DLI I and DLI II; and DLI 2 was split into DLI III and DLI IV. At the MTR, it was found that the original concept of linking water and sanitation DLIs (originally linked because NTP/NRD sanitation investments lagged water supply investments) was no longer needed because sanitation achievements had actually begun surpassing water results. Splitting the DLIs resolved some disbursement delays caused by the original linkages (disbursements for sanitation DLIs had to wait until the slower water supply DLIs were achieved).

o DLI V was added as a new DLI to correspond with the new Intermediate Indicator II-4. o The DLIs were made fully scalable21 and the DLI annual ceilings were removed in order to give more flexibility

to the implementing agencies in reporting and requesting disbursements from year to year. o The total Program target values for each DLI remained unchanged, but the annual targets were redistributed

across the remaining years of implementation (accounting for results achieved at the time of restructuring). The loan closing date was extended to allow time to achieve all of the DLI targets (especially sustainability DLIs).

Achieving sustainable water systems (DLI III) was sensitive to delays because it required two years of continuous operation to be verified. Therefore, early implementation delays (because PforR was a new financing instrument in Vietnam and because government capacity for pre-financing investments was low) led to delays in constructing

19 This indicator was derived from the number of benefitting households and corrected for overlaps in households receiving both water and

sanitation facilities. The number of people per household was 4.4, based on estimates calculated in the MTR process evaluation surveys. 20 The percentage of female beneficiaries (average of 51%) is estimated through census data from the General Statistics Office (GSO). 21 Scalability of DLIs was defined as the product of a “unit value” and the verified result achieved in a particular year, where the “unit value” was

defined as the quotient of the total DLI target over the total amount allocated to that particular DLI.


Page 11 of 61

some water schemes, which delayed connections to households and the start of operations. The extension allowed additional time to operate the WSS systems for two years to verify the sustainability and CWS conditions for DLI III, IV, and V (see below). However, the extension did not postpone direct Program benefits – by 2017, the DLIs for the hard infrastructure (DLI I for FWSC and DLI II for HH latrines) had already been largely met.22

Changes to the DLI matrix and disbursement arrangements: Disbursement amounts allocated to each DLI were revised to account for funds that had already been disbursed, with balances assigned to the new/revised DLIs. The Program documents Financial Agreement, DLI Matrix, Results Framework, etc. were revised accordingly.

II. OUTCOME

A. RELEVANCE

Relevance of PDO 20. The relevance of the PDO is rated as HIGH. The Program remained highly relevant to the strategic objectives of both the Government of Vietnam and the World Bank Country Partnership Strategy/Framework (CPS/CPF).

Relevance to the GoV RWSS Objectives: RWSS is a key priority for the GoV, with investments guided by the RWSS National Strategy and delivered through the NTP and NRD. The GoV has identified key objectives23 to improve NTP/NRD and RWSS service delivery, including: more demand-driven investments, more sustainable O&M, and building capacity of provincial agencies and service providers. Given these objectives, and the recurrent challenges for RWSS in Vietnam (see Section I-A), the PforR experience under this Program was highly important for learning and replicating good practices – especially those related to operating water and sanitation facilities sustainability and mobilizing private investors. The RRD experience is already being scaled up in 21 provinces through the Bank’s ongoing Results-Based Scaling Up Rural Sanitation and Water Supply Program (SupS) (P152693).

Relevance to the CPS/CPF: From Program preparation through Program closure, the PDO was clearly aligned with the key outcomes outlined in both the 2017 Country Partnership Framework for Vietnam (CPF, Report No. 111771-

VN) and the earlier 2011 Vietnam Country Partnership Strategy (CPS, Report No. 65200-VN). The Program supported direct investments in basic WSS infrastructure and improved sustainable operations using an innovative results-based PforR approach that helped improve institutional capacity and efficiency while also catalyzing private sector participation in the rural WSS sector. These Program activities contributed to the 2017 CPF focus on “inclusive growth and private sector participation” (Focus Area 1, 2017); and by focusing on increasing WSS infrastructure in lagging rural areas of the RRD, the Program supported the key outcomes of “improved quality and efficiency of infrastructure services” (Outcome 1.2, 2011), and “improved basic public service delivery and access” (Outcome 3.2, 2011).

Relevance of DLIs 21. The Relevance of the Program DLIs is rated as HIGH. The DLIs were directly linked to the achievement of the PDO and were designed to foster sustainability and incentivize improved performance of the implementing agencies.

There was a near one-to-one linkage between the DLIs and the three parts of the PDO. The objective to improve access to WSS services was linked by counting and verifying the number of people served and the number of new facilities constructed (in DLI I, II, III, and IV). The objectives for WSS sustainability were linked by the clear definitions in DLI III and DLI IV (see below). The objective for improved planning and M&E was linked by the detailed requirements for annual plans and reports in DLI VI.

The sustainability DLIs (DLI III, IV, and V) were an ambitious and innovative solution for the rural WSS sector. Earlier phases of the NTP, the prior Bank-supported RRD-RWSSP, and global WSS experience have faced the major

22 By 2017, DLI I for FWSC and DLI II for household latrines had met 97 percent and 96 percent, respectively, of their target values. 23 See: “Results and Lessons Learnt: Target Program Budget Support to NTP for RWSS, 2011-2015” (MARD, 2015).


Page 12 of 61

challenge of WSS investments often failing to sustain operations (technically and/or financially) beyond a few years. The design of DLIs in this Program was a significant innovation: clear and ambitious conditions for “sustainability” and “community wide sanitation” were included in the PDO and respective DLIs and had to be maintained for a minimum of two years. This greatly boosted sustainability performance in Program areas.

The DLIs provided strong incentives to the client for improved performance. As confirmed by the implementing agencies, the Program DLI’s provided new incentives to improve WSS service delivery and overcome key challenges of the NTP/NRD. For example, because DLI’s I and III were reimbursed based on the number of new FWSCs, planning became more demand-driven (communes with a real demand and a willingness to pay for services were selected, and right-sized water supply schemes were built to match the proven demand). This differed from the previous practice of simply building large water systems whenever funds became available to a commune. Also, because the DLIs were for fixed reimbursement amounts, the implementing agencies became more efficient and used cost savings to finance additional WSS facilities.

In particular, DLI VI (annual planning and reporting) was designed to enable achievement of the other DLIs. DLI VI required implementing agencies to prepare detailed annual investment plans, which guided the NTP3 investments in a more strategic way. This interplay of planning, DLI’s, and predictable results-based finance provided an innovative and efficient incentive to improve RWSS performance and was demonstrated through this Program in the RRD, and which is relevant to the RWSS sector more generally in Vietnam.

Rating of Overall Relevance 22. Overall Program Relevance is rated as HIGH. The Program PDO and DLIs remained highly relevant and aligned to the GoV and World Bank strategic objectives for RWSS in Vietnam, and the PforR approach served to incentivize improvements in Program implementation efficiency and institutional capacity building.

B. ACHIEVEMENT OF PDOs (EFFICACY)

Assessment of Achievement of Each Objective or Outcome 23. For assessment, the Program PDO is composed of three outcomes: “Increase sustained access to (A) water supply and (B) sanitation services, and (C) improve sector planning, monitoring and evaluation [within the context of] the participating Provinces of the National Target Program for Rural Water Supply and Sanitation.” Each of these three PDO outcomes are assessed individually below.

Outcome (A): Increased and sustained access to water supply services

24. Achievement of PDO Part (A) is rated as HIGH. The Program invested in water supply schemes and household connections through the NTP/NRD and over-achieved all targets for PDO Part (A), including PDO indicators, intermediate indicators, and DLIs. Table 1 shows Program results for the two indicators related to PDO Part (A). Both exceeded their targets. Achievement of the goal for increased access to water supply services is demonstrated by the numbers of new beneficiaries and new water connections built under the Program (see also the Results Framework). Moreover, the definition for DLI I required new functioning water supply connections, with rigorous criteria for continuity, water quality, and low failure rates, and which ensured that beneficiaries received high-quality, reliable access to water supply services.


Page 13 of 61

Table 1 - Achievement of PDO Part (A) Indicators Exceeded Target Values

Indicator Name Target Value

Achieved Value

% Achieved

Indicator / DLI No.

Number of people with water supply connections from sustainable water systems24

850,000 1,054,575 124% PI-1 DLI III

Number of new functioning water supply connections 340,000 399,964 118% II-1 DLI I

25. The Program also achieved its goals for sustainability of the supported water supply schemes. As noted in the Relevance of DLIs section, this Program was unique in specifying clear and ambitious definitions for sustainability in the DLIs and the Results Framework, as well as in the Program Operations Manual25 (POM). To be considered “sustainable” (and counted for DLI III/PI-1), water schemes were monitored and had to meet the following key operational, financial, and efficiency conditions two years after the start of operations: i) comply with MoH clean water standards26; ii) have at least 70 percent of the planned number of connections working with bills issued and paid (while providing 24/7 service at under 10 percent average monthly failure rate); iii) have tariff revenues cover O&M costs; iv) have Non-Revenue Water (NRW) less than 25 percent; and v) operate under a recognized management model. These conditions were verified by the IVA through paper and physical audits, and only schemes that met all conditions simultaneously were counted for the DLI. MARD/PCERWASS had the responsibility to inform contractors and operators (public and private sector) of the sustainability conditions and also helped to build the capacity of operators to implement the conditions and then continuously monitor the water schemes over time. MARD/PCERWASS also coordinated the results verification.

26. During the Program implementation period, a subset27 of 80 water supply schemes were verified for the sustainability DLI III. Of these, 78 schemes met all of the conditions and passed verification. Only two schemes failed the verification. Table 2 shows values for relevant sustainability conditions for the verified schemes in each of the eight provinces, and overall. The data shows that the Program overcame a major sustainability challenge in the sector to support water supply schemes that were able to operate for at least for two years meeting all of the defined conditions. Although the verified schemes are a subset of the total, they provide service to 89 percent28 of the new FWSCs built under the Program and their performance is indicative of the positive sustainability results achieved for other schemes – other schemes also followed the Program approach but did not need to be verified to achieve the DLI target.

27. Sustainable operation of the water supply schemes was a major achievement of the Program. The Program demonstrated that robust and strategic sustainability criteria could be met by local operators for a minimum of two years. This achievement sets a strong foundation for the water schemes to continue with sustainable operations into the future.

Table 2 - Sustainability Performance for the Subset of Water Schemes (WS) Verified

Sustainability Condition Total: 8

Provinces Bac

Ninh Ha Nam Ha Noi

Hung Yen

Phu Tho Quang Ninh

Thanh Hoa

Vinh Phuc

Subset of WS Submitted for Sustainability DLI A 80 19 13 10 7 6 12 5 8

24 This indicator was added as a World Bank corporate core indicator and was also disaggregated by gender. 25 The POM contained clear detailed Program definitions and procedures and was updated accordingly as conditions changed (e.g. MTR, etc.). 26 Schemes with design capacity larger than 1,000 m3/day were required to comply with MoH standard QCVN 01, and smaller schemes with

QCVN 02 (specified in the National Technical Regulation on Domestic Water Quality, MoH Circular 05/2009/TT – BYT (June 2009)). 27 Not all water schemes needed to be verified to meet the sustainability DLI target. In total, the Program supported 144 water schemes in

order to meet the other DLI for water supply access (i.e. DLI I for FWSCs). Among these, and by the time of final verification in 2018, only 105 schemes had operated for the minimum two years required for verifying sustainability. However, it was sufficient for implementing agencies to verify just 80 water schemes to satisfy DLI III for the number of people connected to sustainable water systems. This does not necessarily mean that the remaining water schemes were not operating sustainably, only that they were not verified for DLI III.

28 As of 2018, there were 359,581 new FWSCs served by water schemes that had operated for the required two years. Among these, 318,275 new FSCWs (89 percent) were served by water schemes that had been verified for DLI III. Two small schemes (430 FWSCs) failed verification.


Page 14 of 61

Number of WS Verified Sustainable (pass DLI III) B 78 19 13 10 7 6 10 5 8

Number of WS Met Water Quality Standards C 80 19 13 10 7 6 12 5 8

Number of Planned (design) HH Connections 306,225 62,012 39,659 42,100 66,738 20,950 25,820 31,764 17,182

Number of Actual HH Connections 318,038 85,837 41,109 42,901 51,890 19,629 25,983 33,612 17,077

Percent of Planned HH Connections Built 104% 138% 104% 102% 78% 94% 101% 106% 99%

Average Cost-Recovery-Ratio 1.28 1.34 1.17 1.25 1.12 1.11 1.19 1.64 1.46

Average Non-Revenue-Water 19% 19% 20% 21% 20% 15% 14% 23% 19%

Percent of WS w/ Recognized Mgmt. Model D 100% 100% 100% 100% 100% 100% 100% 100% 100%

A See footnote 27. B Only two small water schemes failed the sustainability verification. C Water schemes larger than 1,000 m3/d had to meet MoH standard QCVN 01, and smaller schemes had to meet QCVN 02. D Recognized management models include: Public Utility (PCERWASS, PMU, etc.), Enterprise, and CPC/Cooperative.

28. Finally, informal interviews during field visits to 10 water supply schemes in five provinces indicated that commune residents were satisfied with the new piped water services provided under the Program. Residents interviewed identified a range of meaningful qualitative benefits, such as: time savings, reduced sediments in water, cleaner washed clothes, less corrosion of household appliances, and more convenience in their life. Several residents proudly stated that they “felt like they were living in an urban area now”, and that “the cost of the piped supply was comparable to before, but the most important thing is that the water quality is much better”.

Outcome (B): Increased and sustained access to sanitation services

29. Achievement of PDO Part (B) is rated as HIGH. The Program invested in sanitary latrines for households and public institutions as well as in behavior change campaigns (BCC) and was able to over-achieve all targets for PDO Part (B). Table 3 shows key Program results for the four indicators related to PDO Part (B). This achievement clearly demonstrates that the Program goal for increased access to sanitation services was fully met.29

Table 3 - Achievement of PDO Part B Indicators Exceeded Target Values


Achieved Value

% Achieved

Indicator / DLI No.

Number of people with access to Commune-Wide Sanitation30 1,275,000 1,407,669 110% PI-2 DLI IV

Number of newly constructed improved household sanitary latrines 130,000 142,280 109% II-2 DLI II

Number of new latrines constructed and in use in schools and health centers

1,440 1,492 104% II-3 n/a

Number of communes where schools and health clinics have maintained hygienic status

150 179 123% II-4 DLI V

30. Community-Wide Sanitation (CWS) was an innovative and integrated approach introduced and promoted under the Program. The DLI associated with CWS was DLI IV. To be counted for DLI IV, and designated as a “CWS status commune”, each commune had to meet all of the following conditions: i) 100 percent of public kindergarten, primary, and secondary schools and 100 percent of commune health centers have clean water as defined by MoH standards (QCVN 02: 2009/BYT) and hygienic sanitation facilities (including clean water and soap for handwashing) according to MoH

29 There were initial delays in achieving sanitation results, but once the PforR mechanism was absorbed by the implementing agencies, and with

the support of the TA and capacity building, the Program was able to catch up on all targets. See Key Factors section. 30 This indicator was added as a World Bank corporate core indicator and was also disaggregated by gender.


Page 15 of 61

standards (QCVN 01: 2011/BYT); ii) at least 70 percent of households have hygienic sanitation according to MoH standards (QCVN 01: 2011/BYT); iii) 100 percent of households use latrines of some kind; and iv) the commune had to be open defecation free. All these CWS DLI conditions had to be met for a commune to pass verification, and these DLI conditions were further reinforced by education and awareness activities31, with commune residents encountering demonstrations and training on latrine maintenance and positive WASH32 messages in multiple dimensions of their lives. Sanitation and hygiene were particularly promoted in schools, where students participated in start-of-year lessons on WASH benefits and practices and received detailed training on how to use latrine facilities at school and at home. This collective, community-wide, and school-based approach under CWS has raised the awareness and expectations for sanitation and hygiene and has set a foundation for sustainable maintenance of household latrines and a lifetime of good hygiene behaviors. To implement the CWS concept, VIHEMA coordinated with commune and district level health agencies, along with social organizations like the Women’s Union, to educate commune residents, assign maintenance responsibilities, conduct the IEC/BCC, continuously monitor the communes, and then coordinate verification. Financial subsidies to co-finance toilet construction were also made available by the GoV to low income families (see footnotes 35 and 36). By Program close, 191 communes33 had achieved CWS status, simultaneously meeting all of the sanitation criteria above. Table 4 shows the values for relevant CWS achievements in the Program provinces, and overall, in 2018.

31. The CWS concept provides a ‘critical mass’ of sanitation facilities and behaviors at the commune level. Rather than isolated, piecemeal investments in individual latrines, CWS focuses commune attention on collective and holistic goals to achieve high rates of sanitation in households, schools, and health stations, with all the Program DLIs working together in a stepwise manner. At a base level, DLI II (and indicator II-3) incentivized construction of latrines in the commune households. Achieving this contributed to the more holistic DLI IV for CWS, which linked HH coverage with the criteria listed above and expanded WSS in schools and health centers. Finally, DLI V required ongoing maintenance of sanitation facilities in schools and health centers for long-term sustainability. This positive results chain, mapped to specific and measurable DLIs, was an important innovation introduced by the Program that mobilized more cohesive participation in sanitation improvements at the whole-of-commune level.

Table 4 - Community-Wide Sanitation in Communes Supported Under the Program (2018)

Community Wide Sanitation Conditions Total: 8

Provinces Bac

Ninh Quang Ninh

Ha Nam

Hung Yen

Phu Tho

Hanoi Thanh

Hoa Vinh Phuc

Number of Participating Communes 203 24 29 26 29 20 14 33 28

Number of Communes Achieving CWS Status A 191 24 29 26 25 20 14 31 22

Percent of Households in the communes with Hygienic Sanitary Latrines (after intervention)

89% 88% 87% 89% 95% 82% 97% 82% 93%

Percent of Schools with Hygienic Sanitation Facilities (after intervention)

100% 100% 100% 100% 100% n/a B 100% 100% 100%

Percent of Health Centers with Hygienic Sanitation Facilities (after intervention)

100% 100% 100% 100% 97% 100% 100% 100% 100%

A See footnote 33 B School latrine data from Phu Tho province was not available

31 In total, 600 training events in 238 communes were conducted (with over two million participants). Also, over 300,000 leaflets, 500 posters,

and regular loudspeaker announcements were produced annually to promote clean water, sanitation, and improved WASH behaviors. 32 WASH is “Water Access, Sanitation, and Hygiene”. 33 In total, the Program supported 203 communes for sanitation, however not all communes needed to be verified to meet the CWS DLI target

(see similar explanation in footnote 27).


Page 16 of 61

32. Sustainability of the Program sanitation facilities was assessed differently for commune institutions (schools and health centers) and for households. Through analysis during preparation, and later confirmed at the MTR, installation and maintenance of hygienic latrines in schools and health centers was identified as a significant issue, and the Program included indicators (II-3 and later DLI V) to specifically measure the number of latrines and water facilities constructed, in use, and maintained at these commune public facilities. By program close, in 179 communes34, 100 percent of schools and health centers were verified as maintaining these facilities, and the targets for DLI V (and II-3) were exceeded. DLI V focused new attention on ongoing maintenance of institutional latrines (including water facilities and handwashing with soap), and discussions with provincial agencies indicated that communes were financially incentivized to assign clear responsibility and accountability for regular latrine cleaning and upkeep. For example, in Phu Tho, the Provincial People’s Committee (PPC) enforces accountability; while in Bac Ninh and Hung Yen, the CDC and DoET play the oversight roles.

33. For the sustainability of household hygienic sanitary latrines (HH latrines), evidence from studies35 indicate that where household latrines are acquired in rural Vietnam, proper use and maintenance are not major issues. As rural quality of life increases, rural households in Vietnam increasingly value clean, durable, and easy-to-use toilets; and families that invest36 in a hygienic latrine for their home typically make strong efforts to protect and maintain their investment37. For this reason, the Program did not include specific DLI criteria for sustainability of household latrines. The household latrines constructed under this program were sized to require emptying only every five to eight years38, thereby simplifying the maintenance needs. In addition, for this ICR, the likelihood for sustained maintenance of household latrines was assessed by looking at trends in the data for commune-level coverage of well-functioning HH hygienic latrines (i.e. DLI II). Figure 2 shows the average HH latrine coverage rate for Program communes, by province, for the three years leading up to Program closure. It can be seen that commune coverage rates were not only maintained, but actually increased over time for all provinces in the Program, implying adequate maintenance. These positive trends, alongside evidence from informal interviews39 with commune residents on their familiarity with the required tasks and frequency of cleaning HH latrines, and combined with the multi-dimensional CWS approach, indicate that the Program-supported hygienic latrines were and will continue to be well-maintained in individual households.

34. Finally, HH latrines supported by the Program were sized to be emptied after more than five years and the households contract private sludge operators to empty the septic tanks. Evidence from visits to five communes in five provinces indicated that Program household beneficiaries were aware of the requirements for septage management and cleaning of septic tanks. Contact numbers and costs for septic pumping service providers were known and several residents in Program communes had already had their septic tanks cleaned.

34 Not all communes needed to be verified to meet the target for DLI V. See similar explanation in footnote 27. 35 According to a 2015 study by the London School of Hygiene and Tropical Medicine, supported by the World Bank: "Sanitation and Hygiene in

the Northern Mountains and Central Highlands of Vietnam: Report of a Formative Research Study and Recommendations for SupRWS". 36 Most HH sanitary latrines constructed under the Program were financed by individual families using savings or short-term subsidized loans. The

average hygienic latrine costs 10 million VND. Around 18 percent of the poorest households received a subsidy of 1 to 4 million VND. 37 The Program process evaluation (2015) found that 77 percent of households surveyed use pour-flush toilets with septic tanks, and 85 percent

had built their toilet along with a bathroom, suggesting that these were major household investments likely to be used and maintained. 38 Based on guidelines from Vietnam’s Ministry of Science and Technology (TCVN-10334-2014), the Program used a minimum size of 1.5 m3 for HH

latrines. Average per-capita rural sludge generation is 0.04 - 0.07 m3/person-year (MoH: QCVN 07-2010, page 94). Average HH size is 4.4 people. 39 Task team interviews during the ICR mission in five communes in five provinces.


Page 17 of 61

Figure 2 - Coverage of Households in Program Communes with Hygienic Latrines, by Year in Select Provinces40

Outcome (C): Improve sector planning, monitoring, and evaluation

35. Achievement of PDO Part (C) is rated as SUBSTANTIAL. The Program invested in technical assistance and capacity building and was able to achieve the targets for PDO Part (C), including PDO indicators, intermediate indicators, and DLIs. Table 5 shows key Program results for the two indicators related to PDO Part (C). Achievement of these targets represents the Program’s significant institutional strengthening for RWSS planning, monitoring, and evaluation processes within the implementing agencies of participating provinces.

Table 5 - Achievement of PDO Part C Indicators Met the Program Target Values


Achieved Value

% Achieved Indicator / DLI

No.

Number of Provincial Annual Plans and Program Reports from participating provinces 41

102 102 100% PI-3 DLI VI

Number of Program Reports from MARD disclosed to the public 42

6 6 100% II-5 DLI VI

36. Planning capacity and outcomes were highly improved under the Program, and this was a major achievement and significant contribution to the WSS sector. Prior to the PforR, the earlier RRD-RWSSP identified major shortfalls and challenges with RWSS investment planning in Vietnam, including: processes were ad-hoc and unscientific, selection of communes and planning of water supply schemes was not demand-driven, facilities were over-designed, procurement was not transparent, and unpredictable financing often led to construction delays.43 Rather than defaulting to a follow-on Investment Project Financing (IPF) project, the Bank advocated for this Program to be designed using the ambitious PforR approach and directly address the challenges cited above. The PforR instrument allowed for meaningful institutional strengthening and capacity building to take place directly within the NTP3 and NRD programs, which supported sustained improvement to RWSS planning within the GoV implementing agencies.

40 Hanoi and Ha Nam provinces were not able to provide historic HH latrine data for the ICR; but Table 4 shows high 2018 values for both. 41 This indicator target was based on eight provincial investment plans and nine program reports (i.e. eight provincial and one consolidated)

each year. Original target assumed five years. After restructuring, target was revised for six years of implementation. 42 This indicator target was based on one consolidated program report disclosed each year. Original target assumed five years. After

restructuring, target was revised for six years of implementation. 43 “Vietnam Results-Based Rural Water Supply and Sanitation Under the National Target Program: Impact Evaluation, World Bank” (2016).

Report No: ACS18130.

0%

20%

40%

60%

80%

100%

Vinh Phuc Thanh Hoa Quang Ninh Phu Tho Hung Yen Bac Ninh

2016 2017 2018


Page 18 of 61

37. Under the PforR, the definition for DLI VI in the PAD and POM included specific requirements44 for provincial implementing agencies to produce detailed annual RWSS investment plans following a highly structured and scientific process. Such detailed, scientific planning was not done in rural Vietnam before, and the new plans provided actionable guidance for Program investments and implementation activities. Preparing the investment plans according to the defined requirements built new planning capacity within the implementing agencies; and this improved capacity was key to enabling the other DLI targets in PDO Parts A and B to be met. DLI VI also required that the provincial and consolidated annual plans be publicly disclosed on the new, Program-supported transparency platform45, which further promoted social accountability of the implementing agencies. This process was supported by the parallel AusAID TA and by the direct technical assistance and implementation support from the Bank.

38. The Program’s contributions to improved RWSS planning yielded a significant change in outcomes for water supply and sanitation coverage and for sustainability in the program areas. Table 6 compares outcomes before and after the PforR and also shows a comparison of Program and Non-Program supported areas. Some key outcomes include:

The PforR had a significant impact on household coverage of sanitary latrines: average coverage in PforR communes rose from 56 percent in 2012 to 88 percent in 2018, while coverage in non-PforR communes rose from 56 percent to 76 percent over the same period.

The PforR had a significant impact on water scheme construction efficiency: average construction time was substantially shorter in PforR areas than in Non-PforR areas – 20 months versus 34 months, respectively. Provincial counterparts indicated that the amount and certainty of funding under the PforR gave greater confidence to implementing agencies and to the public or private contractors to avoid delays and work-stoppages from lack of funds common under the past NTPs/NRD and to overall help speed up implementation.

The PforR’s demand-driven approach for commune selection and facility design had a significant impact on water system efficiency: prior to and outside the PforR areas, small NTP/NRD water schemes were typically built to serve a single commune and faced a common problem of low household connection rates (average of 685 HH per scheme, see Table 6). The improved planning and processes under the PforR helped provinces to build larger and more efficient water schemes – serving an average of 2.8 communes and 3,104 HH per scheme.

The water supply sustainability criteria introduced under the PforR also helped to reduce NRW rates to an average of 22 percent, as compared to 34 percent in non-PforR areas.

44 For example: current-year baseline data (coverage rates, number of connections, water source, water quality, etc.); mapping of schools and

health centers; mapping of water sources; assessment of existing facilities; mapping of poverty; identification of demand gaps and priority areas; and detailed estimates of one- and five-year investment requirements (see PAD, Annex 2). Such detailed plans were not made before.

45 The transparency platform was a Program requirement under the PAP (Action #1).


Page 19 of 61

Table 6 - PforR Performance Comparison: Before/After and Program/Non-Program A

PforR Results Compare Before/After Compare PforR/Non-PforR

Comparative Indicator Unit PforR Areas

Before PforR

Relative Improve

Non-PforR Areas

Relative Improve

[Reference Year] [2018] [2012] +/- [2018] +/-

RWSS Coverage Rates (after 2 years of operation)

Percent Coverage, HH Water Supply (after 2 yr) [%] 72% 36% [1] + 36% n/a B n/a

Percent Coverage, HH Sanitation (after 2 yr) [%] 88% 56% + 32% 76% + 12%

Percent of Schools with Latrines (after 2 yr) [%] 100% 87% + 13% 98% + 2%

Percent of Health Centers w/ Latrine (after 2 yr) [%] 99% 85% + 15% 98% + 1%

Construction Planning and Efficiency

WSS Construction Time [months] 20 48 [2] + 58% 34 + 40%

WSS Unit Construction Cost [US$/HH] $233 $630 [2] + 63% n/a B n/a

Scheme Size: Avg. Number Communes Served [number] 2.8 1.0 + 180% 1.1 + 160%

Scheme Size: Avg. Number of HH Connections [number] 3,104 685 [1] + 353% 1,471 + 111%

Operational Sustainability (after 2 years of operation)

WSS Non-Revenue Water Rate (after 2 yr) [%] 22% n/a C n/a 34% + 13%

WSS Cost Recovery Ratio46 (after 2 yr) [ratio] 1.18 n/a C n/a 1.14 + 4%

A Unless otherwise indicated, all values in this table were summarized from data provided by MARD and provincial implementing agencies. B Some water scheme and cost data were difficult to obtain for non-PforR areas and are not included in this table. C NRW and Cost Recovery data for water schemes prior to 2013 was not available. [1] Data source: Program data sheets for 2013 (baseline) provincial annual plans. [2] Data source: “Vietnam Results-Based RWSS Under the NTP: Impact Evaluation, World Bank” (2016). Report No: ACS18130

39. Monitoring and evaluation (M&E) capacity was substantially improved under the Program. During implementation, as a PforR, the Program introduced new indicators and new monitoring requirements that improved M&E in the eight participating provinces. The Program’s M&E experience was reflected in the revisions to the national-level NRD WSS M&E system in 2016. The Program also pilot-tested an innovative phone-based M&E system.

40. The Program introduced a new set of clearly defined M&E indicators for WSS and a new way of measuring and reporting results. Bound to DLIs with ambitious targets clearly assigned to responsible agencies, the new indicators and rigorous PforR verification process ensured high quality data collection. Indicator data was regularly used for Program reporting, for the annual RWSS investment plans, and for preparing this completion report. M&E was a major improvement from previous practice in the NTP/NRD, where indicators were more general in nature and data was often considered unreliable and was mostly used by central-level authorities for statistical purposes in national reports.

41. The Program M&E approach was directly used when the NRD revised its national system for WSS M&E in 2016. In particular, elements from the Program’s DLI III (defining water system sustainability conditions) and DLI IV (community-wide sanitation requirements) were integrated into new and revised NRD M&E indicators. The revised indicators were not exactly the same as in the PforR, but followed important aspects of sustainability that had been tested and proven under this PforR, and which had not been present in NRD M&E before. To implement the new NRD M&E indicators and systems, MARD/NCERWASS and MoH /VIHEMA have developed and approved sector-specific monitoring systems, and

46 Cost recovery was defined in the POM as: O&M costs covered by operating revenue from tariffs for the last three months of the reporting

period. Official data on operational costs and revenues to be registered in the financial records of the organization managing the water supply scheme the year before verification is conducted. Subsidies (if any) cannot be included as operating revenue.


Page 20 of 61

both systems were piloted in the RRD under this Program (from 2017 to 2018) and are now being scaled up to the rest of the country. MARD approved a new set of water supply service indicators, which are fewer than those of the PforR but include key sustainability elements. NCERWASS has conducted training in 63 provinces and has confirmed that most provinces have received designated budget to collect data using this new process as of 2019. VIHEMA also approved a new set of indicators to report on sanitation performance in all 63 provinces, and which are more closely aligned with the SDGs. VIHEMA also established a new M&E data platform, which includes monitoring of sanitation in households, health stations, and schools. This system has already been piloted in all eight Program provinces, and is now being scaled up to the 21 provinces of the ongoing SupS PforR, and will later be scaled up to all 63 provinces in Vietnam.

42. Finally, a mobile phone-based M&E system was developed (through the AusAid-supported TA) to simplify the time-consuming, paper-based data collection under NTP3 and to support quicker local results reporting for verification purposes. The phone-based system was piloted in different provinces for water connection data, household sanitation data, and institutional sanitation data from 2013 to 2015. However, uptake of this new system by the local agencies was relatively low (only two provinces adopted the system at scale). Local data collectors (provincial staff) were still required by existing local regulations to use the official paper-based system (parallel to the phones), and so most did not fully migrate to the phone-based M&E, and data collection and processing continue to be paper-based. The phone-based system was, however, successfully used by SAV/IVA to collect and record data for the Program verification processes.

Rating of Overall Efficacy

43. The overall rating for PDO Achievement (efficacy) is HIGH – the operation fully achieved its objectives. All three parts of the PDO met or exceeded their targets. PDO Parts A and B were rated as HIGH and PDO Part C was rated as SUBSTANTIAL. The combined efficacy rating is HIGH, given the significant achievement in both ‘hard’ infrastructure objectives and ‘soft’ institutional objectives.

44. The number of direct project beneficiaries receiving improved water supply was 1,759,842, which exceeded the target value of 1,496,000 people. The number of direct project beneficiaries receiving improved sanitation services was 1,407,669, which also exceeded the target value of 1,275,000 people. The total number of direct project beneficiaries receiving access to improved WSS services, and accounting for overlap (see Results Framework)47, was 2,149,765 people. This exceeded the original target value of 1,861,000 people. Among these direct beneficiaries, it is estimated that 51 percent48 (1,096,380 people) were women. All of the beneficiaries were rural. See details in the Results Framework.

C. JUSTIFICATION OF OVERALL OUTCOME RATING

45. The Overall Outcome Rating for the Program is assessed as Highly Satisfactory. There were no shortcomings in the Program’s achievement of its objectives (efficacy) nor in its relevance, both of which were rated as HIGH.49 Moreover, the Program made significant contributions to the rural WSS sector in Vietnam and globally by including rural water system sustainability and community-wide sanitation in the PDO and DLIs to achieve comprehensive and sustainable results. These highly relevant and innovative approaches are already being replicated in other Bank operations globally.50

47 At MTR, it was estimated that 10 percent of households under the program benefitted from both water supply connection and sanitary

latrines. Therefore, the calculation for total Program beneficiaries is: the number of people benefitting from improved water sources plus the number of people benefitting from improved sanitation services, multiplied by 90 percent.

48 Data source: Vietnam General Statistics Office (GSO) website, www.gso.gov.vn. 49 The PforR assessment at completion does not require evaluation of program efficacy, per World Bank IEG guidelines. 50 Including the Vietnam Scaling up Rural Sanitation and Water Supply Program (P152693), as well as the Tanzania Sustainable Rural Water

Supply and Sanitation Program (P163732) and others currently under preparation.

http://operationsportal.worldbank.org/secure/P163732/home

http://operationsportal.worldbank.org/secure/P163732/home


Page 21 of 61

D. OTHER OUTCOMES AND IMPACTS

Institutional Strengthening

46. The Program achieved significant improvement in institutional strengthening, which was the focus of PDO Part C, discussed in Section II-B. Specifically, RWSS planning and M&E capacity of the provincial and national implementing agencies was strengthened, and important elements of the PforR approach for RWSS sustainability were incorporated into the NTP3/NRD programs and national-level WSS M&E systems. Institutional strengthening was supported in early years of the Program by the AusAID parallel TA. The TA included six components designed to build institutional capacity in: planning and M&E, water supply and sanitation system sustainability, governance practices, and results verification, among others. See Annex 5 for more details on the TA. In spite of some early procurement and implementation delays, the TA produced specific outputs that benefit not only the institutions under this Program, but also those in the SupS PforR, and the NRD generally, including: a) a new monitoring and reporting system; b) a beneficiary feedback mechanism; c) suggested technical standards for rural water schemes; and d) training manuals for sustainable water supply O&M.

47. The PAP also contributed to institutional strengthening by establishing clear priority actions, co-developed by the Bank and the client, for the implementing agencies to mitigate the RWSS risks and weaknesses identified during the preparation stage assessments. The PAP actions introduced improved procurement methods to strengthen competition and encourage private sector participation. The PAP actions also supported improved transparency and citizen engagement, including a data platform and call center for stakeholders to report complaints and monitor responses. These actions increased agency accountability and improved confidence in the quality of infrastructure built.

48. Finally, it can be noted that because of the higher capacity of implementing agencies and contractors supported by the Program, the quality of the water and sanitation facilities constructed also improved. Annual verification of facilities included regular physical inspections, with digital photographs of individual household connections and latrines systematically stored in a cloud database for agencies, provinces, and the Bank to monitor construction quality.

Mobilizing Private Sector Financing

49. The PforR served as a catalyst for increased private sector participation and financing in the RWSS sector. The PforR introduced a significant, reliable financing mechanism, along with time bound WSS construction targets, larger-scale schemes, and a transparent competitive bidding process. With these institutional improvements, new private sector investors from inside and outside the Program provinces were attracted to bid on program water scheme contracts, and many also began investing their own private capital into the program-supported water schemes. In the Provinces supported by the Program, private enterprises have played a leading role. Of the 144 water supply schemes supported under the PforR (i.e. counted for DLI I, FWSCs), 87 were operated by private enterprises, accounting for 60 percent. Prior to the PforR, the rural water market was seen as an unattractive loss-maker, and there was limited private investment in most of the eight participating provinces. Rural systems were seen as scattered, difficult to manage, and with low potential for profit. In four of the eight Program provinces51, MARD estimated that in 2012, only 16 water supply schemes were fully or partially financed by the private sector, with a total of US$ 2.15 million in private investment. By 2018, the private sector had financed (fully or partially) 63 water supply schemes, with a total of US$ 53.65 million in private investment.52 This private investment reduced the RWSS finding burden of the GoV (see Fiduciary section below).

50. The PforR instrument introduced new flexibility and accountability into the financing and operation of RWSS infrastructure. Provincial implementing agencies were free to combine financing from different funding sources to meet the DLI targets, so long as the PAP and POM were followed. This incentivized agencies to seek out private investors to

51 2012 data was not available for Hanoi, Ha Nam, Quang Ninh, nor Phu Tho. 52 Data provided during ICR mission by PCERWASS implementing agencies in Bac Ninh, Hung Yen, Thanh Hoa, and Vinh Phuc.


Page 22 of 61

pre-finance construction and operate the water supply schemes. Meanwhile, the privately-operated schemes were held to the same service level and sustainability standards as defined and verified by the Program DLIs. Of the 81 water supply schemes that have so far met the sustainability conditions (DLI III), 47 schemes were operated by private enterprises.53

Gender

51. Around one million women benefitted from the Program. Informal interviews with beneficiaries indicated that, as primary household managers of water and sanitation, women now spend less time fetching or queuing for water; now expend less effort to clean and maintain borehole wells and onsite water filters/purifiers; and now report cleaner clothes, healthier skin, and better health and home environments for themselves and their families. Beyond being significant recipients of improved RWSS services, rural women were also a key driver of success for the Program. The Program leveraged the strong network of the Women’s Union54 (WU) in Vietnam to promote RWSS and WASH at the commune and village level. WASH is already one of the core national priorities for the WU. Under the Program, WU members played a leadership role in the commune IEC and BCC activities, responsibility for compiling lists of households without hygienic latrines, sharing information on how to build latrines, and promoting household water supply connections. Around 70-80 percent of commune WU members in the Program Area were active participants or recipients of WASH information during the program period.55 The WU was also one of the core organizations selected to promote and administer household loans from the VBSP for WASH investments, and it is estimated that WU members were responsible for generating between 25 and 50 percent of all such WASH loans in different communes.56

Poverty Reduction and Shared Prosperity

52. This Program was focused on increasing sustainable access to improved water supply and sanitation services to rural communities and rural beneficiaries in the eight participating provinces. In Vietnam, the rural population lags behind in terms of income and access to quality WSS services. This Program, by its nature, focused on improving the quality of life and sharing benefits of development with social-economically lagging populations. According to data from MARD57, 26,911 poor households received subsidies from the Program to build hygienic latrines, out of a total of 104,100 poor households and 5,480,800 total households in the eight provinces.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

53. Use of the PforR instrument was well justified, and the Program was well designed with components and activities that were comprehensive and complementary. As noted in Section II-B (outcome of PDO Part C), the Bank advocated for use of the PforR instrument as a new approach to achieve results and build capacity in the RWSS sector in Vietnam. All of the Program elements (described in the Theory of Change) were complimentary and mutually reinforcing. The Program was designed to provide results-based budget support for RWSS investments through the NTP/NRD program. The design included implementation support from the Bank and the parallel TA component to build capacity of the NTP/NRD implementing agencies to plan, execute, and monitor their RWSS investments. The Program was designed to focus on sustainability and CWS through the DLI definitions to help improve long-term outcomes for RWSS service delivery. These aspects would be institutionally and financially incentivized by the annual planning, verification,

53 Note that this is a conservative figure. Not all schemes were required to be verified for the sustainability DLI III (see Table 2); and it was easier

for provinces to collect data and verify publicly operated (PCERWASS) water schemes. 54 The Women’s Union (WU) is a socio-political organization with more than 15 million members and a well-established organizational structure,

reaching from central leadership down to every village and commune. In most communes, 80 percent of women are active WU members. 55 Based on data from interviews with three Heads of commune-level WU in three representative communes supported by the Program. 56 ibid. 57 See the Borrower’s ICR in Annex 4.


Page 23 of 61

reporting, and disbursement cycle under the PforR, which would still utilize and strengthen the GoV NTP/NRD systems for budget allocation, procurement, and financial management. Finally, the Program design included IEC/BCC and WASH awareness building activities to help educate rural residents and encourage them to connect to the piped water systems and to build and maintain the household latrines supported under the Program.

54. The verification protocol for the DLIs was well-designed to be practical and cost-effective, using robust statistical sampling methods that optimized expensive data collection requirements. Given that the PforR instrument was new to Vietnam (and the first in the water sector in the Bank), the Bank provided strong, early, and ongoing support to develop a robust verification protocol and then train the in-country team of verification surveyors (i.e. SAV and their consultants). The team relied on experienced economists and statisticians to develop specific methodologies for the different DLIs. The Program design also included support for data collection and analysis for a robust baseline survey to be carried out in the first year of verification.

B. KEY FACTORS DURING IMPLEMENTATION

55. The unique and ambitious sustainability definitions and targets were effective and essential to the success of the Program. A major innovation of the Program was the introduction of sustainability requirements for water supply schemes (DLI III) and CWS requirements for communes (DLI IV). These innovative requirements directly addressed the biggest sustainability challenge of the NTP/NRD program. These sustainability DLIs were clearly defined in the POM with a set of specific and measurable sub-indicators that had to be met and maintained for at least two years after construction (see Section II-B). The DLIs were monitored and verified on an ongoing basis, which motivated implementing agencies and operators/owners to ensure Program investments were sustainably maintained. Over the course of implementation, the provinces, communes, and water system operators learned how to achieve the sustainability conditions; and significant planning and implementation improvements were observed (see Section II-B). The sustainability DLIs introduced under the PforR were a major insight into better planning and maintenance for RWSS.

56. The Program included significant emphasis and funding for TA activities, which supported results achievement. The Program included a parallel AusAID-supported TA (see Section II-D). Given that the PforR approach was new in Vietnam, and that the central and provincial implementing agencies were ‘learning by doing’ thorough the Program, the inclusion and design of this parallel TA funding anticipated the capacity building resources that would be needed to introduce results-based planning, monitoring, and evaluation of RWSS investments. The scope of the TA was clear and comprehensive, aimed at building the capacity of the implementing agencies in critical areas to enable good performance. The TA did encounter a number of challenges: approval for the TA component was received several months after the overall Program approval and contracting for TA activities took longer than expected due to weak procurement capacity of the government agencies. As a result, the TA was delayed, but nevertheless, the final outcomes provided significant beneficial impacts to institutional strengthening, supporting sustainable water supply systems and services, supporting sustainable sanitation services, and building capacity in the verification of program results. Further details on the TA are included in Annex 5.

57. The Program mid-term review and restructuring provided relevant, helpful, and timely modifications. Midway through implementation, the Program conducted a detailed mid-term review and a detailed Impact Evaluation (see footnote 43), which together informed the Level 2 restructuring. In these reviews, the Bank team and client worked together to assess the Program progress to date, identify implementation bottlenecks, and prepare modifications for the restructuring. During the mid-term review, it was decided to simplify the linked DLIs, strengthen the tracking of sanitation sustainability in commune institutions, and recognize implementation delays and extend the Program to ensure the WSS sustainability targets could be met. This review informed practical revisions to the DLIs, results framework, and POM that ultimately helped to improve Program delivery and results achievement in the second half of implementation.


Page 24 of 61

58. The “softer” IEC and BCC activities helped to achieve the “harder” infrastructure results. Initially, the IEC and BCC for water, sanitation, and hygiene were undervalued – with less funding allocated early on and delayed implementation of these activities. However, through Program implementation, a more consolidated understanding was developed among the implementing agencies of the fundamental importance of these types of activities to increase awareness, trust, and participation of rural residents/customers in the RWSS systems – including building household latrines and connecting to the piped water systems. When IEC and BCC activities were implemented, together with the other “hard” interventions, better Program results were achieved in shorter time. In particular, there was evidence that increased attention to IEC and BCC in provinces or regions that were lagging led to a quick turnaround in performance and ultimate achievement of the DLIs. Strengthening the role of social and health workers at the village and commune levels to carry out the IEC and BCC activities was found to be the most important aspect of these achievements.

59. Independent results verification incentivized improved results during implementation. As mentioned, there is strong evidence that verification incentivized improved results, and relying on an official government agency (SAV) to take leading responsibility over the verification process helped to develop important internal capacity that is now benefiting other operations. Notwithstanding these improvements, the verification process also faced a number of challenges. The verification methodology was newly developed for this PforR, which took additional effort by the Bank and the client and would vary project-by-project. Also, the costs of statistically reliable verification were high in the early years but went down as knowledge and capacity increased.

60. The PforR concept was a new approach in the water sector in Vietnam and produced significant benefits, although this is still a continuous learning process. As noted, the Program was successful in meeting all of its intended outcomes and the client gained significant institutional capacity. However, since this was the first PforR in Vietnam, some aspects could be improved for even better results and to maximize the PforR instrument. The Program was designed with central-level targets for all of the DLIs (followed by provincial level targets set at the preparation stage according to each provinces’ needs, capacity, and financial planning); and although the PforR performance incentive worked quite well at a central level (all DLIs and outcome targets were exceeded), a mechanism for adaptable performance-based transfers from the central government to provinces was not included in the original Program design, as it was not available in the Vietnam budgeting and transfer system. As a result, throughout implementation, disbursed Program funds continued to be transferred to provinces using the standard internal NTP/NRD procedures (following the original provincial sub-targets assigned at preparation). This constrained the full potential of the new PforR approach to incentivize provinces to flexibly adjust their yearly performance and be rewarded with corresponding results-based budget allocations based on their actual performance. Only in the last implementation period were some provinces’ targets and corresponding budget allocation revised (increased) to reflect their higher performance.

61. Pre-financing of investments and counter-part funds allocation were challenges. During early implementation years, provinces encountered cashflow challenges because most had limited capacity to pre-finance investments due to the low availability of counterpart funds. The PforR allowed advance payments of up to 25 percent, but in some provinces cashflow problems remained, and some found it difficult in early years to meet their annual DLI targets. However, the most successful Provinces were those that were able to identify methods to pre-finance construction from other sources, such as by pre-financing from other local programs, or relying on private sector contractors to finance the construction while the program waited for disbursements under the PforR. This situation was mitigated over time through: i) improved provincial planning that made more realistic requests for resources; and ii) enhanced coordination among central ministries (MARD, MoF, and MPI) to assign and release timely transfers to the provinces.


Page 25 of 61

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION

M&E Design

62. In the previous NTP (before the PfoR), data from the existing RWSS M&E system were often considered unreliable due to a weak data collection process, and were mostly used by central-level authorities for statistical purposes in national reports. The previous M&E focused mostly on water supply only. The PforR Program M&E took into account the existing NTP M&E 14 WSS indicators and formed the basis for provinces to systematically collect data for use in their annual investment planning and tracking purposes. Data collection process was improved to take place on an annual cycle linked to the national planning and budgeting processes, with close participation from the PCERWASSs and NCERWASS; and also from the provincial Departments of Health (DoH). Specific to the PforR, the Program introduced six new DLIs, four PDO Indicators, and a number of intermediate outcome indicators, which also had to be tracked and reported annually (see the Results Framework (RF)). All of the RF indicators and DLIs were well-defined, well-linked to the PDO, and provided meaningful metrics for implementing agencies to monitor relevant Program outputs and impacts. The indicators and DLIs had clear baselines and realistic targets; all were clearly defined in the POM with detailed sustainability and CWS requirements; and all were formally revised, as needed.

M&E Implementation and Utilization

63. During Program implementation, data was collected at the local level (commune and district), compiled and processed by PCERWASS at the provincial level (with support from the DoH), and summarized at the national level by NCERWASS. Program M&E was generally complete, timely, and efficient. There were no major M&E compliance or quality issues. The DLI verification process added a robust annual audit of reported achievements, including a full paper audit followed by physical inspections for a sample of reporting communes. Verified DLI achievements were formally documented in Annual Verification Reports. The verification protocol was well-designed by Bank experts, following the best practices for PforR M&E and using rigorous statistical methods. Training for verification and M&E was provided regularly to the implementing agencies and to SAV, as the Program’s IVA. This was the first PforR to be verified by SAV, and the capacity built has allowed SAV to expand and conduct verifications in other PforR’s in Vietnam.

Justification of Overall Rating of M&E Quality

64. The overall quality of the Program M&E is rated as SUBSTANTIAL. The Program had a transformative effect on how RWSS systems are tracked and monitored in Vietnam. The government counterpart successfully carried out regular, well-designed M&E activities and took active steps to improve the NTP/NRD M&E systems using lessons and experience from the PforR (see paragraph 41). The Bank and the government counterpart agencies provided meaningful training and capacity building on M&E, and the Program M&E sustainability definitions were integrated into the revised NRD M&E system.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

Environmental and Social (E&S)

65. At preparation, the Environmental and Social Systems Assessment required for PforRs concluded that no significant, adverse environmental or social impacts were anticipated and that any E&S issues (e.g. land acquisition and ethnic minority issues) were satisfactorily addressed in the PAP and POM. During implementation, planning and monitoring of E&S aspects were carried out satisfactorily, albeit with some delays. The Bank was proactive to monitor E&S and raise issues during supervision missions, and the client was responsive to address the issues. At Program close,


Page 26 of 61

all required E&S reports were submitted and approved.58 Any land-acquisition-affected households were fully compensated following government regulations. Among Program provinces, only Quang Ninh had beneficiaries who were both ethnic minorities (EM) and land-acquisition-affected people.59 Guidelines in the PAP and POM for consultation and participation of EM beneficiaries were followed. PECERWASS conducted 27 consultations with 1,471 EM participants, of which 66 percent were female.

66. One environmental risk area is proper disposal of fecal sludge after it has been collected from households and institutions in Program communes. This is a common challenge in Vietnam. As noted, there is general awareness among Program-supported households about latrine and septic tank maintenance, and there are sanitation service companies who regularly empty septic tanks in Program areas. These companies are required to be licensed and to dispose of sludge in WWTPs or landfills. However, there is a recognized regulatory and enforcement gap, and safe and proper sludge disposal does not happen every time. This is a larger regulatory matter that needs to be addressed by the Government and is beyond the scope of this operation.

Fiduciary

67. During preparation, the Bank (in close collaboration with the client counterpart) conducted a thorough Fiduciary Systems Assessment, which reviewed existing procurement and financial management systems used in the NTP/NRD to identify a discrete list of improvements to be included as actions in the PAP and POM – including actions to improve auditing, competitive procurement, transparency, and anti-corruption. Where capacity building was needed, technical assistance activities were included under the TA. During implementation, the client executed the PAP successfully, with only relatively minor complications or delays. No major procurement and financial management problems were encountered, and the Bank was overall satisfied with Program fiduciary management.

68. By Program close, all of the DLI targets were over-achieved, but only 90.34 percent of Program financing was disbursed.60 When the Program was prepared, contributions from other funding sources was uncertain and the PforR financing was conservatively estimated to fill the gap. However, as noted above, the PforR catalyzed significant private sector investment in the RWSS sector, which offset the total investment required from the GoV. Therefore, the full IDA/Bank credit was not needed in order to meet all of the outcomes. This represents savings for the client.

C. BANK PERFORMANCE

Quality at Entry

69. Bank performance was highly satisfactory during Program preparation and entry. The Bank took ambitious initiative to advocate for the PforR instrument, and this Program was part of the very first generation of PforR operations. The prior RRD-RWSSP project had recently ended, and the default progression was to follow up with a second IPF project. However, the Bank team recognized the potential to embed the PforR approach in the NTP3/NRD and advocated for the client to use the new PforR instrument. When the client agreed, the Bank helped to establish a robust PforR structure to effectively utilize existing country systems. This also allowed the Bank to focus its support on the key technical aspects of water scheme sustainably and community wide sanitation. The Bank team was proactive to mobilize the parallel TA and collaborate with GoV counterparts to build the capacity needed for success. Program preparation included clear and concise PAD and PAP documents that adequately addressed the sector and institutional challenges and risks. The preparation stage technical, fiduciary, environmental, and social assessments of the NTP/NRD were well-done, based on

58 See in Program files: the report on Environmental and Social Management Plan (ESMP) performance from 2017-2018, the report on overall

water supply compliance with the PAP, and the provincial reports on Environmental Protection Commitment (EPC) for sanitation facilities. 59 Thirty-one communes had EM people, eight of which received water supply and sanitations facilities, the others received only sanitation. 60 The deadline for the GoV to request disbursements was extended to March 31, 2020. As such, this report does not reflect the final figures.


Page 27 of 61

thorough analysis, and raised relevant issues and anticipated challenges that were incorporated into the Program design. The Program was aligned with key GoV RWSS priorities and the DLIs were well-designed to achieve results.

Quality of Supervision

70. Bank performance supervising implementation was also highly satisfactory. The Bank conducted regular supervision missions and clearly communicated Program progress, observed issues, and recommendations in timely and detailed aide-memoires and implementation status reports. The Bank provided effective oversight to help ensure that emerging technical issues were resolved, that capacity was built in the implementing agencies, and that all DLI targets were met. In the first years of implementation, when the TA was delayed because of slower than expected procurement, the Bank team proactively stepped in to provide capacity building and handholding that helped provincial agencies to prepare the annual WSS investment plans and carry out other Program activities. In particular, the Bank provided timely and relevant analyses during the MTR and restructuring. The Bank led a solid mid-term impact assessment examining the results to date and identifying implementation bottlenecks, discussed above. Finally, when progress in the sanitation DLIs were lagging in some provinces because latrine costs were too high, the Bank stepped in to introduce and promote new low-cost technologies for rural household latrines. All of these efforts by the Bank supported and complemented the delivery and achievement of Program results and helped the Program run more smoothly and effectively.

Justification of Overall Rating of Bank Performance

71. The performance of the Bank is rated as HIGH. As the first PforR in Vietnam, and the first PforR in the water sector at the Bank, the Bank team played a critical role in preparing and implementing the Program. As noted, the Bank team provided high quality support to design a comprehensive, relevant, and meaningful Program that addressed the key RWSS issues in the provinces. During implementation, the Bank provided high quality supervision and proactive, strategic advice on the most challenging technical and management issues.

D. RISK TO DEVELOPMENT OUTCOME

72. At the time of Program completion, potential risks to the development outcomes were identified:

This program was successful in demonstrating a model at the commune level for water supply schemes to operate under a sustainable regime for a minimum of two years. The WSS sector in Vietnam in general faces risks of rural water systems for not being sustainable in the long-term. However, for the Program-supported water schemes (the majority of which are now operated by private enterprises more oriented to the bottom-line), this risk is assessed to be low to moderate, because their experience meeting the sustainability DLIs each year has established a strong foundation for effective, long-term system maintenance and financial sustainability – with Program-supported water schemes having an average cost-recovery rate of 1.18.

There is a recurrent challenge in Vietnam that low rural water tariff rates do not keep up with the costs of producing piped water supplies, posing risks to water scheme financial sustainability. Tariffs are established at the provincial level by the PPCs. Schemes supported under the Program have largely mitigated this issue by promoting higher connection rates and thus increasing consumption and revenues. Fully resolving this persistent risk in Vietnam will require dialogue between government regulators (at PPC level), local rural water agencies, and water system operators, and ultimately, also changes in the country’s legal framework for the sector.

V. LESSONS AND RECOMMENDATIONS

73. The Program showed that rural water system sustainability can be achieved and incorporated into RWSS planning, investments, and operations. Operational and financial sustainability of rural water systems had been a major weakness and challenge of the NTP/NRD prior to the PforR. Under the Program, sustainability was included in the PDO and comprehensively defined in clear DLIs. In all eight provinces, program-supported water supply schemes met those


Page 28 of 61

DLIs and, still after two years, were operating under recognized management models, with low NRW rates, a high ratio of paying HH connections, and a positive cost recovery ratio. The clear sustainability definition encompassed operational, institutional, and financial key elements; and was employed and monitored under the Program to shift the mindset and financially incentivize Provincial implementing agencies away from a focus only on capital construction and toward a new approach based on demand-driven design and professional, business-minded operations. Sustainability related DLIs and disbursements were a major innovation and success factor. Future operations should incorporate clear sustainability indicators and DLIs in the program RF, with clearly defined criteria specified in the POM and verification protocol.

74. The PforR instrument can work well, even in contexts with low-capacity governance systems; however, enhanced technical assistance will be needed. When the Program was being prepared, it was recognized that the existing NTP/NRD had limitations and challenges around institutional capacity at the Provincial level. The Bank team took a calculated risk to advocate using the new PforR instrument, when the default would have been a follow-on IPF to the prior RRD-RWSSP. By Program close, all of the DLI’s had been successfully achieved, despite the capacity constraints. The PforR instrument can be successful in low-capacity client settings, can help to address long-term institutional barriers, and can reshape sector incentives. Future PforR operations should carefully consider use of the instrument, and sufficient scope and funding should be included for TA and capacity building. Given that PforR operations are implemented as budget support to existing government programs, it is recommended to provide specific additional funding for TA and capacity building through grant support (as was the case for this Program), through an IPF component, and/or though enhanced implementation support from the Bank. Technical assistance should be synchronized to Program implementation, and capacity building should be targeted to the provinces.

75. Program DLIs should be kept simple and clear. At the MTR, the original set of linked water and sanitation DLIs were split into six independent DLIs, and the revised DLIs were made scalable with no annual ceilings. These revisions simplified the DLI verification and disbursement processes and helped to allow all of the DLI targets to be fully met following the restructuring. Simple and clearly defined DLIs are easier and more straightforward to implement, especially when different implementing agencies have responsibility for different DLIs and on different timetables. Future operations should keep the design of Program DLI's simple and scalable to allow more flexibility for implementing agencies to manage and adapt their implementation progress.

76. Results verification under the PforR was a driving incentive for results achievement and institutional strengthening. During the first year of implementation, because of a lack of familiarity, some Program provinces struggled with the new PforR approach and missed their first year DLI targets. However, because of the results-based verification process, officials quickly learned that missed DLI’s triggered reduced Program disbursements. This was a real lesson in results-based performance assessment, and this initial “setback” drove a change in institutional mindset towards serious consideration of realistic targets, strategic commune selection, and more-robust facility designs in the subsequent years of implementation. The verification-based disbursement process under the PforR was the main learning-by-doing tool that provided a genuine incentive for institutional change and results achievement. Meeting the annual verification targets also helped to improve the quality of infrastructure constructed, and reduce the time taken to build new water and sanitation systems and facilities. Future operations should fully leverage the PforR verification process as an incentive by ensuring that the Program objectives and DLI's are clearly defined and that the verification protocol is practical and robust. Future operations (especially when newly applied within a country, region, or sector) should also be prepared to remain adaptable, ready to accept some degree of early period failure and learning, but backing it up with timely technical assistance to improve planning and catch up for the next periods’ targets.

77. Results-based financing can be effective, but the PforR should be carefully designed with procedures that maximize PforR benefits. In this Program, the full results-based concept of the PforR at times did not fit the central government processes, especially at the provincial level where the implementation was carried out. It would be beneficial to the RWSS sector to find mechanisms that make center-to-province budget allocations and transfers more


Page 29 of 61

results-based, thus institutionalizing the PforR approach. Future operations should conduct careful analysis and simulations to ensure that government program and PforR Program procedures, funding flows, and pre-financing arrangements are synchronized and effective. These can then be formalized into the Program financing agreement, PAP, or POM.

78. The Program was a catalyst for significant private sector participation in the RWSS sector – attracting new private investment and more private enterprises from inside and outside Program provinces to finance and operate water schemes in the Program areas. The private sector operators brought a business-minded perspective which helped to improve the quality of water supply services delivered to rural residents in the Program areas while also helping to achieve the water sustainability criteria of the DLIs. The PforR's promotion of clear, time bound RWSS investment plans, the improved certainty around financing, and the focus on operational and financial sustainability helped to give confidence and attract private enterprises to invest. Future operations, whether PforR or otherwise, should ensure that clear and transparent RWSS planning capacity for counterpart institutions and results-based performance standards are embedded in program/project designs so that an enabling environment is created to attract private sector participants. .


Page 30 of 61

ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN

Annex 1A. RESULTS FRAMEWORK (i) PDO Indicators Objective/Outcome: Increased and sustained access to water supply services

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Number of people with water supply connection from Sustainable Water System

Number 0.00 850000.00 1054575.00

01-Jan-2013 01-Jan-2013 31-Jul-2019

Comments (achievements against targets): This indicator achieved 124 percent of its target value. All figures were independently verified.


Target


People provided with access to improved water sources

Number 0.00 1496000.00 1759842.00

16-Jun-2016 16-Jun-2016 31-Jul-2019

People provided with access to improved water sources - rural

Number 0.00 1496000.00 1759842.00


Page 31 of 61

Comments (achievements against targets): This indicator achieved 118 percent of its target value. This indicator measures the cumulative number of people who benefited from improved water supply services that have been constructed through operations supported by the World Bank. And this is an additional core indicator introduced by the Bank during the program implementation. All figures were independently verified.


Target


Direct project beneficiaries (number) of which female (percentage)

Number 0.00 1861000.00 2149765.00

16-Jun-2016 16-Jun-2016 31-Jul-2019

Comments (achievements against targets): Of which female: 51 percent. This indicator achieved 116 percent of its target value. All figures were independently verified. Demographic data source: Vietnam General Statistics Office (GSO) website, www.gso.gov.vn.

Objective/Outcome: Increased and sustained access to sanitation services


Target


Number of people with access to Community-Wide Sanitation

Number 0.00 1275000.00 1407669.00

01-Jan-2013 01-Jan-2013 31-Jul-2019


Page 32 of 61



Target


People provided with access to improved sanitation services

Number 0.00 1275000.00 1407669.00

01-Jan-2013 01-Jan-2013 31-Jul-2019

People provided with access to improved sanitation services - rural

Number 0.00 1275000.00 1407669.00

Comments (achievements against targets): This indicator achieved 110 percent of its target value. The indicator measures the cumulative number of people who benefited from improved sanitation facilities that have been constructed through operations supported by the World Bank. And this is an additional core indicator introduced by the Bank during the program implementation. All figures were independently verified.

Objective/Outcome: Improved sector planing, monitoring and evaluation


Target


Number of Provincial Annual Number 0.00 85.00 102.00 102.00


Page 33 of 61

Plans and Program Reports from Participating Provinces

01-Jan-2013 01-Jan-2013 16-Jun-2016 31-Jul-2019

Comments (achievements against targets): This indicator achieved 100 percent of its revised target value. This indicator target was based on eight annual provincial investment plans and nine annual program reports (i.e. eight provincial plus one consolidated report) each year. Original target assumed five years. After restructuring, target was revised for six years of implementation. All figures were independently verified.

(ii) Intermediate Results Indicators

Results Area: Adequate water and sanitation infrastructure is in place and in working condition


Target


Number of new Functioning Water Supply Connections

Number 0.00 340000.00 399964.00

01-Jan-2013 01-Jan-2013 31-Jul-2019


Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at


Page 34 of 61

Target Completion

Number of newly constructed Improved Household Sanitary latrines

Number 0.00 130000.00 142280.00

01-Jan-2013 01-Jan-2013 31-Jul-2019



Target


New water supply and sanitation facilities in schools, and health centers constructed and in use

Number 0.00 1440.00 1559.00

01-Jan-2013 01-Jan-2013 31-Jul-2019



Target


Number of communes where Number 0.00 150.00 179.00


Page 35 of 61

schools and health centers have maintained hygienic status

16-Jun-2016 16-Jun-2016 31-Jul-2019

Comments (achievements against targets): This indicator achieved 119 percent of its target value. This indicator was added during the restructuring. All figures were independently verified.

Results Area: Monitoring, Supervision and Reporting of the overall Program


Target


Program Report from MARD disclosed to the public

Number 0.00 5.00 6.00 6.00

01-Jan-2013 01-Jan-2013 16-Jun-2016 31-Jul-2019

Comments (achievements against targets): This indicator achieved 100 percent of its revised target value. This indicator target was based on one consolidated Program report to be disclosed each year. Original target assumed five years. After restructuring, the target was revised for six years of implementation. All figures were independently verified.

ANNEX 1B. DISBURSEMENT LINKED INDICATORS


Page 36 of 61

DLI IN0073 1753 TA BLE

DLI 1: DLI I. Number of new Functioning Water Supply Connections (Number)

Baseline Prior Results 2013 2014 2015 2016 2017 2018 Total

Original values 0.00 0.00 0.00 51,541.00 141,541.00 231,541.00 301,541.00 340,000.00

Actual values 0.00 0.00 51,541.00 134,660.00 218,625.00 329,408.00 399,964.00

Allocated amount ($) 0.00 0.00 0.00 0.00 0.00 0.00 56,772,000.0

0 56,772,000.00

Disbursed amount ($) 0.00 0.00 0.00 0.00 0.00 0.00 55,235,955.2

8 55,235,955.28

Comments (achievements against targets):

Currency is reported in SDR and the disbursement are shown as per January 20, 2020. This indicator achieved 118 percent of its target value. All results achieved were independently verified. Annual financial allocation and respective disbursements are not available. In 2016, the DLI schedule was restructured to be fully scalable, resulting in only aggregated end-of-project DLI allocations. The Actual Amount figures shown represent end-of-program total disbursements converted using the currency exchange rate on January 20, 2020, which may differ from the figures shown in the Financing table on page 1 of the ICR (see the note in Annex 3 for more detail).


DLI 2: DLI II. Number of newly constructed Improved Household Sanitary latrines (Number)


Original values 0.00 0.00 0.00 30,000.00 60,000.00 90,000.00 117,050.00 130,000.00

Actual values 0.00 14,216.00 30,578.00 59,234.00 90,033.00 124,041.00 142,280.00


0 21,445,000.00


Page 37 of 61


6 20,555,236.96




DLI 3: DLI III. Number of people with water supply connections from Sustainable Water System (Number)


Original values 0.00 0.00 0.00 0.00 0.00 205,000.00 630,000.00 850,000.00

Actual values 0.00 0.00 0.00 0.00 284,865.00 634,780.00 1,054,575.00


0 12,750,000.00

Disbursed amount ($) 0.00 0.00 0.00 0.00 0.00 0.00 9,894,328.71 9,894,328.71




Page 38 of 61


DLI 4: DLI IV. Number of people with access to Commune-Wide Sanitation (Number)


Original values 0.00 0.00 250,000.00 500,000.00 750,000.00 1,000,000.00 1,250,000.00 1,275,000.00

Actual values 0.00 170,949.00 519,925.00 729,980.00 932,447.00 1,361,501.00 1,407,669.00


0 25,950,000.00


4 21,362,856.74




DLI 5: DLI V. Number of communes where schools and health clinics have maintained hygienic status. (Number)


Original values 0.00 0.00 0.00 0.00 0.00 50.00 100.00 150.00

Actual values 0.00 0.00 0.00 0.00 49.00 93.00 179.00


Page 39 of 61

Allocated amount ($) 0.00 0.00 0.00 0.00 0.00 0.00 4,720,500.00 4,720,500.00

Disbursed amount ($) 0.00 0.00 0.00 0.00 0.00 0.00 3,070,360.53 3,070,360.53




DLI 6: DLI VI. Number of Provincial Annual Plans approved by participating provinces and Number of Program Reports disclosed to the public (Number)


Original values 0.00 0.00 17.00 34.00 51.00 68.00 85.00 102.00

Actual values 0.00 17.00 34.00 51.00 68.00 85.00 102.00

Allocated amount ($) 0.00 1,643,750.00 1,643,750.00 1,643,750.00 1,643,750.00 1,643,750.00 1,643,750.00 9,862,500.00

Disbursed amount ($) 0.00 1,643,750.00 1,643,750.00 1,643,750.00 1,643,750.00 1,643,750.00 457,145.02 8,675,895.02


Currency is reported as SDR. This indicator achieved 100 percent of its target value. All figures were independently verified. This was the only DLI that remained non-scalable over the duration of the Program.


Page 40 of 61

ANNEX 1C. PROGRAM ACTION PLAN

PAP_T BL

Action Timing Achieved (Yes/No)

Completion Measurement

1. Participating Provinces enhance transparency with public consultation/corruption/fiduciary/ procurement/ Program databases on implementation complaints and responses. MARD will submit an aggregated report to the Bank on a periodic basis

Recurrent Semi-Annually Yes Transparency platform and call center are fully operational.

Comments:

The transparency platform is in place and has been regularly updated by the implementing agencies (except for Hung Yen province with no documents updated in 2018). The beneficiary feedback system is in place and NCERWASS/PCERWASS/DARDs have been tracking the comments.

2(a) Procurement methods will be enhanced: All proposals for detailed designs/ construction supervision/ bids for civil works shall be evaluated

Recurrent Semi-Annually Yes Procurement reports prepared by MARD and provinces.

Comments:

According to a report from MARD, no bid or proposal was rejected due to the offered price exceeding the cost estimate. This action has therefore been complied with.

2(b) For water supply sub-projects, at least Recurrent Yearly Yes Procurement reports prepared by MARD and


Page 41 of 61

50% of consultancy contracts and 50% of works contracts will be competitively bid, increasing to 80% by end of the Program period

provinces

Comments:

As reported by MARD, this action has been complied with. Water supply schemes financed by private entities, for which procurement is not governed by national procurement laws and regulations, were not monitored in this clause.

2(c) Dependent (Province/MARD) SOEs are excluded from participating

Recurrent Yearly Yes Procurement reports prepared by MARD and provinces.

Comments:

As reported by MARD, no SOEs participated in the works that generated results for the Program.

2(d) Firms on the local/national/Bank debarment list not allowed to participate

Recurrent Yearly Yes Procurement reports prepared by MARD and provinces.

Comments:

As reported by MARD, no debarred firms participated in the works that generated results for the Program.

3.Government will build and implement effective Program financial and results audit capabilities: prepare annual Program financial statements; create Program internal audit function; appoint external auditor with acceptable audit scope and

Recurrent Semi-Annually Yes Audit reports delivered on time and satisfactory quality


Page 42 of 61

approach

Comments:

This action was completed satisfactorily.

4. Provinces will avoid/minimize land acquisition impacts and compensate people affected by asset/land/income losses. An efficient, independently verifiable market valuation mechanism, accepted by the Bank, will be created for lost assets/land/income

Recurrent Continuous Yes Reports prepared by MARD and provinces.

Comments:

Land acquisition, compensation, and resettlement have been successfully completed in all project provinces. Influx of labor was monitored during the last two periods of program implementation and there were no issues reported.

5. The provinces will develop and implement guidelines, acceptable to the Bank, to work with Ethnic Minorities that shall fully operationalize existing Vietnamese Legislation on Ethnic Minorities through free, prior, and informed consultations

Recurrent Continuous Yes Reports from MARD and provinces.

Comments:


Page 43 of 61

The Ethnic Minorities Development Plan was prepared and implemented by PCERWASS in the two provinces that have ethnic minority people. Ethnic minority people were provided with information on the Program including activities, benefits, implementation schedule, and compensation.


Page 44 of 61

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Parameswaran lyer Task Team Leader

Hoa Thi Hoang Co-Task Team Leader

Victor Vazquez Alvarez Water and Sanitation Specialist

Dean Cira Lead Urban Specialist

Joseph A. Gadek Senior Sanitary Engineer

Hisham A. Abdo Kahin Senior Counsel Legal

Huong Thi Mai Nong Junior Counsel Legal

Yuling Zhou Lead Procurement Specialist

Hoai Van Nguyen Procurement Specialist

Giang Thi Huong Nguyen Program Assistant

Nghi Quy Nguyen Social Development Specialist

Hoa Thi Mong Pham Sr. Social Development Specialist

Son Van Nguyen Environmental Specialist

Robert Gilfoyle Senior Financial Management Specialist

Kien Trung Tran Senior Procurement Specialist

Mai Thi Phuong Tran Financial Management Specialist

Supervision/ICR

Lilian Pena Pereira Weiss, Hung Sy Pham Task Team Leader(s)

Hoai Van Nguyen Procurement Specialist(s)

Mai Thi Phuong Tran Financial Management Specialist

Cecilia Belita Team Member

Pratheep Ponraj Team Member

Maya Gabriela Q. Villaluz Environmental Specialist

Berta Adelaide Da Silva Macheve Team Member

Hang Diem Nguyen Team Member

Phyrum Kov Team Member

Linh Thi Thuy Tran Team Member

Claire Chase Team Member

Thang Duy Nguyen Social Specialist

Ha Thi Thu Nguyen Team Member

Luis Felipe Duchicela Team Member


Page 45 of 61

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY12 67.911 475,562.98

FY13 27.317 193,167.64

Total 95.23 668,730.62

Supervision/ICR

FY13 30.867 467,170.85

FY14 60.251 412,139.69

FY15 54.862 326,047.29

FY16 37.998 362,985.23

FY17 25.117 154,501.80

FY18 33.684 130,011.12

FY19 23.691 126,044.35

FY20 15.217 102,715.52

Total 281.69 2,081,615.85


Page 46 of 61

ANNEX 3. PROGRAM EXPENDITURE SUMMARY

Source of Program Financing (US$)

Type of Co-Financing Estimates at

Appraisal (US$ million)

Actual Expenditures (Disbursement)

Actual (US$ million)

Percentage of Appraisal

Percentage of Actual

1. World Bank(i) 200 164.01(ii) 82.05(iii) 90.33 (iii)

2. Borrower and users(i) 45(iv) 10.91 24.24 24.24

3. Other Funds(v) 15 123.84 825.6 825.6

3.1 NTP3 Other NTP3 funding contributions from central and provincial governments

0 18.67

3.2 NRD NTP Other NRD NTP funding contributions from central and provincial governments

0 8.30

3.3 Concessional Credit Social and Policy Bank of Vietnam(vi) 15 60.60

3.4 Users Contribution from users to construction costs 0 1.77

3.5 Private sector investments

0 34.50

Total (1+2+3) 260 298.76 114.91 164.56

(i) Figures provided (and audited) by SAV and client connection. Disbursements based on January 20, 2020. Based on SAV’s audited reports only (rows 1 and 2 above), the Program of

expenditures are VND 3,791,137,619,269, which is equivalent to US$ 172,016,000 using annual exchange rates in the Withdrawal Applications as provided by MARD.

(ii) Note that the actual expenditure (disbursement) of the World Bank loan shown in this table differs from the figure shown in the Financing table on page 1 of this ICR. The figures on page 1 use the cumulative sum of disbursements over program implementation at variable currency exchange rates at the time of disbursement, whereas this table uses the final total disbursement value at a currency exchange value taken in January 2020.

(iii) Allocation at approval was SDR 131,500,000, equivalent to US$ 200,000,000. By January 20, 2020, the US$ equivalent to approved SDR allocation is US$ 181,550,215, due to valuation of SDR (thus reducing the IDA allocation in actual terms in US$). The disbursements of US$ 164,009,058.54 are equivalent to SDR 118,794,633.20 (on January 20, 2020).

(iv) At approval, the estimation of US$ 45 million from borrower/recipient included: US$ 20 million from central government; US$ 10 million from provinces and US$ 15 million from users.

(v) Figures provided by MARD (unaudited by SAV).

(vi) Loans provided to users to connect to piped water systems and to build latrines.


Page 47 of 61

ANNEX 4. BORROWER’S COMMENTS

Evaluation on Completion of Results-Based Rural Water Supply and Sanitation Under the National Target Program in 8 Red River Delta Provinces, Financed by the World Bank (Credit Number: 5176-VN)

MINISTRY OF AGRICULTURE AND RURAL DEVELOPMENT WATER RESOURCES DIRECTORATE

Hanoi, Vietnam July 2019 SUMMARY OF REPORT PRESENTED – TRANSLATED AND CONDENSED FROM THE ORIGINAL (The full Borrowers report in Vietnamese can be found in the project files)

SECTION 1 INTRODUCTION

BACKGROUND 1. The National Strategy for Rural Water Supply and Sanitation (National Strategy for RWSS) up to 2020 was approved on August 25, 2000. The National Target Program on Rural Water Supply and Sanitation (RWSS NTP) - in three phases (Phase I from 1999 to 2005, Phase II from 2006-2010, Phase III from 2012-2015) - has contributed to realizing and speeding up water supply and sanitation services for rural people, thereby contributing to improving living standards and reducing poverty for rural areas. After 15 years of implementing the RWSS NTP however, challenges have been encountered, for example: water supply services do not adequately meet the needs of the people, service effectiveness is very low, a high ratio of non-revenue water, and many schemes have not been properly maintained and do not guarantee service quality. In addition, investments have mostly focused on new construction. As a result, the RWSS NTP has not fully achieved the objectives set out in the National Strategy for RWSS, especially for sanitation; the main issues being budgets favoring water supply over sanitation and a lack of capacity at all levels of government.

2. In this context, this Program was implemented using output-based disbursement (i.e. PforR). RWSS in Vietnam is an appropriate sector for such support because the large National Target Program is already in place. The PforR program was implemented from 2013 to 2018 in eight provinces (Hanoi, Ha Nam, Hung Yen, Bac Ninh, Quang Ninh, Vinh Phuc, Phu Tho and Thanh Hoa) and in around 240 communes with investments in the construction of centralized water supply schemes and sanitation facilities in schools and health centers, as well as with activities to change sanitation behaviors among participating program provinces. The results achieved contribute to improved household water supply and sanitation services in rural areas and to the realization of the National Strategy on RWSS. The Program also supports implementation of the Comprehensive Poverty Reduction and Growth Strategy and Vietnam's commitments towards achieving the MDGs on RWSS.

PROGRAM OVERVIEW Main Expected Results

3. The main outcomes of the Program were defined in the six Disbursement-Linked Indicator (DLI) targets:

DLI I “Number of new functioning water supply connections”: 340,000 household water connections provide clean water to 1.7 million people in 240 communes;

DLI II "Number of newly built or rehabilitated household hygienic latrines": 130,000 latrines; DLI III "Number of people with water supply connections from sustainable water systems": 850,000 people; DLI IV "Number of people benefiting from commune-wide sanitation communes": 1,275,000 people; DLI V "Number of communes where schools and clinics maintain hygienic status": 150 communes; DLI VI "Number of approved annual plans and report disclosed": 48 plans, 54 reports.


Page 48 of 61

Program Financial Mechanism

4. The World Bank loan proceeds were allocated and used in accordance with the State Budget Law. The annual budget of the Program was allocated to eight provinces along with the budget of the RWSS National Target Program but in separate budget lines. The World Bank loan was divided into two parts: one part as a grant to implement Program activities at the Central level, and the remaining part on-lent to the Provincial People's Committees (PPC). Counterpart funds from the State budget were allocated to Program implementation activities at the central level. Counterpart funds from the provincial budget were used for Program activities at the local level. Community contributions were used to construct connections to the water supply system (accounting for 10% of scheme's total capital construction investment).

Changes during Program Implementation

Amendment of the Financing Agreement 5. Following Bank's recommendations provided in the mid-term review mission in June 2016, the Government of Vietnam carried out the following changes:

Extend by one year the Program implementation (to July 31, 2019). Add DLI V: "Number of communes where schools and clinics maintain hygienic status" (target of 150 communes). Modify the Program Feasibility Study Report.

SECTION 2. EVALUATION OF PROGRAM IMPLEMENTATION RESULTS Program implementation and management At the Central Level

6. Before 2016, the Program was operated within the framework of the RWSS NTP Phase III. The close implementation monitoring along with the close coordination among the central agencies has clearly shown benefits to the roles, functions, and tasks carried out by the ministries and sector agencies in managing and administering the Program. From 2016 onwards, after the completion of the RWSS NTP Phase III, despite multiple staffing changes that caused many difficulties in implementation, the Coordination Board completed its assigned tasks. During this period, a number of new legal documents on management and use of ODA, concessional loans, and management of public debt were issued and these had certain impacts on the management, administration, implementing the PforR Program.

At local level

7. Through implementation in the provinces, facts show that any province that had the strong interest and leadership of the provincial leaders on the PforR Program, and on the rural water supply sector in general, achieved better performance. Experience also shows that to ensure the effective implementation of the PforR Program, there should be close coordination between departments and agencies at the local level. In particular, the Department of Agriculture and Rural Development has done a good job with the responsibilities of the focal point.

Planning and target allocation

8. Since 2016, the PforR Program was in the list of programs and projects, so the formulation and consolidation of the annual plans of the Program were required to comply with the provisions of the Law on Public Investment. In the process of transition from ODA-funded programs and projects to programs and projects, due to the weak timely collaboration between three ministries (Planning and Investment, Finance, Agriculture and Rural Development), the tasks of directing and administering the Program, particularly guiding the development and consolidation of the annual plans, faced a number of difficulties related to the budget plan, the DLI targets not being feasible, and DLIs not adhering to the actual needs of fund use and the actual disbursements of the Program.


Page 49 of 61

9. On financial management: Upon the Ministry of Finance's issuance of Circular No. 111/2016 / TT-BTC61, there were new regulations on financial management and many provinces were confused on how to apply these new provisions. Coordination between source and branch account holders of the Program and the State Treasury was not always timely, leading to delays in annual accounting of revenues and expenses of state budget under the PforR Program.

Examination and monitoring of implementation

10. The Water Resource Directorate / Coordinating Office of Rural Water Supply Program performed well the overall coordination role among agencies of the Program in examination, monitoring, and evaluation of program implementation. At the central level, MARD coordinated with the ministries and sectors (health, education and training) to organize missions to inspect and evaluate the implementation results annually.

11. The World Bank conducted two or three Program monitoring and implementation support missions each year. Monitoring focused on the annual budget allocated to the Program, disbursement, verification of results, monitoring of Program implementation, compliance with the Action Plan, and reporting of annual results, etc.

12. The Technical Assistance (TA) resources provided by the Australian Agency for International Development Cooperation (AUSAID) contributed significantly to Program implementation, helping the Program owners to achieve the expected results and scale-up nationwide the application of lessons learned on rural WSS. Specific impacts include:

Capacity building at the provincial and national levels supported better planning, monitoring and evaluation (M&E) and facilitating the verification of results.

Timely support was provided to direct on-site guidance at provinces (through training sessions, etc.). The project built an integrated system using mobile phones, website, and software to monitoring and verify

results and handle and resolve beneficiary feedback using a free call center. Improved management systems strengthened technical capacity for design and procurement, environmental

management and financial management capacity. Improved efficiency and sustainability of the central water supply systems and enhanced sanitation conditions

was provided through training courses, technical documents, and technical assistance. 13. In addition to the achieved results as mentioned above, the TA to the PforR had some challenges, including:

The procurement process for selection of consultants was prolonged. The effectiveness of the assistance provided to some aspects was not timely because of delays. Coordination between agencies for planning, implementing, and reporting results was delayed and not so close.

On financial audit

14. An annual financial audit was carried out by the State Audit of Vietnam (SAV), ensuring compliance with the audit process and requirements. All of the provinces implemented financial audits and complied with SAV’s recommendations.

On results verification

15. The State Audit of Vietnam performed well the role of verifying and certifying the annual implementation results of the Program. However, due to the quality of provinces' reports having multiple errors, verification of results was prolonged and delayed, affecting the annual reporting of results and resulting in delayed disbursements of the program.

61 On financial management of ODA-funded programs and projects on June 30, 2016


Page 50 of 61

Performance on the borrowing funds, counterpart funds other funds for implementation

Investment resources allocated under the RWSS National Target Program, NRD Target Program and the PforR

16. From 2011 to 2018, total budget allocated to RWSS NTP, NRD NTP and the PforR (referred to as three Programs) of eight Red River Delta provinces is VND 6,454.36 billion, in which WB loan is VND 3,517 billion, accounting 57.1%.

Assessment of PforR Program funding performance

17. The total WB loan fund of the PforR Program contributed 57.1% of the total investment capital for the implementation of three Programs from 2011-2018. This is a high contribution rate, which helped to create abundant resources for rural water supply and sanitation. This also showed the important role of the World Bank in supporting the Government of Vietnam to gradually achieve the objectives of the National Strategy on RWSS and achieving the MGDs for RWSS. As of the end of 2018, total disbursement from the loan amount attained 89.3% of total loan amount as per the Financing Agreement. As of the end of 2018, user contribution and local counterpart funds compared with total WB loan amount remained low (respectively 2.4% and 1.3%). Other funding sources for program implementation included the contribution of preferential credits from the Social Policy Bank (accounting for 35.9% of total WB loan).

Compliance with Program Action Plan (PAP) 18. Action 1 on strengthening transparency: Eight provinces established and maintained a database of complaints and accusations on corruption in the procurement and program implementation. For complaints about the Program, the investors provided written reply in a timely manner. Annual report on complaints, denunciations, and corruption were submitted to the World Bank by the Inspectorate of the MARD.

19. Action 2 on procurement: The provinces complied with regulations on procurement such as: ensuring the ratio of competitive bidding among consultancy and works contracts according to the agreed rate on a yearly basis.

20. Action 3 on internal audit and financial audit: The provinces prepared annual financial statements and followed internal audit process. Annual internal audit results report were submitted to the World Bank by MARD.

21. Action 4 on resettlement: According to the detailed verification results, 1,116 households were affected with the area of land acquired of 136,616 m2 of public land and 260,384 m2 of agricultural land; four households had over 30% of cultivated land acquired; no households lost their residential land / houses that had to relocate. The Program implemented technical measures such as selecting reasonable locations to avoid and minimize land acquisition, relocation of affected households, and negative social impacts. The Program implemented a mechanism to valuate compensation prices for land and assets acquired in order to compensate the affected people. The Compensation, Assistance and Resettlement Council of districts of the Program's eight provinces set up compensation and assistance records, paid compensation, and provided support to the affected households. The affected households of the Program fully received all compensation and support in accordance with the State's policies

22. Action 5 on Ethnic Minorities (EM): Only Quang Ninh and Phu Tho had areas where ethnic minorities reside. The provinces selected suitable locations to build water schemes in order to minimize the resettlement of ethnic minority households and to avoid affecting the livelihoods and daily life of citizens. Consultations with EM households were conducted to introduce people to the details of the central water supply schemes that were under construction or would be constructed in the local area. Also included was an introduction of the investment mechanism, investment funding sources and contribution obligations of the people, what benefits people will get from the project, and the rights and obligations of the people to the schemes. The program received support from the ethnic minority communities. The ethnic minority households Quang Ninh and Phu Tho provinces have been using clean water from water schemes built under the PforR.


Page 51 of 61

Adherence to environmental safeguards policy 23. During program implementation, PCERWASS, with the support from environment experts, carried out activities associated with the compliance of the environment safeguard policy for the program, as follows:

The project owner of water supply schemes reviewed and screened the potential environmental impacts of the water supply schemes. Locations where water supply sources were contaminated with arsenic were taken out.

The water of all Program water supply schemes is free of arsenic. Preparation of the environmental protection commitments: all the water supply schemes under the program

made a commitment to environmental protection in accordance with the environmental protection law. The contents of the environmental protection commitments were integrated into Program bidding documents

and in contractors' construction contracts. All construction contracts included a chapter on environmental protection for contractor’s execution and functional agencies' supervision.

Contractors’ compliance with environmental management plans was supervised.

SECTION 3. ASSESSMENT ON THE SUSTAINABILITY OF THE PROGRAM

Water supply component: During implementation, the Program focused on achieving the sustainability DLI of water supply schemes. According to the annual verification reports, the quality of water supplied to the water supply schemes meets the standards of the Ministry of Health (QCVN 02: 2011 / BYT). To ensure the stable and efficient operation of the water supply schemes, the PPCs assigned water supply schemes in the provinces to a number of operators. However, at present, as the provinces are reducing personnel of the management and intermediary focal points, some provinces are considering transforming the management models of their water supply schemes to suit the actual needs. Most of the constructed water supply schemes ensured that the management unit has sufficient revenue to cover the costs of management, operation and maintenance of the water supply schemes. According to the annual verification reports, the rate of NRW in the provinces were all below 25%. Based on evaluation of the Program's water supply component, it is possible to conclude that the water supply schemes built under the Program are in sustainable operation.

Sanitation Component 24. Maintaining the hygienic status of the communes is very important for PforR communes, with impacts spreading positively to neighboring communes and the whole society. Therefore, during implementation, the Program focused not only on construction and upgrading of public latrines at schools and health centers in the communes, but also on a communication and training program to raise the awareness of the people about sanitation through the use of clean water and hygienic latrines. The sanitation component achieved the target of "commune-wide sanitation communes" and also maintained the hygienic status in the commune-wide sanitation communes. The maintenance of hygienic status of the communes will ensure the program's sustainable development.

SECTION 4. LESSONS LEARNT AND RECOMMENDATIONS

EXPERIENCES AND LESSONS IN PREPARATION AND IMPLEMENTATION OF THE PROGRAM 1. At the Central Level

The preparation of the Program must be implemented urgently and receive unanimity from sector line ministries in order to make sure the Financing Agreement (FA) is signed according to the planned schedule.

Upon FA effectiveness, MOF should speed up the procedures for signing sub-loans with PPCs. Line agencies should coordinate with Ministry of Planning and Investment and the Ministry of Finance to provide

guidance to provinces in development of 5-year plans for the Program, and at the same time, carry out procedures to advance funds for the following year to ensure budget is in place for the first implementation year.


Page 52 of 61

Line agencies should play the role of focal points to provide guidance, management, and organization of program implementation. Line agencies should proactively coordinate with related Ministries (i.e. Planning & Investment, Finance, Health, Education and Training, State Bank of Vietnam, State Treasury, etc.) in the planning, financing, supervision, and evaluation of program implementation.

The Project management units should support line agencies in management, coordination, and arrangement of program implementation. In this, the organizational layout, functions, responsibilities and obligations, rights and authorities, and terms of reference for the key positions should be specified clearly.

2. At local level Preparation for investment

The list of communes selected for Program investment must be included in the provincial master plan for rural water supply, based on actual demand for using water, and with customers ready to contribute funds for construction and payment of fee for using water.

Multi-commune water schemes should be prioritized, to take advantage of to reduce investment costs. Selected consulting contractors must have technical capacity and relevant experience, especially consultants for

bidding documents and evaluation of bids. Difficulties during implementation should be envisaged, to include in bidding documents, and to request bidders to propose remedy solutions in their bids.

Selected construction contractors should have construction experience, and especially should have financial capacity to advance funds for construction and completion of the schemes.

Implementing arrangements: Delegation of responsibilities should be defined specific to each agency implementing the PforR. Annual budget allocations should be clear, focused, and complete - without scattered, piece-meal budgets. The capacity for audits, inspections, and examinations should be strengthened for implementation. Leading agencies, companies, and communes in the areas of environmental protection and clean water supply

should be identified and commended each year. This will help to create a visible forum for exchange, learning, and dissemination of good experiences in construction, communication, and scale-up.

A manual to guide design of rural water supply and sanitation facilities should be issued, to help ensure consistent application of advanced models for water treatment and sanitation technologies.

RECOMMENDATIONS ON ENSURING EFFECTIVENESS AND SUSTAINABILITY OF THE PROGRAM Water supply schemes built using PforR program funds, for the first period after completion should use the

PCERWASS management model while they continue to be refined and gradually strengthened. After one or two years, the O&M unit should be converted to a joint stock company and transferred to enterprises.

In some rural areas where people's lives face a lot of difficulties without conditions for using clean water and where people live in scattered areas, it is necessary to have policies for providing subsidies to the rural water supply stations so as to ensure their revenue for proper O&M of the schemes.

Efforts should be continued to communicate, disseminate, and mobilize people to use clean water to enhance their lives and minimize diseases resulting from non-hygienic water sources.


Page 53 of 61

ANNEX 5. SUMMARY OF TECHNICAL ASSISTANCE IMPLEMENTATION AND COMPLETION REPORT

This is a summary of the Implementation Completion Report (ICR) for the Recipient-Executed AusAID Trust Fund (TF 13061) financed by the Australian Government, completed in 2016. Further detail can be found in the full TA ICR document in the program files.

TA Objective 1. The objective of the Technical Assistance was to provide capacity building support to the implementing and related agencies of the Results-Based Rural Water Supply and Sanitation Program under the National Target Program. This objective was intended to be reached by building capacity at the provincial and national levels, focusing on supporting better planning and M&E practices, facilitating results verification, improving governance and strengthening the technical capacity for the design and procurement of cost-efficient piped water supply systems, sustainability of water supply operations and facilitating the scaling up of improved sanitation.

Components and Activities 2. The TA was split into six separate components, each of which are split into separate consultant packages. Each of these Components and Packages are listed below defining the main activities that were planned for each package:

Component A. Planning, Monitoring and Evaluation: The main objective of this component was to strengthen the monitoring capacity of the implementing agencies of the National Target Program (NTP) for Rural Water Supply and Sanitation in the 8 Red River Delta Provinces, and to promote transparency and social accountability in the NTP through improved M&E practices, implementation of a Beneficiary Feedback Mechanism, and the development of an IT Transparency Framework

Component B. Improvement of the Sustainability of Water Supply Systems and Services: The objective of this component was to improve implementing agencies’ understanding of best-practices in the technical design of rural water supply systems and build capacity at the provincial level to better manage and more actively participate in the design process. This component also focused on strengthening the capacity of Water Supply Systems’ management teams to ensure the sustainability of the operations through improved financial management practices, operations and maintenance, and water quality management.

Component C. Capacity Building to Promote and Provide Sustainable Sanitation Services: This component focused on strengthening the capacities of the implementation agencies under the NTP rural sanitation component to deliver the expected targets under the Program and to increase demand for sustainable hygienic sanitation. Activities within this component were aimed at improving capacity for strengthening demand and supply of hygienic sanitation in eight Red River Delta Provinces including the development of annual Sanitation Action Plans, and a Community Rewards Mechanism.

Component D. Improvement of Governance: This objective of this component was to strengthen capacity at the national and provincial implementing partners in the management and coordination of the annual planning process, the financial and procurement structures and management capacity, and the capacity to coordinate and enforce environmental and social management policies.

Component E. Verification of Program results and External Audit of the Program: The objective of this component was to build the State Audit of Vietnam’s (SAV) capacity to implement large scale independent verification surveys. The main activity of this component is to conduct the annual verification of results for the PforR program reported by the provinces.

Component F. Communication and Information Sharing: This objective of this component was to support the Rural Water Supply and Sanitation partnership (RWSSP) is to organize workshops and working sessions to share the lessons learnt on the PforR program with the Partnership members and to analyze implementation of best practices in other provinces.


Page 54 of 61

Changes During TA Implementation 3. During implementation, the TA encountere a number of challenges and had to be revised. The most significant challenges were:

o Significant delays in procurement of key consultant packages; o Budget cuts and scope reductions due to changes in the Australia Partnership Program with the World Bank

in Vietnam (the trust fund financing for the TA was reduced by 680,000 AUD); and o Exchange rate fluctuations between US$ and AUD.

4. An agreement was reached between the Task Team and the implementing agencies to eliminate some activities that could not be implemented in the remaining time of the TA period. Although all packages were important to the development of capacity in the government stakeholders, the Bank and the Implementing Agencies worked together to ensure that the packages and activities essential to the implementation of the Program remained. Table A5-1 shows a list of packages that were eliminated or reduced in scope as a result of this agreement.

Table A5-1 - TA Packages Revised Due to the Budget Cuts

Implementing Agency

Package Decision Original Allocation (US$)

Final Contract Amount (US$)

NCERWASS P03 - Review of Technical Norms and Standards

Scope reduced (pilot designs eliminated)

295,000 120,891

NTP SO P11b - Development of a Circular on Management, Operation and Protection of piped water schemes

Cancelled 80,000 -

NTP SO P12 – Environment and Social Management

Cancelled 575,000 -

NTP SO P14 – Fiduciary Management (including Procurement, Financial Management, Anti-corruption/ Governance)

Scope reduced (contract management training eliminated)

475,000 451,250

SAV P17 – Verification of Results (Period 3, 4 and 5)

Time frame reduced to Period 3 only

1,000,000 320,000

SAV P21 – Financial Auditor Cancelled 100,000 -

Outcomes

Relevance of TA Objectives 5. The development objective of the TA was highly relevant in providing support to the implementation of the PforR Program, as well as to the wider National Target Program for Rural Water Supply and Sanitation under which the PforR is implemented. For instance, one of the main justifications for the use of the Program for Results instrument was to emphasize sustainability and efficiency of water supply schemes. Also, this type of financing instrument requires accurate and efficient M&E. The TA objective of providing capacity building support to national and provincial implementing agencies was directly developed to support related capacities in these and other key Program management areas.

Achievement of TA Objectives 6. The majority of the TA’s six components were successful in achieving their stated goals; and the most significant achievements are listed below. Table A5-2 at the end of this annex shows the TA outcome indicators, targets, and results.


Page 55 of 61

Supporting Sustainable Water Supply Systems and Services: The TA provided essential support to water supply system operators, reinforcing skills for sound technical operations, financial management, accounting, asset management, and non-revenue water reduction.

Supporting Sustainable Sanitation Services: The TA strengthened the capacity of the Provinces and VIHEMA to better promote, manage, and monitor sustainable sanitation services. This directly supported the Program-wide achievement of DLI II and DLI IV. The new approaches introduced through the TA have been directly included in Provincial Sanitation Action Plans (including the state-of-the-art approach for Behavior Change Communication), which improves the likelihood of sustained sanitation results in the future.

Building Capacity in the Verification of Program Results: The TA helped to significantly increase the technical capacity of the State Audit of Vietnam (SAV) for independent verification of Program results. SAV had not previously conducted such large-scale ‘physical’ verification (thousands of latrines and water connections). Over the course of implementation, SAV progressively increased its leadership of the entire verification process, clearly demonstrating strengthened capacity for field surveys, sampling, logistics, and verification protocols.

Other TA components had mixed outcome, with clear areas for improvement. 7. The TA also produced a number of specific outputs that are being taken forward into the implementation of the SupS PforR Program, and into the New Rural Development NTP; including: 1) the new more efficient monitoring and reporting system, 2) the beneficiary feedback mechanism which can be rolled out to other provinces, 3) the IT transparency Platform, 4) the suggested technical norms and standards for rural water supply schemes, 5) the training manuals for sustainable water supply operation and maintenance, and 6) the newly established fiduciary structures and software.

Efficiency 8. In spite of the significant delays in the procurement and implementation of the TA activities, the final outcomes and results achieved by the implementing partners have provided beneficial impact to the implementation of the PforR program. The procurement delays forced implementing partners and consultants hired through the TA process to perform efficiently in order to achieved the development objective of the TA. With the exception of the remaining capacity gaps, the main objectives of the TA were achieved with positive impacts on the capacity of the implementing agencies at the central and provincial levels.

Overall TA Outcome 9. Rating: Moderately Satisfactory

10. The overall outcome of the implementation of the TA was successful with clear areas for improvement to address. Outputs from the TA direclty contributed to the successful implementation of the larger PforR Program; and there are many outputs and results from the TA that can be taken forward by the implementing partners into the ongoing SupS PforR Program and the New Rural Development NTP. The moderately satisfactory assessment accounts for implementation delays.

Lessons Learned 11. Below is a summary of key lessons learned and recommendations identified from the implementation of the TA. More lessons learned can be found in the full TA ICR document.

A key lesson from the implementation of the TA is that synchronization of TA support with Program activities is essential. For that to happen, efficient procurement processes are crucial to avoid delays. Many of the remaining gaps under this TA would have been avoided had key procurement processes like P12, P14, P01 been sped up.

Identification of critical capacity areas and targeted support to address key gaps is essential to boost performance. In this case, the work conducted with provinces have proved to be the most efficient one. Packages


Page 56 of 61

7 and 8 positively influenced best practices in DoHs to monitor report sanitation outputs and also promote demand through BCC. The result was a more than optimal performance in sanitation DLIs of provinces.

As one of the key gaps identified in previous RWSS-NTP assessments is the sustainability of water schemes, much attention needs to be given by NCERWASS and the NTPSO to give continuity to the capacity building activities under this TA. The RWSE model (Join stock companies) are working well in many areas of the RRD, and this model could perhaps be further adopted. The Government wants to keep exploring options to promote the participation of the private sector in rural water provision, but for that to happen strong public control of the different PPP modalities needs to be developed.

Despite the support provided under the TA, the sustainability of the service provision still remains. Ensuring sufficient cost recovery levels, good degree of compliance with asset management plans, keeping NRW under control, and ensuring high quality-customer oriented service provision are some of the key aspects that NCERWASs, Provinces and operators need to keep improving the performance of the sector. IEC activities should be boosted to maximize the number of billable connections per scheme. Provinces should also limit special promotions of “free water” and start progressive billing in areas where demand is low.

There is room for innovations rural water supply sector in Vietnam one clear one identified in this TA are the ones introduced in M&E. Rapid and accurate information is necessary to take decisions. The TA assessment identified follow up actions to ensure the continued and sustainable running of the M&E system. These include: that MARD mandate the provinces to use the mobile-based data collection methodology; provide further training ahead of the 2016 reporting period; develop a coordination plan before the next data collection and reporting schedule; form a working group with a champion from one of the provinces to discuss how to operationalize the platform, and establish a support contract and SLA for the data collection period with a software development firm; in case there are technical issues.


Page 57 of 61

Table A5-2 - Outcome Indicators and Results for AusAID-Supported TA Grant

Indicator Baseline by 09/01/2012 Target by 06/30/2016 Achieved Value 06/30/2016

Strengthened Planning, Monitoring and Evaluation capacity

Low capacity to use the existing M&E system

The M&E system is properly used with no gaps in data inputs. New IT platform created and increasingly used by program stakeholders (increasing data and information shared, increasing feedback from users).

There is interest from the provinces to use the system due to its usefulness. Provincial capacity in using the system developed, but clear instruction from central level is needed for them to use the system in substitution of the paper-based system. So far, the system has been used by provinces on pilot basis.

Number of provinces using the new IT platform for M&E

None Eight provinces Eight Provinces, however the use was not universal: Six provinces used the mobile based system to collect water connection data on pilot basis. Five provinces used the mobile based system to collect household sanitation information on pilot basis Three provinces used the system for collecting data for institutional sanitation.

Number of provinces producing analysis from the (phone-based) M&E system that is integrated into the annual provincial plan

None Eight provinces Eight Provinces, however, the provinces only used the system on pilot basis.

Number of provinces with complete annual monitoring data sets62

None Eight provinces None. MARD is suggested to send official letter to provinces to use the system for generating report.

Status of the transparency IT platform No transparency IT platform exists

A transparency IT platform has been set up and is being operated by Government. Users are able to send anonymous feedback on water supply and sanitation services to the platform via their mobile phones. Users are able to access the transparency website to track plans and progress on water and sanitation

IT platform developed, and public can see the project progress and the project documents. The aggregated feedbacks provided through feedback system can be also seen in the platform.

Improved Sustainability of Water Supply Systems

Rural water supply schemes are often over-dimensioned and expensive to operate. O&M performance is often poor.

Lower capital cost per capita in rural water supply systems. O&M performance is optimal

Lower capital cost per capita in NEW rural water supply systems. O&M performance is still to be measured.

62 A complete data set is defined as data on all 8 NTP and all Program indicators from all communes in a province


Page 58 of 61


Water quality monitoring is weak.

The use of appropriate63 technical norms and standards for the design and construction of water supply schemes

Technical norms and standards are inappropriate/do not exist.

Appropriate technical norms and standards are being used for 75% of all new water supply schemes designed in 2016

Appropriate technical norms and standards are being used for only a few (~ 15%) of all new water supply schemes designed in 2016

The number of provinces with PCERWASS technical staff trained on best practices for the technical and management aspects of implementing rural water supply schemes in the eight provinces

None Eight provinces Eight provinces

The number of provinces who have adopted technical, operational and financial management manuals created specifically for rural water supply schemes in Vietnam


The number of staff trained in monitoring64 water quality65 at all piped water schemes constructed under the program

No staff trained At least one staff per scheme 105 staff from 8 PCERWASS’ and 40 water schemes

Increased capacity to promote rural sanitation sustainable services

Demand for sanitation is low in some areas. Number of sanitation options is reduced, and sometimes expensive for the poor and near-poor.

Range of cost-effective sanitation options widened. Number of applications for sanitation loans increased

Actual starting time is Jan 2015. Household behavior is reported to have been improved in terms of awareness of impacts of unhygienic latrines, the knowledge about the latrine options and costs. For the supply side however, there has been little progress on results. Although, health staff has been able to collaborate with masons and shop owners in providing latrine options, no sanitation convenient shop set up in the 8 provinces to provide latrine product to households.

The status of provincial sanitation action planning in the eight provinces

No provincial sanitation action plans exist

Eight provinces have up-to-date sanitation action plans

All eight provinces have sanitation action plans developed.

Number of provinces with Preventive Medicine staff trained on effective methods to create demand and stimulate supply for sanitation, tailored to the RRD area

None Eight provinces 432 people from 8 provinces were trained on the rural sanitation service delivery. The participants include: health sector at province, district, commune and village level, Masons and shop owners School teachers

63 Cost efficient, technically and environmentally sound, using realistic market costs and specific to rural water schemes in Vietnam. 64 Capacity is defined as the required human resources, water quality testing equipment, consumables and laboratory conditions 65 MOH QCVN 02: 2009/BYT (includes testing frequency and test parameters)


Page 59 of 61


278 (64%) of participants were female.

Number of provinces submitting bi-weekly sanitation monitoring reports to VIHEMA

None Eight provinces Biweekly reports submitted. Each report contains consolidated information of all the eight provinces in terms of progress and challenges.

Improved fiduciary, environmental and social management and planning capacity

Provincial Plans are not realistic. Audit reports not systematic and comprehensive enough for a PforR operation, % of competitive bidding processes is low, implementation of env. and soc. regulations are weak

Periodic internal and external audits of acceptable quality to the Bank. Number of competitive processes for RW supply contracts increases. Env & Soc reg are well implemented. Provincial Plans prepared including all relevant data are realistic.

Periodic internal and external audits of acceptable quality to the Bank. Number of competitive processes for RW supply contracts increases. Env & Soc reg are well implemented. Provincial Plans prepared including all relevant data.

The financial audit capacity at national and provincial level

Audit procedures that meet International standards are not being followed at national or provincial levels There are no trained internal auditors assigned to The Program at the national or provincial levels No risk based Internal Audit Strategic Plan exists for the full duration of The Program

An internal audit TOR and reporting mechanism that meets International standards66 has been adopted by MARD The MARD Inspectorate has a national audit coordinator assigned to The Program and trained on Program standard audit procedures Eight Provinces have one internal auditor each trained on Program standard audit procedures A risk based Internal Audit Strategic Plan exists for the full duration of The Program

An internal audit TOR and reporting mechanism that meets International standards67 has been adopted by MARD The MARD Inspectorate has a national audit coordinator assigned to The Program and trained on Program standard audit procedures Eight Provinces have one internal auditor each trained on Program standard audit procedures A risk based Internal Audit Strategic Plan exists for the full duration of The Program

The standard of accounting and financial reporting systems and practices used by implementing agencies at national and provincial level

Accounting and financial reporting systems and practices at national and provincial level do not meet International Public Sector Accounting Standards

At least two staff in each province and 2 staff at the national level are trained to record Program transactions, prepare Program financial statements, and prepare management reports for planning and budget monitoring that meet International Public Sector Accounting Standards

A total of 20 staff have been trained to record Program transactions, prepare Program financial statements, and prepare management reports for planning and budget monitoring that meet International Public Sector Accounting Standards

66 As defined by the Internal Auditing of Institute of Internal Auditors (IIA) 67 As defined by the Internal Auditing of Institute of Internal Auditors (IIA)


Page 60 of 61


The standard of procurement systems for filing, record keeping, reporting and monitoring used by implementing agencies at national and provincial level

Procurements systems do not meet The Program standards68 at national or provincial level

Procurement systems meet The Program standards at the national level and in the eight provinces

Procurement systems meet The Program standards at the national level and in the eight provinces

The number of staff from implementing agencies at national and provincial level that have been trained in the use of procurement systems, procurement oversight procedures and contract management procedures that meet The Program standards

None At least: 2 staff at national level 2 staff in each province

A total of 25 staff have been trained in the use of software of procurement, procurement filing, and the record keeping system at national and provincial levels. Yet, training was not conducted on contract management due to the cancellation of this activity in the contract.

The number of staff trained at implementing agencies at the national, provincial, district and commune levels to implement government environmental management practices, including government regulations and procedures for safeguards document preparation and implementation

None At least: 2 staff at national level 1 staff at provincial level in each province 5 staff at district level 10 staff at commune level

A total of 25 staff have been trained to implement government environment management practices including representatives from PCERWASSs and PHCs of the 8 provinces.

The capacity of local environmental authorities to monitor and supervise Environmental Assessment (EA) compliance.

No local environmental authority staff trained on monitoring and supervision for EA compliance No environmental performance indicators or compliance mechanism developed for local environmental authorities to enforce environmental safeguards implementation

At least: 15 staff trained Environmental performance indicators and compliance mechanism developed

A total of 25 staff have been trained to monitor environmental performance indicators and compliance including representatives from PCERWASSs and PHCs of the 8 provinces.

The number of staff at national and provincial levels trained on methods and techniques to improve social participation, information disclosure, working with ethnic minorities, and the valuation of land and asset losses

None At least: 2 staff at national level 2 staff at provincial level in each province

A total of 25 staff have been trained in social participation, information disclosure, working with ethnic minorities, and valuation of land and asset losses, including representatives from PCERWASSs and PHCs of the 8 provinces.

The number of provinces that have adopted operational guidelines on social participation,


68 As defined in Action 2 of the Program Action Plan


Page 61 of 61


information disclosure, working with ethnic minorities, and the valuation of land and asset losses by provincial implementing agencies

Capacity to verify Program for Results in place. PforR verified

SAV has never conducted a verification of a Program for Results

SAV submits verification reports that are acceptable to the Bank

SAV submits verification reports that are acceptable to the Bank

The capacity of SAV to carry out verification of The Program DLIs

SAV has no capacity to carry out verification of the Program DLIs

SAV is capable of replicating the same or similar verification models in future Program for Results

SAV is capable of replicating the same or similar verification models in future Program for Results

Improved transparency and information sharing in the RWSS sector

Transparency in the Rural Sanitation Sector is low. Exchange of information is limited among key stakeholders. There are no specific tools for information sharing.

Information is shared among stakeholders through the IT platform. Number of platform visitors increases on a yearly basis.

Information is shared among stakeholders through the IT platform (under progress)

Number of workshops and working sessions organized by the RWSSP to share the lessons learnt from The Program with the RWSSP members, and to analyze the implementation of best practices

No workshops held Bi-annual workshops held at the national level Bi-annual workshops held at the national level

INTERNATIONAL DEVELOPMENT ASSOCIATION Public … · 2020. 6. 2. · 06 12-Jun-2015 Moderately...

Documents

Transcript of INTERNATIONAL DEVELOPMENT ASSOCIATION Public … · 2020. 6. 2. · 06 12-Jun-2015 Moderately...