1

INDEPENDENT EVALUATIONWSLIC-2 AND PAMSIMAS

PROGRAMS

Presented to the DSF

Jakarta, Juli 2007

2

Background of Evaluation Study• Government of Indonesia (GoI) plan to carry out Poverty Alleviation

Program through Program Nasional Pemberdayaan Masyarakat (PNPM)

• Bappenas wants to know the effectiveness of existing community-based poverty reduction projects

• The experiences and lessons-learned from planning up to the continuity of these projects will be used as input for implementing PNPM Program

Evaluated Programs:

Water Supply and Sanitation for Low Income Communities Project (WSLIC2), implemented by the Ministry of Health.

PAMSIMAS (still in design phase) implemented by Public Works Ministry

3

These Programs are chosen because:

a) They are all community development related;

b) Target primarily the sub district/kecamatan and village levels,

c) In term of government budget, they are some of the largest programs.

Evaluation Objectivesa) Asses which ones are effective at meeting their stated objectives,

b) Learn of the ways to improve the programs and draw lessons and best practices for other community development, poverty reduction programs,

c) Provide the GOI with useful and timely information to manage and guide public resources and interventions.

4

METHODOLOGY Scheme of Independent Evaluation of Water Supply and

Sanitation Programs

5

Evaluated Location

No Province District Sub-District Village

1. East Java Lamongan Maduran Pangean

Sugio Sidobogem

2. West Nusa Tenggara

LombokTimur

Sikur Kembang Kuning

Aikmel Aikmel Utara

3. West Sumatera PesisirSelatan

Linggo Sari Baganti,Kanagarian Punggasan

Lagan GadangMudik

Sutera, KanagarianSurantih

Rawang

6

Scope of Study

There are 7 aspects of analysis:Result Achievements Site Selection/Poverty TargetingCapacity BuildingFunds Flow and Cost Effectiveness Program SustainabilityCommunity OrganizationProgram Satisfaction

7

Program Description: WSLIC-21. Project name : Water Supply and Sanitation for Low

Income Communities-2 (WSLIC-2)2. Implementer : Ministry of Health3. Location : 8 provinces, 36 districts, 2.461 villages4. Project duration : 16 November 2000-20 June 20095. Project budget : $US 106.700.000,00 (total)6. Objectives : To improve the health status, productivity

and quality of life of poor communities in under-served rural villages by improving health behavior, health services of the communities related to water borne diseases, providing safe, adequate cost effective and easily accessible water supply and sanitation services and developing sustainability and effectiveness of services through community participation.

7. Components : Water supply, sanitation and Improvement of people’s styles of living cleanly and healthily

8

Program Description: PAMSIMAS1. Project name : Penyediaan Air Minum dan Sanitasi Berbasis

Masyarakat (PAMSIMAS)2. Implementer : Public Work Department3. Location : 15 provinces, 110 district, 5.000 villages and 1.000

replication villages4. Project duration : September 2006 – June 20135. Project budget : $US 375.000.000,00 (total)6. Objectives : To increase the accesses to water and

sanitation services for low income community in rural and pre-urban; To increase the quality of health and hygienic living behavior by developing/providing water and sanitation services based on continuing community development and adaptable by community. This Program will be a model for replication, scaling up and mainstreaming in others area, in order to reach MDG goals.

7. Components : Water supply, sanitation and Improvement of people’s styles of living cleanly and healthily

9

Selection of Programs Location/ Poverty Targeting (1)

WSLIC-2:Percent of poor population (score = 30 points)Scope of clean water services (30)Scope of sanitation services (10)Village population (10)Incidence of diarrhea (20)Other criteria: water resource potential, no other similar project existed, community’s ability to contribute

10

Selection of Programs Location/ Poverty Targeting (2)

PAMSIMAS:Community contribution (20% of budget: 16% in kind, 4% in cash)Community readiness to finance the O&M expenses and to pay retributionHigh poverty index Limited access to water and sanitation infrastructureHigh incidence of diarrheaNo other similar projects for the past 2 years

11

Fund FlowFund Liquefaction

Application by TKM

Termin III (25%)Phisic Report Progress 75%

Termin II (50%)Balance account

<10%

Termin I (25%)SPPB

PMC Verification

SPP-LS

CashArea

TKM's Bank Account

TKM

KPPN

1. SPM/2. SP2D

Yes

No

Yes

No

Keterangan:SPPB: Surat Perjanjian Pemberian BantuanSPPLS:Surat Perintah Pembayaran LangsungTKM : Tim Kerja MasyarakatKPPN: Kantor Pelayanan Perbendaharaan NegaraSPM: SuratPerintah MembayarSP2D: Surat Perintah Pencairan DanaPMC: Process Monitoring Consultan

12

Community OrganizationCommunity organizations are formed to perform operation & maintenance of clean water facilitiesThere are two alternative types of community/user organization:

Water User Group:This organization is used when the type of water infrastructure built is shallow well and hand pump well.

Water User Organization: This organization is used when the type of water infrastructure built is piping network that supply water for many people.

13

Community and Local Institutions Capacity Building

Community and local institutional capacity building activities are done through trainings and community empowerment. Training programs assist communities to organize effectively and to obtain technical capacity to plan, manage and control water, sanitation and community health programs.Types of training provided are:

Demand Responsive Approach/DRA and in-situ training;On the job training for local labor;On the job training for village midwife;Financial Administration Training;Management, operational and maintenance of water and sanitation

14

Program Sustainability

Sustainability

Sustainability Factors in Water and Sanitation Programs

15

Achievement of Results (1)WSLIC-2 West Region: Prop. Sumbar Central Region: Jatim East Region: Prop. NTB

Kab. Pesisir Selatan Kab. Lamongan Kab. Lombok Timur

Ds. LGM Ds. Rawang Ds. Sidobogem Ds. Pangean Ds. Kembang Kuning

Ds. Aikmel Utara

Location Condition

Plain Area10 m dpl

Coastal Area5-20 dpl

Plain Area40 m dpl

Plain Area6 m dpl

Mountain800-830 m dpl

Mountain3.726 m dpl

Prosperity Composition

Rich : 10%Middle: 30%Poor : 60%

Rich : 7,5%Middle: 14,6%Poor : 77,9%

Rich : 10,7%Middle:39,7%Poor :49,6%

Rich :11,0%Middle:46,%Poor:43,%

Rich : 6%Middle : 17%Poor : 77%

Rich : 6%Middle: 23%Poor : 71%

Built 2002 2005 2004 2005 2002 2002

Water Supply System

Off sitePiping

On siteDug Well

Off sitePiping

Off sitePiping

Off sitePiping

Off sitePiping

Source of Water River Shallow Ground Water

Deep Well River Spring Spring

Treatment None None None WTP None None

M/E None None Pump Pump None None

Continuity Benefit

Fee

Continue, developContinue

Continue,

None

Continue, developContinue

Continue, developContinue

Continue,

Not smooth

Continue

Continue

Latrine built by WSLIC-2

Public latrine (JamUm): 3 unit

Family Latrine: 100 unit

Family Latrine (JAGA) : 51 unit

Family Latrine : 109 unit

JAGA: 15 unitSchool latrine:3JamUm: 1 unit

JAGA: 13 unitJamlah: 2 unitJamUm: 1 unit

16

Achievement of Results (2)

Villages

Coverage of Clean Water Service Coverage of Sanitation Services

BeforeWSLIC-2 (%)

After WSLIC-2 (%)

Enhancement (%)

Before WSLIC-2 (%)

After WSLIC-2 (%)

Enhancement (%)

Pangean 25,24 100,00 296,20 25,85 100,00 286,85

Sidobogem 21,00 100,00 376,19 37,00 87,60 136,76

KembangKuning

20,50 94,75 362,20 15,50 17,94 15,74

Aikmel Utara 10,38 72,19 595,47 12,47 13,10 5,05

Lagan AgamMudik

33,47 82,00 145,00 19,83 76,00 283,26

Rawang 35,56 93,00 161,53 21,83 44,40 103,39

17

Achievement of Results (3)WSLIC-2 has been able to achieve its stated objectives:

Clean water & sanitation facilities available at the villagesIncidence of waterborne diseases declined (based on interview with stakeholder in villages such as community, Kades, Midwife and teachers, no impact evaluation has been conducted yet)Community’s health & sanitation behavior improvedCommunity’s access to clean water & sanitation improved

18

Finding: Poverty Targeting & Location Selection

Overall, the villages selected for WSLIC-2 programsare the right targets, because:

The majority of population are poor

High prevalence of waterborne diseases

There are clean water resource potentials that canbe developed

There are participation from village community

19

Finding: Fund Flow and Cost Effectiveness

Based on the achievements so far, the costs to build the facilitiesand trainings are quite reasonable; budget plan and fund distributionwere also quite transparent.However, fund ceiling of Rp.200.000.000,00 for watsaninfrastructure and training in each village wasn’t effective enoughbecause it couldn’t cover the targeted 80% coverage service area. Ithappened especially in piping and pumping system for supplyingwater.Unit price used in community action plan (CAP) was based on localmarket price and local labor price. But, some CAP didn’t take intoaccount the inflation.Funds are disbursed in 3 terms after being verified by Process Monitoring Consultant (PMC). Sometimes, the verification process impeded the funds disbursement.

20

Finding: Community Organization & Capacity Building

Water user organization has already functioned in each village and couldperform operation and maintenance of the watsan infrastructure. Someof them can even increase their service coverage financed through theirmonthly retribution (except in Rawang village).For capacity building, trainings have been held in each village with topicssuch as:

Financial management for Watsan Infrastructure,Operational and technical trainings for clean water preservation,Techniques of health counseling,Healthy lifestyle for the local people and teachers,Training for “little” doctors (in Rawang Village)Training on SODIS (in East Lombok) ,

WSLIC-2 program trainings are good enough, but to get watsaninfrastructures sustainability, trainings for organization management andO/M of watsan infrastructure post-construction are still needed. Theseneeds are already answered by PAMSIMAS concept.

21

Finding: Program Satisfaction

• All level of stakeholders are satisfied with WSLIC-2 program in their villages:

• Easy access to get clean water

• Community’s health & sanitation behavior improved

• Time efficiency to get water

• Sense of belonging of communities to the watsan infrastructure

• There were complaint mechanisms in WSLIC-2 Program and PAMSIMAS but not all communities knew about that. Problem-solving in WSLIC2 was handled by TKM through communication with village facilitator. So far, there are few complaints by communities about WSLIC-2 program because they (men and women; rich and poor) knew this program since beginning and were involved in all stages of the program (from planning, implementing, to maintaining).

22

Finding: Program SustainabilityOverall, the villages visited in the evaluation process havereached the condition of sustainable clean water and sanitationprogram based on following findings:

Watsan facilities built still function normally and satisfybeneficiaries.Watsan services cover >80% of program target communitiesCommunities have found ways and practical actions toensure the continuity of WSLIC-2 program (for example, inKab Lamongan where water user organizations formed alegal association to develop their professionalism inmanaging village watsan infrastructure and to build networkneeded in maintaining the facilities, eg. With materialsdistributor).

Sense of belonging is the main factor behind sustainability

23

Technical Sustainability Institutional Sustainability

Social Sustainability Financial Sustainability Environmental Sustainability

24

CONCLUSIONS (1)

1. Infrastructure designs in WSLIC-2 are quite good because they weredesigned by technical consultant that monitored by communities (TKM).Implementations design are also good because TKM is involved inconstructing, monitoring and supervising.

2. Overall, WSLIC2 programs have achieved their planned objectives.This success is felt by all layers of stakeholders ranging from projectbeneficiaries, the infrastructure management, TKK members, NGOs,CF, DPMU and PPMU

3. The villages selected for WSLIC-2 programs are the right targetaccording to the location selection criteria, which include povertytargeting

4. Funds for WSLIC-2 program has been used properly according to fundallocation.

25

CONCLUSION (2) 4. There is water user group/organization in each village.

Good example is in Lamongan district where the water usergroups from all villages in that district formed a water usergroups association that has legal status.

5. In terms of community empowerment, the WSLIC-2program approach has encouraged community to involve inplanning, constructing, and operating water infrastructure,and also contributing in funding.

6. In terms of capacity building, the trainings provided havebeen able to improve the skills and knowledge of TKM andcommunity about water and sanitation infrastructuremanagement and healthy life behavior.

26

Lessons-learned 1. Management: Community Based Organization (CBO)

approach in WSLIC-2 program need longer implementationtime because it emphasized more on process

2. Project Administration: Project Financial Administration at thevillage level seems too complicated for the community; itneeds to be simplified

3. Water and sanitation infrastructure operational andmaintenance: Need to make practical guidelines forcommunity level on topics such as maintenance of waterinfrastructure in each house.

4. Increasing the scope of water and sanitation services: Somevillages have enhanced their water services out of theiradministration boundary because of excessive water resourceand supporting topography. There is cooperation betweenvillages without any written document. However, system needsto be made to enable such practice in other situation.

27

Lessons-learned (2)5. Environmental conservation and preservation: Need to protect the

sustainability of water resource by cultivating the right plants thatcan store water.

6. Organization of clean water management: Existing organizationsthat manage clean water facilities have not got formal and legalstatus. Legal status for organization is important to strengthen itsposition in collecting and managing the retribution from themembers or water customers.

7. Continuity of Program: In order to maintain the continuity of theprogram:

Establish association of water user organizations that hasfunction to support and help solve problem of its members.Select local working team members democratically based ontheir competency; highly competent candidate is an essentialpart to ensure the continuity of the program.Post-program trainings to strengthen water managementorganization is very important to ensure their sustainability

8. In order to support program continuity, selection of technology andinfrastructure material should take into account existing capacityand resources of the community.

28

Recommendations (1)A. Program with CBO approach should be

implemented in multiple stages: the first stage focusing in preparation, socialization, institution formation and design, and the next stage focusing in constructing, operating, maintenance and strengthening institutional.

B. Funds allocation for each village should be based on technologies and systems that will be used. For this, planning of fund allocation should be made at the district level, and should be based on the results of MPA PHAST.

29

Recommendations (2)C. In order to simplify the operation and

maintenance of clean water facilities after construction, practical operational and maintenance guidelines for community need to be made for all types of existing technology.

D. In order to determine the impact of WSLIC-2 program on local people’s attitudes of living a clean and healthy life and on the improvement of their health, special evaluation on healthy and clean life behavior (PHBS) needs to be conducted.

30