from country to country and over time,they are summarized in Table I.

The selection and quantification orqualification of indicators relating tothe factors listed above depends onsubjective judgement, as most factorsare difficult to quantify, and somecarry more weight than others, depend-ing on the sector concerned. Anapproach based on these factors wouldbe difficult and expensive to imple-ment, and would be of limited use oncethe project was completed. As re-sources are usually limited for activi-ties which may appear to have noimmediate benefit, a simpler andcheaper form of measurement is neces-sary, and needs to be established andbuilt into recurrent costs as early aspossible.

Functioning as an indicatorWater supplies are meant to functionreliably throughout the period forwhich they are designed, and theymust do so to provide the impactenvisaged. Short-term impacts areoften used as indicators of the prob-ability that the supply will be sustain-able. But impact is notoriously difficultto calculate, and usually involvesmeasuring such aspects as the degreeto which the supply is used, thebenefits in time saved and its use, thechanges in patterns of health, etc., allof which are labour intensive to assess,prone to bias during measurement, andrequire large sample populations to bestatistically significant.

On the other hand, whether water isactually available from a supply in therequired quantities and quality on areliable basis is relatively easy tomeasure. This information can beprovided by people with little training,and the channels of communicationdeveloped to secure this informationcan be used for other purposes, suchas the emergency reporting of break-down, or the maintenance of stores ofsuitable spare parts.

Measuring sustainabilityIt requires a great deal of effort to keepwells and pumps in working order, andthis effort will not be made andsupplies kept functioning for years onend if the supplies are not used and arenot regarded by users as having apositive impact on their lives. (Thereverse argument, as is argued in theMinimum Evalution Procedures? is

regularly repeated to define trends overtime. The identification of trends en-ables donors to see when there is aneed for new interventions, such asrehabilitation or further communityeducation, and indicates wherechanges to, for example, the designsof wells or the education materials maybe needed, before any further worksare undertaken.

In countries where different donorshave used a variety of technical andsocial designs, a simple indicator oftheir positive impact will help toidentify, at minimal expense, thosecombinations which offer the highestprobability of success.

What ensuressusta inability?Sustainability can only be proven withthe passage of time, so in the short termany evaluation must be based onindirect indicators which aim to showthat the vital prerequisites for sustain-ability have been established. Al-though the factors which influencesustainability are complex and vary

Measuring sustainability in the water sectorby Joanne Harnmeijer and Sally SuttonMost methods of monitoring new water supplies,where they exist at all, rely on sophisticatedindicators and, usually, on outsiders. Why notaccept that if a community is making the effortto keep the supply working, then it must be asuccess?

SUSTAINABILITY IS THE ultimategoal of development efforts, an objec-tive which implies that external agentsshould leave a community with some-thing that it finds appropriate to itspriorities, and that is useable, afford-able, and maintainable. If any develop-ment has not, on balance, a positiveimpact on a community, then it willnot be sustainable. A programme canbe considered to be sustainable when'it is able to deliver an appropriatelevel of benefits for an extended periodof time after major donor assistance isterminated'. 1 But although sustainabil-ity is certainly something to strive for,if the evaluation of its achievement istoo complicated, it will require re-sources that are simply no longeravailable once the donor has left.

The monitoring of sustainabilitycould both improve the effectivenessof government and community plan-ning for recurrent and capital needs,and provide channels for feedback tocommunities. It could be similar to theuse of mortality rates, which are usedas an overall indication of the healthof a population at anyone moment,but which are of greater value if

~It requires a great deal of effort to keep supplies in working order~ andfh-lseffort will not be made unless the supply is filling a genuine need.

28 WATERLINES VOL.12 NO.1 JULY 1993

The key factors affecting the proper functioning of water supplies are design,construction, and maintenance.

Table 1: Decisive factors that influence sustainability

o Recipient country policies and commitmento Institutional capacityo Finance() Technologyo Socio-cultural factorso Environmental and ecological factorso External factors (economic, natural disasters, etc.)() Project/programme design factors

perhaps even stronger: impact can onlybe expected when supplies are used,which in tum depends on functioning.)Water supply programmes should not,therefore, worry so much about meas-uring impact when those who use andbenefit from a supply are making theeffort to ensure that it remains inworking order.

Whether a pump works or a well hasenough water for a growing commu-nity is a product of all the factors inTable I. These are changing all thetime. They may be assessed once ortwice, but are far too complex for theirindividual effects to be continuouslymonitored. It is much easier to measuretheir combined effect on the supplyitself and, when necessary, to analysethe causes for the trends identified. Thearguments for using continued func-tioning as a measure of sustainabilityare summarized in Table 2.

Factors affectingfunctioningThe key factors affecting the properfunctioning of water supplies are de-sign, construction, and maintenance.A well that is poorly designed orconstructed may let in so much sand,for example, that the pump breaksdown frequently, making the supplytoo expensive to maintain. A pumpthat is badly designed or built withpoor materials will result in a similarsituation. If design and constructionare adequately addressed in the earlyphases, the single issue that needscontinuous care is maintenance duringoperation.

The causes of poor operation andmaintenance performance are all toofamiliar. Table 3 summarizes a particu-lar example.

It is worth noting that when thecategories that determine sustainabilityas listed by OECD (Table I) areexamined (with the possible exceptionof socio-cultural factors), they all relateto maintenance and functioning, butdo not affect the levels of water use orimpact. Thus the continued function-ing of a water supply provides aquantitative measure of sustainabilitythat should replace other less quantifi-able and more subjective parameters.

Functioning over time?If it were all as simple as this,functioning would be measured moreoften in Water supply programmes.There are at least three reasons whythis is not so:

First, responsibility for maintenancedoes not, in many cases, rest with thedepartment responsible for construc-tion. Maintenance may be in the handsof the local council (as in parts ofZambia, for example) or the localpanchayat (as in India), so that aftercommissioning, a water supply is nolonger the concern of the organizationresponsible for its construction. Coun-cils and panchayats are part of theadministration. Their membership var-ies with the political tide, and watersupply is only one of their concerns,and one that seldom has any potentialto provide significant income.

Secondly, few programmes consis-tently manage to provide any orenough manpower for monitoring pur-poses. This is not only because of astructural lack of manpower, but alsobecause there is no tradition, structure,or budgeting facility that enables the

departments responsible for construc-tion to stay involved after the supplyhas been commissioned. Both mainte-nance and monitoring have a lowstatus, and if they are carried out bycentralized (district/panchayat) man-power the high transport costs will betoo much for either communities orlocal government.

Thirdly, there is often a long transi-tion time before monitoring and main-tenance is handed over to local institu-tions and users. Monitoring is thereforemost important in the final stage, whenno outside support is available; yet fewdonors look back on the results ofprojects and programmes once theirfinancial commitments have ended,

Methods of measurementSophisticated methods of communica-tion are necessary to update constantlythe degree to which supplies arefunctioning. This is seldom feasible inareas which can only afford watersupplies in the form of a bucket andwindlass, or a handpump on a well orborehole. It is therefore more commonto attempt a periodic collection ofinformation, usually during the driestseason, when water levels are at theirlowest, the demand for water fromreliable sources is at its highest, andaccessibility to sites is easiest. As thisis also the period of maximum activityin the construction and repair ofsupplies, there is seldom any sparecapacity for monitoring to be carriedout using regular employees.

u..wUz3:c~

WATERLINES VOL.12 NO.1 JULY 1993 29

A vital supply of water will be kept in order by the community.

Table 2: Continued functioning as an indicator of sustainability

Functioning represents:o A direct link with the objectives of improved water supply (not operating, no

impact)o A straightforward translation of time-proven sustainability (life expectancy

prediction from trends)o A visible and easily measurable indicator that can be assessed without needing

highly literate personnelo An outcome of successful operation and maintenance. which in turn depends

on a range of complex factors that would otherwise be measured to predictsustainability (Table 1)

o An indication of a sustainable level of use and of positive impact (i.e. enoughusers, with sufficient willingness to cover recurrent costs)

Table 3: Causes of poor operation and maintenance (Kerala)3

o Lack of revenue• low number of private connections• poor collection of user fees• resistance against user fees

o Shortcomings in technical arrangements• inefficient repair arrangements• inadequate fault reporting• insufficient accountability towards users

o Shortcomings in village arrangements• lack of users' involvement• lack of regulation regarding proper distribution of water at village level

The alternatives are to employschool-leavers to collect the informa-tion on bicycles or on foot, or to askschools, health centres. village head-men or local councillors to collect theinformation and feed it back to thedistrict offices. The latter system maytake longer to establish, and may notbe universally successful initially, butif supported by community education

until some of the benefits becomeapparent, then it can become a valuableregular activity.

While there are many constraints,continuity in functioning is a suitableindicator for sustainability. For watersupply programmes that are still underimplementation, among the indicatorsof functioning, reliability must meritspecial attention. An indicator such as

'proportion of supplies out of order atanyone time' tells something about thetechnical performance of water sup-plies, as well as about the efficiencyof response of both users and mainte-nance teams.

Given the increased interest in sus-tainability, funding agencies must con-sider supporting ex-post evaluations,particularly in the water sector, whereone can actually see the proof of realimpact - continued functioning .•

AcknowledgementsThallks are due 10 Pieler SlIuflalld. AllnWalers-Bayer and Sluarl SuI/Oil for Iheir com-me/Us alld ideas.

ReferencesI. Organisation for Economic Co-operation and

Development. Suslainahilily in Del'elopmenlProgrammes: A compendium of el'aluatiollexperience. Selected Issues in Aid Evalu-ation-I. OECD. Paris, 1989.

2. World Health Organization, Minimum Emili'alion Procedure for waleI' supply andsallilalion projecls. ETS/83. I CDD/OPR/83.1. WHO. Geneva, 1983.

3. The example in Table 3 refers to a Dutch-funded water supply project in Kerala, India(personal communication. M. de Graaf). Thelist would look slightly different for otherwater supply projects.

Joanne Harnmeijer is a community healthconsultant working for ETC, Consultants forDevelopment, Kastanjelaan 5, PO Box 64, 3830AB Leusden, The Netherlands; Sally Sulton is ahydrogeologist specializing in planningcommunity-based water supplies at SWLConsultants, 10 Thomey Hedge Road, Chiswick,London W4 580, UK.

30 WATERLINES VOL.12 NO.1 JULY 1993