June 2010 Japan International Cooperation Agency (JICA) Capacity Assessment Approach by JICA to...

June 2010

Japan International Cooperation Agency (JICA)

Capacity Assessment Approach by JICA

to Urban Water Supply Sector and Water Supply Utilities in Developing Countries

Contents

2

1. What is capacity assessment (CA)?

2. Capacity Assessment in urban water supply Major challenges in urban water supply in developing

countries Points to be noted Efforts of other donors etc. Approach by JICA

3. Capacity assessment tool of JICA Overview Basic Tool 【 1 】： List of Performance Indicators Basic Tool 【 2 】： Sector Checklist Basic Tool 【 4 】： Utility Basic Checklist Basic Tool 【 5 】： Utility Detailed Checklist Additional Tools: Environmental Scan / Capacity

Vulnerability Analysis Usage example of Basic Tool [4]

4. Summary

Capacity Assessment is The process of broadly assessing both the current

state of the developing countries’ capabilities for handling issues (capacity)

at multiple levels—including the individual, organizational, and societal level—

and the extent to which development process has brought about positive changes (Capacity Development: CD),

and then sharing the results from this with concerned parties in order to formulate CD strategies.

3

Definition of Capacity Assessment by JICA

１．

Wh

at is

Cap

aci

ty A

sse

ssm

ent

(CA

)?

1 ． What is Capacity Assessment (CA)?

4

The purpose of Capacity Assessment (CA) is Understanding capacity and environment Identifying needs Determining targets to achieve Identifying entry points of cooperation by donors Examining approach and scope of cooperation by

donors Enhancing the awareness of development

challenges and proactive attitude of relevant people of developing countries themselves

Why Capacity Assessment is necessary?１

．W

hat

is C

apa

city

Ass

ess

men

t (C

A)?

5

Capacity Development (CD)= The process of improving the developing countries’ capabilities for handling issues as an integrated whole at multiple levels—including the individual, organizational, and societal level

Key point is “comprehensive” and “endogenous” process

1. Support the enhancement of the developing countries to handle issues In order to achieve it,2. Perceive the capacity from a comprehensive view point3. Provide indirect support for the endogenous CD process

Capacity= The developing countries’ capabilities for handling issues= Collection of various elements including the institutional, policy and social system

Capacity development in terms of JICA１

．W

hat

is C

apa

city

Ass

ess

men

t (C

A)?

6

Target of cooperation

Individual level of developing countries

Organizational/ social level of developing countries

Human resources

for cooperation

Human resources of developing countries

Japanese experts

Sharing

Culture, society, history, custom of developing countriesCulture,

society, history and custom of Japan

Uptake/

adaptation

1. Japanese experts transfer the knowledge and technology to human resources in the developing countries.

2. The human resources in the developing countries not only acquire the knowledge and technology but also share it among the entire organization.

3. In that case, mutual understanding between Japan and developing countries and adaptation process of technology and knowledge is required.

Philosophy of CD 　～ process ～

Communication

１．

Wh

at is

Cap

aci

ty A

sse

ssm

ent

(CA

)?

7

What is “Capacity”? <Comprehensive Process>

Individual level

Organization level

Societal level(Social environment, institutional/political environment )

１．

Wh

at is

Cap

aci

ty A

sse

ssm

ent

(CA

)?

Capacity= The developing countries’ capabilities for handling issues= Collection of various elements including the institutional, policy and social system

Category Definition Assessment Target

Tech-nical

Capacity

Particular technical capabilities such as knowledge, skills and the tacit knowledge accumulated within the organization

C/Ps’ knowledge and skills (Including other agencies ), knowledge within the organization, presence or absence of information sharing systems and their quality

Core Capacity

Management capability, will/attitude and leadership to handle issues proactively by utilizing technical capacity

Organization’s conduct and way of thinking(speed and efficiency of decision-making, degree of implementing , etc.), various organizational systems (Incentive system, personnel, management institutions, etc.)

Enabling Environ-

ment

Conditions that make it possible for organization targeted to utilize capabilities to produce results

Financial system Institutional environment, Human resources, Physical resources, Financial base, Social capital

Perfor-mance

Daily results generated by the entity of CD through enhancement of its capacity

Emergence of results through efforts by organization (project planning and implementation performance, recipients’ satisfaction, etc.)

ImpactProblem solution which is gradually realized through the accumulation of results.

Continuous enhancement of service coverage and quality, increase in satisfaction of beneficiary, and continuous reduction of related morbidity rate, etc. 8

Perspectives for Perceiving CapacityC

ap

acit

y

１．

Wh

at is

Cap

aci

ty A

sse

ssm

ent

(CA

)?

• Underdeveloped legal system• Poor regulation and supervision• Unestablished standard and

guideline• Difficulty in financing

9

Major challenges in urban water supply in developing countries

• Low coverage of water supply • Part-time water supply and water

failure• Low of water pressure• Lack of water quality management• Lack of capability for operation

and maintenance• Aging of facilities• High non-revenue water ratio• Low collection rate of water tariff• Dysfunction of or failure to install

water meters• Lack of water supply to the

poor 　 etc.

• Unplanned expansion of facilities• Weak governance of

organizations• Lack of development of human

resources• Lack of customer response• Insufficiency of management

information• Lack of fund to construct and

maintain facilities

2.Capacity Assessment in Urban Water Supply 2

.Cap

ac

ity

Ass

ess

men

t in

Urb

an

Wa

ter

Su

pp

ly

10

It is important to understand the capacity of the entire water supply sector which includes not only water supply utilities but also the agencies regulating water supply service.

Not only the analysis tool of individual project but also the analysis tool to examine the development scenario organically combining analysis of overall water supply sector, examination of cooperation strategy and input of multiple projects are required.

１

２

Points to Be Noted2

.Cap

ac

ity

Ass

ess

men

t in

Urb

an

Wa

ter

Su

pp

ly

11

Necessity to clearly specify and monitor the outcome and impact of cooperation.

Practical tool to apply the CA in urban water supply in developing counties to actual cooperation activities for on-site practice.

４

５

Necessity of methodology incorporating the quantitative performance evaluation through performance indicators and Capacity Assessment including qualitative analysis 　

３

2.C

apa

cit

y A

sse

ssm

ent

in U

rba

n W

ate

r S

up

ply

Points to Be Noted

12

Therefore, it is necessary to provide support so that the water regulatory agencies in developing countries can appropriately determine the performance indicators to control water supply utilities according to their respective circumstances.

InternationalOrganization forStandardization

(1) Compliance of with International Standard 24500s for water supply service

Performance indicators and the items to be checked are selected in compliance with ISO 24510, 24511 and 24512 targeting the activities concerning potable water and sewage service.

ISO standard mentioned above encourages to select performance indicators appropriate to each country.

Efforts of other donors etc.2

.Cap

ac

ity

Ass

ess

men

t in

Urb

an

Wa

ter

Su

pp

ly

(2) Example of collection and accumulation of performance indicators data of water supply utilities in developing countries (IBNET)

IBNET: The International Benchmarking Network for Water and Sanitation Utilities

Survey and database on multiple countries by the World BankRegistry of indicators data of 2,000 water supply utilities in 85 countries including many developing countries79 kinds of performance indicators in total

13

Benchmarking (comparison between water supply utilities) is possible by identifying quantitative elements of performance.

Remaining challenge is qualitative understanding of capacity.

2.C

apa

cit

y A

sse

ssm

ent

in U

rba

n W

ate

r S

up

ply

Efforts of other donors etc.

(3) Example of collection and accumulation of performance indicators data of water supply utilities in developing countries (GTZ)

GTZ: Deutsche Gesellschaft für Technische Zusammenarbeit GmbH

GTZ provides support to continuous collection and evaluation of performance indicators data of water supply utilities in Kenya for the regulatory agencies.

Scoring based on evaluation standard

14

This enables benchmarking through the quantitative insight into performance （ Comparison among water supply utilities and evaluation of achievement level of goals)

Qualitative identification of capacity is a remaining issue.

2.C

apa

cit

y A

sse

ssm

ent

in U

rba

n W

ate

r S

up

ply

Efforts of other donors etc.

15

JICA addresses the comprehensive Capacity Assessment approach for urban water supply field as below:

(1) Water supply sector + water supply utilities

Approach by JICA2

.Cap

ac

ity

Ass

ess

men

t in

Urb

an

Wa

ter

Su

pp

ly

16

(2) Capacity + Performance + Impact

C ： Capacity

The water quality control ability of staff is enhanced.They can operate water quality analyzer.

P ： Performance

A Water supply utility can control water quality systematically and continuously.

I ： ImpactPeople can obtain safe water.

2.C

apa

cit

y A

sse

ssm

ent

in U

rba

n W

ate

r S

up

ply

Approach by JICA

17

(4) Expanded scope of assessment target

(3) Quantitative + Qualitative

Quantitative　Indicators

QualitativeQuestions

Mutually complementary

Analysis of core capacity Accessibility of the urban poor to water 　

(5) Dialogue tool with the stakeholders in a developing country

2.C

apa

cit

y A

sse

ssm

ent

in U

rba

n W

ate

r S

up

ply

Approach by JICA

18

CA for detailed analysis(including improvement status)

CA of sector agencies etc

Urban water supply sector in general

CA by comparison of water supply utilities within a country

Particular water supply utility

General information

Analysis by quantitative indicator

Analyze mainly by qualitative questions

Analysis mainly by qualitative questions

Which one is the problem?

Pair

Ind

ivid

ua

l an

aly

sis

In more details

Multiple analysis by both quantitative indicators and

qualitative questions

　 Analysis by qualitative questions followed by quantitative indicators

3 ． Capacity Assessment Tool of JICA

Overview

3．

Ca

pac

ity

As

ses

sm

en

t To

ol o

f J

ICA

CA for understanding outline

19

【 1】 List of Performance Indicators

【 3】 Utility General Form

【 5】 Utility Detailed Checklist

【 2】 Sector Checklist

【 4】 Utility Basic Checklist

【 1】 List of Performance Indicators

Environmental Scan

Environmental Scan / Capacity Vulnerability Analysis

Goal of Assess

ment

Goal A : Understanding

entire water supply sector

Goal B : Understanding outline of water

supply utility

Goal C : Understanding details of water

supply utility

Bas

ic

Too

l

Ad

di-

tio

nal

Too

l

Overview3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

(1) Assessment tool listing total 38 performance indicators, prioritized into three groups

(2) Comparison of multiple water supply utilities in the target country <Metric benchmarking> Identify general issues of water supply utilities within a country Cleary identify what kind of assistance is required to which

water supply utility Identify the model water supply utility Select indicators from IBNET except for some Calculate score from 7 indicators among 8 indicator in the first

priority group Detailed comparison is possible by incorporating 10 indicators

in the second priority groups

(3) Analysis of the improvement of particular water supply utility over time <Process benchmarking> Selectively use all the 38 performance indicators including

those in the third priority group

Basic Tool 【 1 】： List of Performance Indicators

20

3．

Ca

pac

ity

As

ses

sm

en

t To

ol o

f J

ICA

(1) Mainly assess jurisdiction of sector agencies (government agencies, regulatory bodies etc. ）

(2) Assessment can be conducted for each of 5 categories as below:1) Current status of water supply service etc. in the target

country （ including quantitative indicators)

2) Presence and utilization status of national policy, plans at national and regional levels, regulations, guidelines, etc.

3) Soundness of relationship between various water supply sector agencies and water supply utilities

4) Implementation status of training at national and regional level

5) Other stakeholders such as residents etc.

(3) Classify questions into two priority levels for quick and efficient assessment

First Priority: 35 questions 　　　 Second Priority: 49 questions

Basic Tool 【 2 】： Sector Checklist

21

3．

Ca

pac

ity

As

ses

sm

en

t To

ol o

f J

ICA

(1) Understand the utility’s basic information in advance through Basic Tool 【 3 】 Utility General Form

(2) Basic 26 questions to understand the outline of the capacity of water supply utility All of them are questions of the highest priority Facility 　 investment (FI)-related, technical CD-related, non-

technical CD-related and sector approach-related questions

(3) Evaluation and scoring on a 5-point scale Level 1 ： Very serious, Level 2 ： Serious, Level 3 ： Slightly

unsatisfactory, Level4 ： Target level of developing countries, Level 5 ： developed country’s level

Calculate the average score for the entire category, large and medium category respectively

Level assessment and scoring by on-site discussion. Easy-to-understand scoring method is effective as dialogue tool.

(4) Examine direction of a project

Basic Tool 【 4 】： Utility Basic Checklist

22

3．

Ca

pac

ity

As

ses

sm

en

t To

ol o

f J

ICA

23

(1) Check list of 193 items to assess the capacity of water supply utility in details. Use for planning, monitoring and evaluation of a project.

(2) Based on the result of the Basic Checklist, pick up from total 193 questions in second to fourth priority groups.

Select additional questions from the same category in the Detailed Check list as the category whose items are found to be serious by the assessment using the Basic Checklist in order to analyze more in detail.

Select additional questions which correspond to the selected form of assistance （ whether capacity development or facility investment)

(3) Easy customization such as addition of items to be checked and change of priority etc.

Basic Tool 【 5 】： Utility Detailed Checklist3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

24

(1) Additional Tools are used to highlight the core capacity of the organization and the relationship with the external environment which are hard to identify by Basic Tool.

(2) Promote awareness of stakeholders by participatory method

(3) ES: Visualize important elements of environment (external factors) for water supply utility systematically through mapping. It is possible to clarify demarcation of responsibility between sector agencies and water supply utility, identify the external factors of a project towards the water supply utility, and internalize such external factors.

(4) CVA: Discuss the strength and weakness concerning the management capability of water supply utility from various aspects using a matrix. It is possible to comprehend core capacity of water supply utility and examine effective method for improvement.

Additional Tools - Environmental Scan (ES)

- Capacity Vulnerability Analysis (CVA)

3．

Ca

pac

ity

As

ses

sm

en

t To

ol o

f J

ICA

25

Case studies of using Basic Tool 【 4 】 : Utility Basic Checklist for understanding outline of capacity of water supply utilities

Scoring through the discussion with stakeholders of the counterpart country 　⇒ 　 Capacity Assessment process itself through dialogue is a measure to strengthen capability

Usage Examples of Basic Tool 【 4】3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

26

Category Question

Aspects to be

Improved Mainly by

Facility Investment

(FI)

OverallQ1: Existence of Long or Mid-term PlanQ2: Continuity of Supply

ExpansionWater Supply

Service Coverage Q3: Overall Water Supply CoverageQ4: Water Supply Coverage over the Poor

Purification Plant Q5: Surplus Capacity of Purification Plant

Rehabilitation &

Replacement

Conditions of Facilities

Q6: Conditions of Civil StructuresQ7: Conditions of Trans. & Distrib. MainsQ8: Conditions of Service ConnectionsQ9: Conditions of M&E Equipment

Aspects to be

Improved Mainly by Capacity

Development (CD)

Technical Aspects

Overall Q10: O&M of the Facilities

Distribution Network

Management

Q11: Drawings of Pipe FacilitiesQ12: Zoning of Distribution NetworkQ13: Water Pressure at Supply Points

NRW ReductionQ14: NRW RatioQ15: Installation of Customer MetersQ16: Installation of Bulk Meters

Water Quality Control

Q17: Water Quality Tests at PurificationQ18: Drinkability of Tap Water

Non-technical Aspects

Financial Improvement

Q19: Cost Recovery LevelQ20: Collection Ratio

Organizational Development

Q21: Rules for Human Affairs & IncentivesQ22: Implementation of Training

Public RelationsQ23: Public Complaint HandlingQ24: Public Awareness Enhancement

Aspects to be Improved Mainly by Sector Approach

Q25: Laws and Regulations on WaterworksQ26: Sewerage Coverage

Questionnaire Items of Utility Basic Checklist3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 261

2

3

4

5

Over-all

Expan-sion

Rehabili-tation &

Replacement

Technical Aspects


Sector

Approach

3.0 2.7 4.9 4.0 3.0Improvement by

Facility InvestmentImprovement by

Capacity Development3.5 3.5

Overall Level of the Water Utility

3.5 1.5

27

Q1: Long / Mid-term Plan

Q2: Continuity of Supply

Q3: Supply Coverage

Q4: Supply over the Poor

Q5: Surplus Capacity

Q6: Civil Structures

Q7: Trans. / Distrib. Mains

Q8: Service Connections

Q9: M&E Equipment

Q10: Facilities O&M

Q11: Drawings of Pipe

Q12: Zoning

Q13: Supply Pressure

Q14: NRW Ratio

Q15: Customer Meters

Q16: Bulk Meters

Q17: Water Quality

Q18: Drinkability

Q19: Cost Recovery

Q20: Collection Ratio

Q21: Human Affairs

Q22: Training

Q23: Complaint Handling

Q24: Public Awareness

Q25: Laws & Regulations

Q26: Sewerage Coverage

TargetLevel

Ave-rage

Score

Case study in small-to-medium local city (1)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

【 Strength 】 The condition of existing facilities is good and the necessity

for repair or renewal is small.【Weakness 】 The capacity in non-technical categories such as finance,

management, human resources development and customer response is weak in general.

The problem is significant in mid-and-long term plan, coverage of water supply system and the access of the poor to water.

The development of the legal system is delayed.

It is necessary to prepare a mid-and-long term plan for expanding the coverage ratio of the water supply system including the service to the poor in its entire jurisdiction.

Water supply sector in charge of development of legal and institutional system requires support, too.

If the water supply act etc. is enacted, capacity development in non-technical area will become easier. 28

Features of case study in small-to-medium local city (1)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 261

2

3

4

5

Over-all

Expan-sion

Rehabili-tation &

Replacement

Technical Aspects


Sector

Approach

4.0 3.3 3.3 3.6 2.8Improvement by




3.4 3.0

29



Q3: Supply Coverage






Q9: M&E Equipment

Q10: Facilities O&M


Q12: Zoning


Q14: NRW Ratio


Q16: Bulk Meters

Q17: Water Quality

Q18: Drinkability

Q19: Cost Recovery


Q21: Human Affairs

Q22: Training





TargetLevel

Ave-rage

Score

Case study in small-to-medium local city (2)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

30

【Weakness】 Capacities of some items in non-technical categories such as

finance, personnel and human resource are low. The condition of the civil engineering structures is significantly

poor. The condition of mechanical and electric equipment is not so

good, either. They have a problem in controlling water distribution pressure.

Support for capacity development should be considered mainly for the items in levels １ and 2.

Facility investment is required in accordance with the mid-and-long term plan.

It is necessary to strengthen maintenance and management of mechanical and electric equipment, control of water distribution, and human resources development through capacity development via technical cooperation.

Features of Case study in small-to-medium local city (2)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 261

2

3

4

5

Over-all

Expan-sion

Rehabili-tation &

Replacement

Technical Aspects


Sector

Approach

4.5 3.7 4.5 4.4 4.8Improvement by




4.4 4.0

31



Q3: Supply Coverage






Q9: M&E Equipment

Q10: Facilities O&M


Q12: Zoning


Q14: NRW Ratio


Q16: Bulk Meters

Q17: Water Quality

Q18: Drinkability

Q19: Cost Recovery


Q21: Human Affairs

Q22: Training





TargetLevel

Ave-rage

Score

Case study of big city (1)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

32

【 Strength】 The capacity is high in general. The scores are high in non-technical categories.【Weakness】 The capacity in facilities （ in particular, category of

expansion ） is somewhat weak.

It is recommended to consider future investment concentrating on the facilities in the category of expansion.

Features of Case study of big city (1)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 261

2

3

4

5

Over-all

Expan-sion

Rehabili-tation &

Replacement

Technical Aspects


Sector

Approach

5.0 4.5 5.0 4.4 4.8Improvement by




4.7 2.5

33



Q3: Supply Coverage






Q9: M&E Equipment

Q10: Facilities O&M


Q12: Zoning


Q14: NRW Ratio


Q16: Bulk Meters

Q17: Water Quality

Q18: Drinkability

Q19: Cost Recovery


Q21: Human Affairs

Q22: Training





TargetLevel

Ave-rage

Score

Case study of big city (2)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

34

【 Strength】 The capacity is high in general. 【Weakness】 There remains a challenge in preparation of drawings of

distribution network, because GIS has not yet developed. The legal and institutional system is underdeveloped.

It is enough to focus on development of legal and institutional system from the view point of strengthening of sector agencies, and building GIS for better management of piping network.

Features of Case study of big city (2)3

．C

ap

acit

y A

ss

ess

me

nt

Too

l of

JIC

A

35

４． Summary

(1) Comprehensive Capacity Assessment covering both urban water supply sector and water supply utility

(2) Paying attention to every aspect of Capacity, Performance and Impact

(3) Analyzing capacity which is difficult to identify with only quantitative indicators by combining quantitative indicators and qualitative questions

(4) Expanding the target of analysis including the accessibility of the urban poor to water etc.

(5) Placing importance on dialogue with stakeholders in the developing countries. Assessment led by the target country respecting the ownership. Mutual understanding and sharing of perception among stakeholders.

(6) Practicing with the practical tool using the Excel sheet

Features of Efforts by JICA in Capacity Assessment in Urban Water Supply

4．

Su

mm

ary

36

(1) Improvement in methodology and assessment tool

(2) Possible expansion of methodology in urban water supply area (Diagnosis of function of water supply facilities, procurement conditions such as suppliers, project implementation capability, and so on)

(3) Coordination with other sectors (sewerage, sanitation, hygiene education, etc.)

(4) As for ensuring access to water for the urban poor, expand Capacity Assessment methodology to cover not only water supply utility but also beneficiary community

(5) Partnership with other donors

Future improvement and expansion4

．S

um

mar

y

37

Reference Materials"Capacity Development Handbook for JICA staff: For Improving the Effectiveness and 　Sustainability of JICA’s Assistance"http://gwweb.jica.go.jp/km/FSubject9999.nsf/3b8a2d403517ae4549256f2d002e1dcc/e7e6a65d3a5766b7492575e5002bdcbc?OpenDocument

"Capacity Development and JICA's Activities (February 2003)"http://www.jica.go.jp/english/publications/reports/study/capacity/200302/index.html

"Capacity Development: Technical Cooperation of JICA in the Health Sector (January 2003)"http://www.jica.go.jp/english/publications/reports/study/topical/cap/

"Supporting Capacity Development in Solid Waste Management in Developing Countries -Towards Improving Solid Waste Management Capacity of Entire Societies- (July 2005)"http://www.jica.go.jp/english/publications/reports/study/topical/waste/index.html

ISO 24512:2007 December 2007"Activities relating to drinking water and wastewater services - Guidelines for the management of drinking water utilities and for the assessment of drinking water services"http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=37248

IBNET ： The International Benchmarking Network for Water and Sanitation Utilitieshttp://www.ib-net.org/

4．

Su

mm

ary

June 2010 Japan International Cooperation Agency (JICA) Capacity Assessment Approach by JICA to...

Documents

Transcript of June 2010 Japan International Cooperation Agency (JICA) Capacity Assessment Approach by JICA to...