KYRGYZ REPUBLIC Programmatic Public Expenditure Review

Monitoring and Evaluation

The MOF/Donor Workshop on PPERThe MOF/Donor Workshop on PPER

Bishkek, September 26, 2005Bishkek, September 26, 2005

2

Why Monitoring & Evaluation In order to improve public’s satisfaction the In order to improve public’s satisfaction the

government needs to government needs to improve service deliveryimprove service delivery Improved service delivery needs Improved service delivery needs strategic strategic

planning of public interventionplanning of public intervention (it is understood (it is understood that expenditure execution is a must) that expenditure execution is a must)

Good planning is based on Good planning is based on feedback on service feedback on service provision through monitoring and evaluation provision through monitoring and evaluation of outcomesof outcomes to incorporate results into decision to incorporate results into decision making practicesmaking practices

3

What is Evaluation? Assessment of progress toward achievement of Assessment of progress toward achievement of

policy objectives using proxi indicators policy objectives using proxi indicators Start with a limited number Start with a limited number

of of key indicators key indicators (e.g. student (e.g. student achievement in math, condition achievement in math, condition

of local roads)of local roads) Methodology includes extensive data Methodology includes extensive data

collection, evaluations techniquescollection, evaluations techniques Can be selectively outsourced to consulting Can be selectively outsourced to consulting

firms, universitiesfirms, universities

4

PREPARATION

APPROVAL

EXECUTION

EVALUATION

Integration into Budget cycle

5

What is already done: Functional reviews undertaken in the key line Functional reviews undertaken in the key line

ministries with donor support ministries with donor support They consistently point to the need of strong They consistently point to the need of strong

policy units in the line ministriespolicy units in the line ministries Background information for the policy Background information for the policy

framework:framework:- - CDF, NPRS, working documents, etc. CDF, NPRS, working documents, etc.

- DFID study – general on the current system- DFID study – general on the current system- WB PPER CASE analysis of Education, Transport and Agriculture- WB PPER CASE analysis of Education, Transport and Agriculture- Multi-donor analysis in Health sector - Multi-donor analysis in Health sector OtherOther

Information databases initiated in a number of Information databases initiated in a number of line ministries e.g. MOE and MOT with donor line ministries e.g. MOE and MOT with donor supportsupport

6

Findings: PPER CASE M&E Study There exists good basis for monitoring quantitative and qualitative activities in There exists good basis for monitoring quantitative and qualitative activities in

the key sectors, which could become the basis for the effective operative workthe key sectors, which could become the basis for the effective operative work This information is not demanded, collected and aggregated at the higher This information is not demanded, collected and aggregated at the higher

level. Use of rich data is inefficient. level. Use of rich data is inefficient. No sectors examined have a system of indicators to evaluate outcomes/results No sectors examined have a system of indicators to evaluate outcomes/results

(e.g. student performance in math) (e.g. student performance in math) Multiple efforts on M&E systems do not receive adequate support from the Multiple efforts on M&E systems do not receive adequate support from the

Government. Government. Monitoring system for school conditions financed from the ADB project Monitoring system for school conditions financed from the ADB project

is under the threat of failure. is under the threat of failure. A unique road database on road conditions in three cities supported by the A unique road database on road conditions in three cities supported by the

WB is not used. WB is not used. The ministries do not have organizational function and personnel capacity The ministries do not have organizational function and personnel capacity

(e.g. high turnover of the administrative staff) and resources to perform (e.g. high turnover of the administrative staff) and resources to perform continuous monitoring and analytical work. continuous monitoring and analytical work.

Parallel flows of identical information have been revealed (departmental and Parallel flows of identical information have been revealed (departmental and statistical in ME). statistical in ME).

Many policy decisions are made on political grounds, rather than on the Many policy decisions are made on political grounds, rather than on the current needs. current needs.

7

Progress on M&E has been slow for a

number of reasons, e.g. M&E process is driven not by demand and understandingM&E process is driven not by demand and understandingbut by consistent request from the donors and upperbut by consistent request from the donors and upperlevels in the Government to produce immediate outcomeslevels in the Government to produce immediate outcomes

Lack of coherent and shared strategy on M&E and lack of Lack of coherent and shared strategy on M&E and lack of clear guidance form the Government clear guidance form the Government

Soviet legacy – central planningSoviet legacy – central planning Absence or weak policy unites in the line ministries Absence or weak policy unites in the line ministries M&E task is loaded as an additional responsibility on M&E task is loaded as an additional responsibility on

budget departments budget departments Lot of data on inputs but lack of adequate data/no Lot of data on inputs but lack of adequate data/no

collection of data on outcome indicators collection of data on outcome indicators

8

Pre-requisites for “Success”

Strategic leadership – not micro-management from the Strategic leadership – not micro-management from the governmentgovernment

Involvement of all stakeholders: President, Prime-minister, Involvement of all stakeholders: President, Prime-minister, Parliament, Government, Civil SocietyParliament, Government, Civil Society

Cooperation from the part of the MOF and the line Cooperation from the part of the MOF and the line ministriesministries

Clear and simple policy framework: streamline on-going Clear and simple policy framework: streamline on-going fragmented actions into one policy framework owned by fragmented actions into one policy framework owned by all involved parties with clear implementation all involved parties with clear implementation arrangements arrangements

Develop selected baseline indicators for sector strategies Develop selected baseline indicators for sector strategies Recognition of gradual change (long history of attempts to Recognition of gradual change (long history of attempts to

introduce results orientation in other more advanced introduce results orientation in other more advanced countries (e.g. 1947-2003 in the US)countries (e.g. 1947-2003 in the US)

9

Next steps Develop strategy based on the existing analytical Develop strategy based on the existing analytical

backgroundbackground

Review and implement results of functional reviews Review and implement results of functional reviews i.e. establish/strengthen strategic planning units in the i.e. establish/strengthen strategic planning units in the MOF and the line ministries charged with the role of MOF and the line ministries charged with the role of M&E (e.g. Policy unit in the MOE under the Rural M&E (e.g. Policy unit in the MOE under the Rural Education Project, IDF)Education Project, IDF)

Provide training to the MOF budget analysts and line Provide training to the MOF budget analysts and line ministries for developing evaluation capacity (e.g. ministries for developing evaluation capacity (e.g. IDF)IDF)

Make an inventory of all donor activity in M&E area at Make an inventory of all donor activity in M&E area at central, sectoral level and at level of statistical office central, sectoral level and at level of statistical office

10

Governement’s Evaluation Road-Map

January

Budget 2006 NPRSIIMTBF 2006-2008

Prepare diagnostic of the current situation based on the background studies

Government to validate the work (broad discussion, including with donors) and finalize report

Set of options/suggestions presented to the Government

Government selects options, sets work plan for the committee: prepares roles and responsibilities, calendar, resources,

Jan-Feb Feb-March March-June

2006

March-June