Development Project Planning 5

Monitoring and Evaluation

Indicators

This Session

Recap – about M&E and Indicators

M&E Planning

Some Indicators o choosing

o setting up

Monitoring

Performance Measurement

o of the Process

o against the sections of the Logical Framework

o routine

Inputs

Outputs

Outcomes (Results Based Management)

What Indicators Show

Inputs Shows what goes in to providing a service - the resources used, e.g.

• Amount spent on travel per week

Activity or

Output

Shows what a service has done or provided, e.g.

• Number of condoms distributed

Utilisation Utilisation Shows if a service is being used, e.g.

• Number of people attending a nutrition course

Coverage Coverage Shows what proportion of people / groups in need receive a service, e.g.

• Proportion of schools with an AIDS awareness club

Performance Performance shows how well something was done, e.g.

• Number of reported cases of STIs

Monitoring (Performance Measurement)

Indicators show success at different levelso Strategic

• Goals, Objectiveso Sustainability

• after project completeo Effectiveness

• on the objectiveso Efficiency

• inputs against outputs

Quality, Quantity,

Time

Monitoring Criteria

o Relevance to goal/purpose and in-country needs

o Efficiency in providing inputs and converting to outputs

o Effectiveness – has production of outputs achieved objectives?

o Impact – is purpose making anticipated level of contribution to high-level goals

o Sustainability – the benefits will continue, without future negative impact

Exercise 5 mins

“Look back to our proposal for a watsan project in Ratanakiri.”

Come up with one indicator for ‘pump maintenance’ AT EACH LEVEL o Activity (efficiency)

o Objectives (effectiveness)

o Strategic - Goals (impact)

o Sustainability – (continues after project)

Data Collection Flow in Projects

What should you monitor and how?

You should monitor information that will help you to track what you are doing and to measure the success of your work.

So you should review your objectives and indicators and decide what information needs to be collected to allow the indicators to be measured.

You should only collect information that you will use.

Evaluation

“episodic” – happens at certain times, not routine

Looks at the Impact (Impact Assessment)

Evaluation: Impact Assessment

Approacho Impact from beneficiaries’ point of viewo What do they think is significant?o To whom is it important?o Ex-ante and Ex-post

Criteriao Efficiency – relate inputs to outputso Effectiveness- extent to which achieved objectiveso Consistency- methods/approaches with objectiveso Impact – change to lives/environment

Audit

Mainly look at financial operations and statementso compliance with legal and contractual

obligationso external, imposed

Can have “Performance Audits”o efficiency and good managemento Data Quality audits

What is an Indicator ?

a variable …

that measures one aspect of a project

that is directly related to the program’s objectives.

Break

Please – back here ON TIME !

A good indicator should:

produce the same results when used repeatedly to measure the same condition or event;

measure only the condition or event it is intended to measure;

show changes in the state or condition over time;

have reasonable measurement costs; and

be defined in clear and unambiguous terms.

Good Indicator - is…

Valid - measures what it is intended to

Reliable - accurate, repeatable

Precise - measures intended condition only

Independent - gives a line of possible results

Timely - available when needed

Comparable - with other similar situations Varies - shows changes in the metric over time; Costed - reasonable measurement costs; and Defined - in clear and unambiguous terms.

Other Factors InIndicator Choice

Data availability

Resources

Program needs

Donor requirements

Indicators: An ExampleObjective: Polluted water put into the Bassac is reduced

Select the indicator: Concentration of heavy metals (Pb, Cd, Hg)

Define the targets:

• the quantity: Concentration of heavy metal compounds (Pb, Cd, Hg) is reduced by 75% from year 2008 levels …

• the quality: ... to meet the limits for irrigation water ...

• the target group: ... used by the farmers of Phan village, ...

• the place : ... in Buphon District, ...

• the time: ... 2 years after the project has started .

Reporting Times

Level of Indicator

Frequency of Reporting

Examples of Data Collection Methods

Input/Process

Continuously ●Health services statistics●Health facility surveys●Program monitoring

Output Quarterly, semi-annually, or annually

●Health services statistics●Health facility surveys●Program monitoring

Outcome 1 to 3 years ●Population-based surveys●Health facility surveys●Special studies

Impact 2 to 5 years ●Surveillance●Population-based surveys●Special studies

Means of Verification

M&E plans should include multiple data sources. they should include indicators that use data from each sector, and include data from the program facilities, population measures, and special surveys.

The M&E plan should clearly state the sources for collecting data and how often the data should be collected.

Means of Verification

Data source must be:o Available

• time• cost

o Accurate• Showing what you want to show• Show what is really there

Exercise

Look at our watsan logframe

Fill in the MoV column

Data Systems

Tools:

o Progress reports

o Team meetings, team briefing reports

o HIS records

Evaluation Plan

A part of the project documents

Use to plan staff needs and budget

Contents of a Typical Evaluation Plan (1)

Brief project description

Objectives of the project

Objectives of the evaluation

Contents of a Typical Evaluation Plan (2)

Evaluation methodology – type of evaluation, indicators for each objective, data collection methods, sample size, methods of sampling / selecting participants

Resources needed, timetable, and budget – material, human, financial, transportation and logistics

Planned use of results – for example, improve the project mid-course, plan future projects, guide decision-making

Thankyou GrKuN

Produced by Tony Hobbs

Health Unlimited,

Ratanakiri, Cambodia

www.healthunlimited.org

With the support of Australian Volunteers Internationalwww.australianvolunteers.com

© 2009 HU. Use with Acknowledgement