Indicators

Dr Murali Krishna Public Health Foundation of India

Lesson plan• Objective: At the end of session every

participant should be able to – Define indicators– say the importance of indicators in M & E – Able to choose appropriate indicators

• Time: 1 hour 30 minutes• Mode of lecture: Interactive session • Media: Power point presentation• Evaluation: in group work

Introduction

• Essential part of a monitoring and evaluation system as they are what you measure and/or monitor.

• Through the indicators you can ask and answer questions such as: – Who? – How many? – How often? – How much?

Definition

• a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement or to reflect the changes connected with an intervention.

• Indicators are compared over time in order to assess change.

E.g How did your class went on?

Characteristics of indicators

• The desired properties of indicators, also known as variables, will depend on the approach adopted and the nature of the programme or project being evaluated. All indicators have specific characteristics: – Numeric = the values are numbers– Nominal = the values have names (e.g. male and female)– Continuous = the values are infinite or very large– Ordinal/categorical = the values have a known order (e.g.

low to high)

Characteristics of good indicator

• Indicators will vary from one project to another, according to the work and its context, but in general they are often expected to be: – SMART (specific, measurable, attainable, relevant and time-bound)– SPICED (subjective, participatory, interpreted, cross-checked,

empowering and diverse)– CREAM ((Schiavo-Campo 1999, p. 85)

• Clear : Precise and unambiguous,• Relevant : Appropriate to the subject at hand, • Economic: Available at a reasonable cost, • Adequate : Provide a sufficient basis to assess performance• Monitorable: Amenable to independent validation

Practicality of indicators include Measurability: Is the indicator measurable? Is it sufficiently

sensitive to an improvement or deterioration in conditions? Ease and cost of collection: How easy is it to obtain the

information required? How costly will this be? Can the community participate? Are some relevant data already collected?

Credibility and validity: Are the indicators easy to understand, or will people end up arguing over what they mean? Do they measure something that is important to communities as well as implementing organizations?

Balance: Do the selected indicators provide a comprehensive view of the key issues?

Potential for influencing change: Will the evidence collected be useful for communities, implementers and decision-makers?

Common Indicator Metrics

• Counts– Number of providers trained– Number of condoms distributed

• Calculations: percentages, rates, ratios– % of facilities with trained provider– Maternal mortality ratio, Total fertility rate

• Index, composite measures– Quality index comprising the sum of scores on

six quality outcome indicators– DALY (Disability Adjusted Life Years)

• Thresholds– Presence, absence– Pre-determined level or standard

Types of indicators– processing of information: Elementary, derived

and compound indicators– comparability of information: Specific, generic and

core indicators– phases of completion of the programme:

Resource, output, result and impact indicators– evaluation criteria: Relevance, efficiency,

effectiveness and performance indicators– mode of quantification and use of the

information: Monitoring and evaluation indicators

Developing indicators

Step 1: Identify the problem situation you are trying to address. The following might be problems:

1. Economic situation (unemployment, low incomes etc) 2. Social situation (housing, health, education etc) 3. Cultural or religious situation (not using traditional

languages, low attendance at religious services etc) 4. Political or organizational situation (ineffective local

government, faction fighting etc)

Step 2: Develop a vision for how you would like the problem areas to be/look. This will give you impact indicators.

a. What will tell you that the vision has been achieved?b. What signs will you see that you can measure that will

“prove” that the vision has been achieved? c. For example, if your vision was that the people in your

community would be healthy, then you can use health indicators to measure how well you are doing.

a. Has the infant mortality rate gone down? b. Do fewer women die during child-birth? c. Has the HIV/AIDS infection rate been reduced? If you can answer “yes” to these questions then progress is being made.

Developing indicators

Step 3: Develop a process vision for how you want things to be achieved. This will give you process indicators.

Example: • you want success to be achieved through community

efforts and participation, then your process vision might include things like – community health workers from the community trained

and offering a competent service used by all; – community organizes clean-up events on a regular basis,

and so on

Developing indicators

Step 4: Develop indicators for effectiveness. • For example:• if you believe that you can increase the

secondary school pass rate by upgrading teachers, then you need indicators that show you have been effective in upgrading the teachers

• e.g. evidence from a survey in the schools, compared with a baseline survey.

Developing indicators

Step 5: Develop indicators for your efficiency targets.

Here you can set indicators such as: • planned workshops are run within the stated

timeframe, • costs for workshops are kept to a maximum of US$ 2.50

per participant, • no more than 160 hours in total of staff time to be

spent on organizing a conference; • no complaints about conference organization etc.

Developing indicators

Common Pitfalls in Indicator Selection

• Indicators not linked to program activities• Poorly defined indicators• Indicators that do not currently exist and

cannot realistically be collected• Process indicators to measure outcomes &

impacts• Indicators that are not very sensitive to

change• Too many indicators