14 Quality Control: Monitoring, Evaluation & Adjustment...

COMiT: Steering Project Implementation - Quality Control 151

14 Quality Control: Monitoring, Evaluation &Adjustment (MEA)

14.1 Definition and Functions of Monitoring, Evaluation andAdjustment

14.1.1 Introduction

The function of monitoring - evaluation - adjustment (MEA) in themanagement of implementation processes is concerned with a project'sinternal affairs. It is distinct from evaluation of a project by external agents.

As projects intervene in complex socio-economic situations, which are inthemselves dynamic, a project plan can never exactly foresee a future desiredsituation. Therefore, an instrument is required which continuously deliversstructured information concerning the actual track of the project. Plans arenever perfect. Project implementation, therefore, needs to be continuouslymonitored and adjusted to the dynamic reality.

Thus, MEA has to be oriented towards decision-making (and not reportoriented).

MEA has to provide information for all decision-makers involved in a project:for the management, and line personnel, co-operating organisations as wellas target groups. Information must not be monopolised, because it is a powerinstrument.

The System MEA is a means to support the continuous learning processes ofthe project team during project implementation. It needs to be integrated intothe organisational procedures of the project. The basic methodology appliedin MEA is to compare actual project status with what was planned.


14.1.2 What does the acronym MEA stand for?

M onitoring+

E valuation+

A djustment

CONTROLLING

M & E & A

M = observation + documentation

E = assessment of information

A = steering of the project

The three elements of project internal MEA can be described as follows:

• monitoring refers to the systematic observation and documentation ofinformation on the implementation of a project, based on the project plan;

• evaluation concerns the project internal assessment of information(which is collected through monitoring) regarding the extent to whichactual progress in implementation conforms to (or deviates from) theobjectives set in the project plan.

• adjustment means steering the project the project management (on thebasis of empirical facts ascertained in the monitoring and a verifiableevaluation process takes decisions, organises actions and allocatesresources, according to a system of re-planning).

These three elements are inseparably interlinked:

• the steering of a project must be based on the assessment of itsperformance and impacts (and not on outside pressures for action);

• assessments must be based on hard facts (and not on guess-work);

• but also! no observation must be made without processing it further (nodata graveyards),

• any form of monitoring and evaluation can not be an end in itself but mustbe directly translated into corrective actions or adjustments. (no report-driven monitoring)


14.1.3 The yard-stick

In order to observe, assess and steer in an objectives oriented way, clearlydefined objectives are required. These are to be specified in terms of what isto be achieved, how much, how well, where and when (refer to indicators andmilestones in PPM and Plan of operations in Project planning) MEA willcompare plans to actually achieved objectives and projected frame-conditions(assumptions). Logically consistent and sufficiently detailed project planning,documented (in a summarised version) in the form of a project planning matrix(PPM) and an operational plan, is the pre-condition for efficient MEA.

14.1.4 Additional functions of MEA

In addition to supporting the managerial task of steering a project the MEAsystem is also useful with respect to presenting and justifying theperformance and impact of the project to external agents:

• towards the co-operating or supervising agencies, and

• towards donor or commissioning agencies.

Secondly, the MEA system is the basis for reporting. The extent to whichobjectives set during Planning have been actually achieved must be the focusof reporting (and not only the deployment of resources and the carrying out ofactivities).

Thirdly, the MEA system becomes a reference system for externalevaluations. External evaluators require information as well as an insight into


how the information was acquired. They also need to know how the project issteered if they are to contribute constructively to the adjustment of the projectin the medium term.

14.1.5 Further clarification of the three elements:

monitoring = observation

evaluation = assessment

adjustment = steering

M What is to be monitored / observed?

The system has to deal with three different levels:

• Monitoring the performance of the project, includes observing, measuringand recording:

ο the deployment of project resources in comparison with what wasscheduled in the plan of operations / work plan and relatedpersonnel, input and budget plans as well as the accounting system

ο how timely, competently and efficiently the project activities havebeen executed, and what milestones have been achieved comparedto what was scheduled in the Plan of Operations

ο the outputs/results achieved in comparison to what is specified in theindicators indicators which established in the PPM.


• Monitoring the framework, includes observing, measuring and recording:

ο changes in the risks/conditionalities and/or assumptions inrelationship with specified indicators or minimum thresholds whichneed to be monitored/secured to ensure project success

ο unforeseen side effects of project interventions with special emphasison negative side effects,

• Monitoring the impact, of the project, by observing, measuring andrecording

ο the extent to which the various target groups are adopting/utilisingthe goods and services produced/delivered by the project comparedto what was estimated in the indicators

ο the change in behaviour/practices and capacities of the various targetgroups, resulting from their use of the goods and services offered bythe project

ο the benefits which have accrued to the target groups as a result ofthe changes stimulated by the project.

The implication of these definitions is that the data collection within a MEAsystem must only cover the areas outlined above. Not all data requirementswithin the project will be met by the MEA system. e.g. no base line data, noresearch results, no special sectoral surveys, etc.)

The "yard stick"

PPM

Plan Op

Goal+ assumptions

Purpose+ assumptions

Outputs+ assumptions

Activities+ assumptions

Impact

Results

Activities

Performance

Inputs Inputs

M & E & A

side effects

Unintended


E What is meant by evaluation / assessment ?

Evaluation or assessment means the analysis of observations andmeasurements made in the course of monitoring the impact, frame conditionsand performance, and drawing conclusions from these observations withrespect to the fulfilment of the project objectives and the project's policies ingeneral. This can be achieved by asking the following questions:

• what have we planned and what is the situation now? (comparison ofplanned versus actually achieved objectives activities and resourceutilisation); what are the current frame-conditions compared to thethresholds we established during the planning periods?

• why is the situation as it is? (analysis of causes);

• what can we learn from this? (conclusions);

• which alternatives do we have for proceeding further? (options);

• how do we assess the alternatives? (in the light of our policy criteriaand resource limitations)

• which suggestions/recommendations do we propose therefore?


A What is meant by adjustment / steering ?

Adjustment or steering follows assessments. Adjustment of implementationcan only concern manageable areas, i.e. resources and activities (which haveresults). Neither the project purpose, its goal, assumptions nor unintendedside-effects are directly controlled by the project management. But themanageable areas can be adjusted to better achieve objectives or suit thechanged conditions. Adjustments can consist of:

ο no changes in the project measures, if no or tolerable deviations fromthe project plan are observed;

ο changes in activities, if deviations from the Plan of Operations can becounteracted by adjusting resources and activities;

ο adaptation of the plan, if the project concept does not yield theexpected results and effects,

ο changes in the project strategy or termination of the project if purposeand goal turn out to be unachievable due to misconceptions andchanges in frame conditions or due to a dominance of unexpectednegative side effects.

The adjustment will result in planning the project with greater care in thoseareas which have become complicated, and in less detail in the more straight-forward areas. The MEA - system (and the information base it generates anduses) will have to be adjusted accordingly.

14.1.6 Participants in the MEA-system

It is neither only the project management which is concerned with MEA, noronly a specialised unit or person who / which has been put in charge. MEAconcerns all persons/units/agencies involved in or benefiting from the project:

• the project management / directorate (including staff-divisions)

• the entire line staff of a project

• the groups of beneficiaries

• the co-operating organisations and agencies.


Each group has different information requirements and decision-makingfunctions, which have to be taken care of when designing the MEA system.

Therefore:

• a meaningful participation of different parties concerned has to becatered for;

• only those information which are needed by one or the other partyhave to be collected;;

• the MEA system has to be an integral part of the normal work andrespective procedures in a project;

• the feedback of information into the decision-making process has tobe institutionalised within the project.


Specifically, one has to determine, with respect to the:

• project management:

ο which decisions does the management reserve for itself (e.g.policy matters, etc.)

ο which decisions are taken in co-operation with the linepersonnel?

ο which decisions are taken in co-operation or co-ordinated withother organisations / institutions?

• line personnel:

ο which results of monitoring and evaluation are immediatelyinfluencing the implementation of the project by the linepersonnel?

ο how should one co-ordinate monitoring and reporting functionswith other intervening agencies?

• target groups of envisaged beneficiaries:

ο to what extent are envisaged beneficiaries actively co-operatingwith the project?

ο how can the different groups bring in their views and feedback?

ο what information do they require, and how can they collect andprocess them?

• co-operating agencies:

ο about which details of the project and its performance must otherco-operating organisations and institutions be informed? (bewareof possible repercussions)

ο in which areas is the adjustment of the process the exclusiveresponsibility of the project and where do other agencies(including superior authorities) need to be involved?

14.1.7 MEA's - systems are not a "counter-intelligence agency"

MEA can be abused. If it is used to survey and supervise the performance ofthe project personnel instead of providing people with an opportunity to learnthrough their experiences; if it creates mistrust rather than openness; if notcontrolled by the project itself but by external agents, then MEA can do moreharm than good.


Therefore:

• the objectives and detailed designing any system of MEA must always bedetermined with sufficient participation of those concerned;

• self control must have priority over external controls;

• those responsible for the implementation process must also be involvedin its monitoring;

• feedback and the assessment of information must be institutionalised;

• decision-making procedures foradjustment must be participatory andclear;

• an emphasis should be placed onmutual help in devising correctiveactions required and not on looking for'the guilty';

• it is important to note that unrealisticplanning rather than individual mistakesor omissions is the most common causeof failures.

M


14.2 Design of a MEA-System

14.2.1 Overview of the important steps

In many cases a MEA-system is set-up and operated by specifying theplanning documents, e.g. by detailing indicators for the objectives andassumptions, and then observing whether the implementation processadheres to the plans. Implementation procedures, e.g. filing systems,internal project decision-making and reporting, may follow a distinctadministrative rationale which is not necessarily linked to the planningrationale. This, however, is not the approach advocated in project cyclemanagement. In order to ensure the quality of a project it is necessary tointegrate whatever was planned, i.e. the detailed indicators, into theprocedures and organisation of a project. Thus MEA is closely linked toorganisational matters and advice.

Planning Implementation

PPMOrganisational Procedures

Organisational StructurePlan Op

Detailing andSpecifying Indicators

Recording / Filing System

Data Requirements Internal and External

Reporting

Collection

Processing

Decision Making Steering

The following procedures provide an outline of how to put the above-mentioned features of an MEA system into practice.

1. Clarifications prior to planning a system of MEA (see 3.2.2)

• examining existing concepts of MEA in co-operating agencies, whichmay form a basis for setting up a project system (see 3.2.2.1)

• clarifying organisational procedures and organisational structureswithin the project (see 3.2.2.2)

• re-examining the project planning documents for consistency (see3.2.2.3)


2. Design of the MEA system (see 3.2.3)

• performance level related MEA (see 3.2.3.1)

• impact level related MEA (see. 3.2.3.2)

• assumptions related MEA (see 3.2.3.3)

3. Implementation of a MEA system (see 3.3)

• organisational alternatives in the implementation of the MEA system(see 3.3.1)

• variations according to different phases and project life-cycles (see.3.3.2.1)

• cost implications of a MEA system and other matters (see 3.3.2.2)

14.2.2 Clarifications prior to planning a system of MEA

14.2.2.1 Examining existing concepts of MEA of co-operating agencies

Most projects cooperate with implementing agencies,or even support co-operating institutions in order toachieve the planned results / purpose / goal. Co-operation in MEA may have two aspects:

Many of these institutions have procedures for(planning and for) controlling the implementation

(probably referred to as "physical and financial progress control", "auditprocedure", etc.). It would be ideal if such procedures could be the basisof a MEA system for a project or if such existing procedures could beimproved in a way that they could be institutionalised in general.

• Data from routine surveys (for example population or household surveys,agricultural or health statistics, etc.) which are carried out by the localadministration may constitute a source of information for the projectmonitoring. (avoid double collection of data).

a


14.2.2.2 Clarifying organisational procedures (decision making structures) and theorganisational set-up of a project

As the MEA system has to lead to decisions on how tosteer the project, it is indispensable that this systemreflects the organisational procedure actually used inthe project, which in its turn is closely linked to theorganisational structure of the project. Organigrams,job-descriptions etc. are one basic input into theconstruction of an MEA system.

Specifically, one may pursue the following steps:

1. List the "management tasks" referring to (continuous) planning,reporting, and (most importantly) adjustments during theimplementation procedure;

2. Describe the decision-making which is involved in these differentmanagerial tasks by specifying "who" takes "which decisions""when";

3. Deduce the indispensable information required by the managerialdecision-making structure in order to be able to implement itsmanagerial tasks.

The last step in establishing a monitoring system can be seen as re-examination the organisational design of a project.

14.2.2.3 Re-examination of the planning documents

If the planning documents are to constitute the basicpoint of reference for MEA, they have to beconsistent, comprehensive and sufficiently detailed.The PPM and Plan of Operations therefore are to beexamined if one bases a MEA system on them.

If the planning documents do not provide aconsistent and clear guideline for the implementation process (e.g. themeans-end-relationships are not established satisfactorily, different planningdocuments do not match with each other, indicators are not sufficiently

ð

&


elaborated or specified) these observed deficits have to be made a topic ofdiscussion and necessary improvement have to be made. Such planadjustments may be subject to approval at higher levels.

In case agreement on the planning documents cannot be reached by thevarious parties involved (e.g. donor agency, umbrella institution, executingagency, project management, etc.), the MEA may focus on such points ofdisagreement and provide respective information and assessment withrespect to clarifying the direction of implementation. Eventually, externalevaluations will also have their role to play in fine-tuning the project plans tobe internally consistent:

14.2.3 Design of the MEA system

14.2.3.1 Performance related MEA system

"Performance" pertains to the manageable areas of a project, i.e. inputs andactivities

• Concerning: Inputs

The timely acquisition of materials, inputs, finance and the employment ofpersonnel in required quantities (and the necessary quality) is animportant condition for the successful implementation of a project. Thefollowing points need to be addressed:

ο acquisition and procurements of inputs

ο deployment of inputs according to plans and targets

ο cost calculations

The acquisition or procurement of inputs is based onthe Plan of Operation and the work plans, whichspecify which resources are needed at what time.

Administrative procedures of contributing agencies determine the details anddates of how and when the person with the relevant responsibilities will haveto deal with orders, employment, and liquidity. As these procedures may differamong varying parties contributing to a project (e.g. donor agencies,executing agencies, etc.) the need to co-ordinate efforts with the inputsmanagement may be high.

The actual deployment of financial resources on the basis of the PlanOp andPPM is usually unproblematic (= book-keeping system); but often theinformation required in order to adequately deal with the deployment ofpersonnel and material inputs are not sufficiently detailed. Here, records of allstaff activities (e.g. punch-cards), records on the use of equipment (e.g.logbooks of cars, etc.), delivery notes for materials or expenditure vouchersfor financial support (e.g., groups of beneficiaries) are required. Such proof


of or justification for the deployment of the project inputs need to be summedup in order to be able to draw managerial conclusions. No general guidelinescan be provided for this as the type of summary depends on the particularmonitoring requirements or inputs of different projects.

Cost accounting is one of the two aspects for monitoring the economicperformance of a project (the other aspect being the monitoring of benefits).Cost accounting requires common definitions of cost centres (e.g. sections ofa project) and cost categories (personnel, materials, current cost, externalcontributions, depreciation). Difficulties may arise where accounting systemsvary among the different contributors to a project. If costs are to be calculatedby clusters of activities or results then cost items are redistributed accordingto such cost units using a project specific conversion key for those itemswhich cannot directly/fully be attributed to one cost unit.

• Concerning: Activities

The plan of operation and the work plans are the basis for MEA ofactivities. There is no general rule on how to control activities. This willlargely depend on the assessment of all staff involved and on where thesupposed weak or critical points of the implementation process withregard to specific activities are. In principle, each activity can be checked:

ο in the beginning, in order to know whether the activity was initiated ontime;

ο during implementation in order to assess the progress of actions interms of the attainment of defined milestones;

ο at the end of implementing an activity, when the rate of achievementof a planned result can be checked.

The documentation of such monitoring can be done:

• by registering in the work plan certain relevant achievements duringthe process of carrying out activities within adequate time intervals(advisable for projects with simple structures);

• by designing special forms which list the activities and sub-activities ofthe work plan and contain further columns for timing, responsibility,steps in the achievement, assessment and comments, correctiveactions;

• by designing more detailed activity monitoring sheets (in the case ofvery complex projects).


Example of Format:

Section:Period:

Compare Identify Analyse Recommend

Deviations Causes Lessons Alternativeactions

Reasoning

Activity

Sub-Activity

Mile-stone /Indicator

Provisionskept

ActualAchieve-ments

Actualdisbursement

fromPlans

in Cost

14.2.3.2 Impact MEA system

The monitoring of project/program impact is hardly practised; theassessment of impact is often left to external evaluations. Therepercussion of such practice is that the project gets steered from outsideThe project management , being charged to steer according to definedobjectives, should not fear to systematically observe the project's impact.Periodic impact assessment presents the best opportunity to projectmanagers to learn about their development hypotheses. If this is donemanagement would be considerably strengthened with respect to co-operating and commissioning agencies as well as towards dealing withbeneficiaries.

1. First step of setting up an impact MEA: specify - detail - improveindicators

In order to design an impact MEA system one has to refer to the PPM.But usually, the indicators provided in the PPM will not be sufficientlydifferentiated to act as points of reference for the MEA system if it is tomake a useful contribution to the steering of the project implementationprocess. In many projects indicators will, therefore, have to be specified,detailed or improved in order to make them 'monitorable'. This is thefirst step in setting up an impact MEA system.

The following indicator-elaboration sheet may be used (possibly in aproject-adjusted version) as a checklist for or guide to revising indicators:


Indicator Elaboration Sheet

0. Identify the items of the PPM which are crucial for the impactmonitoring and need to be elaborated (maybe an 'objective', an'assumption' or an additionally identified, undesired 'negative sideeffect'): indicate the kind of item and PPM-level (goal, purpose,outputs, sometimes even activities);

1. Identify the important parameters in each: elaborate on what ismeant by the item = describe important aspects, in other words: whatis the rationale of the item; such specifications that can more easilybe made measurable;

2. Check already existing indicators for their validity andcomprehensiveness;

3. Identify all required indicators for the parameters :a) What exactly should be measured?b) Who is the target group? (in case this is applicable)c) Which area is meant ? (in case this is applicable)d) Which quantitative changes (targets) are envisaged (increase,

decrease, high, low)?e) By when should how much of the quantitative targets be

achieved?year:figures:

4. Justification: Why is this indicator a valid measurement of theparameter?

5. Definitions: What exactly is meant by key terms in the formulation ofthe indicator? In case calculations are used to compute the indicator:indicate the formula (will be of importance for summarisinginformation from project files in relation to the PPM indicators)

6. Targeting: How, by whom, on the basis of which information are thequantitative targets elaborated?

7. Validity / Limitations: Under which conditions is the indicator valid?Which aspect(s) of the item is not covered? Which other indicatorsare linked to this one in order to cover all aspects of the parameter?

2. Second step: identify sources of verification

The major task in this step (after having re-examined the indicators)concerns the identification of sources of verification and will largely bedetermined by the decision on how the MEA-system is integrated in theorganisation of the project. The following checklist may be used tooperationalise the sources of verification (the checklist is exhaustive andwould overburden the MEA system of most projects). The design of aMEA system has to determine for which indicators it will be necessary toestablish a fully fledged monitoring table and where reductions are


justified in view of practicability and / or of high costs in running the MEAsystem.

Checklist for elaborating sources of verification

A. Check on indicators

1. Validity of the indicators: is the (set of) indicators substantial,target-oriented, plausible, independent, measurable, verifiable?

2. Specification of indicators: is the (set of) indicators precise with

respect to quality, quantity, envisaged beneficiaries, location,time?

B. Operationalising sources of verification

3. Initial status: which initial situation (or bench mark) is theindicator related to?

4. Data and information requirements: which data / informationis required for the precise verification of the indicator?

5. Method of acquiring data and information: how can therequired data and information be acquired?

6. Date and frequency of acquiring data: when and how often isthe required data and information to be acquired?

7. Responsibility for acquiring data and information: whoacquires ?

8. Inputs for acquiring data and information: which means(working time, transport, etc.) are required? What are thecosts involved ?

C. Preparing the evaluation of collected data and information

9. Quality of data and information: How can the quality andreliability of data and information be assessed ?

10. Processing of data and information: How can the data andinformation be put together, so that they reflect the indicatorand facilitate assessment? Who is responsible for suchprocessing/summarising ?

11. Competence for evaluation: Who assesses data andinformation and elaborates proposals for steering the projecton the basis of such assessments ? Who will have to takedecisions ? Who will have to implement corrective actions?


If set threshold-levels (as specified by indicators or milestones) are reached,a warning-signal would trigger further investigation with regard to the causesand implications. Adjustments may then become necessary. Here it isadvocated that effects of project measures on the environment, employment,income distribution, the socio-cultural situation, etc. are not only assessedonce in the conceptualising of a project before it actually starts, but areintegrated into the MEA system.

14.2.3.3 MEA of assumptions

A project as a change agent will only be one of several intervening set-upsorganisations which cooperate with other institutions and agencies toimplement an intervention. It functions under frame conditions, which arespecified in the assumptions. Both the co-operating agencies and the frameconditions will have to be monitored with a special focus. In addition to this,project interventions will also include how it reacts to unforeseen side-effects, (be they positive or negative), which may result from theimplementation.

For MEA purposes it is thereforenecessary

• to specify assumptions withindicators and to identify sources ofverification for assumptions- /indicators (the same procedure aswas described for indicators andsources of verification for theobjectives of the project can beapplied), and

to add a special list of unintended,particularly negative, side effects (the 'worst case scenario') to thestructure of the PPM, which again should be specified by indicators andsources of verification (same procedure).

The questions to be asked when elaborating side-effects are:

• "What undesired negative effects may an activity, a result/outputhave?"

• "What can go wrong on purpose and goal levels?"


14.2.3.4 Example of Steps for integrating indicators and sources of verification intothe organisational procedure of a project involving a high level of interactionwith envisaged beneficiaries

Step 1 Elaborate as specifically as possible all indicators pertainingto objectives, assumptions and negative side-effects for alllevels of the PPM, (the same information required as inplanning); avoid abbreviations

Step 2 List the management tasks of the project for the plannedperiod and aspects (e.g. 3-years-planning, annual replanning,financial budgeting, etc.), including reporting and controllingof the implementation process(e.g. short-term projectsteering, policy-making, controlling of support to envisagedbeneficiaries, decision-making on support, supervision,impact surveying, etc.);

Step 3Describe the decision-making which is involved in thedifferent managerial tasks by specifying "who" takes "whichdecisions" "when";

Step 4Deduce the information requirements pertaining only to thedecision-making tasks

Step 5Connect the "information required" to the project's internalreporting and filing system (e.g. reports of line personnel oninteraction with envisaged beneficiaries to the staffpersonnel) which constitutes the project's central "source ofinformation";

Step 6Review the internal reporting and filing system so as to


integrate all information requirements into the reports;

Step 7Design adequate procedures for the staff so that they areable to adhere to policy guidelines and can provide adequateinformation (e.g. questionnaires for interviewing groups ofenvisaged beneficiaries, checklists for decision-making onsupport measures granted by the project to beneficiaries,etc.);

Step 8 List indicators according to sources of verification;

Step 9Establish an information flow chart concerning the timerequirements of reporting


14.3 Implementation of MEA System

14.3.1 Organising the implementation of MEA-System

Introduction:

Designing and establishing a MEA system requiresa certain degree of specialised knowledge which aproject may have to contract from the outside. Ofequal importance is the involvement andparticipation of the project staff concerned, as only

they have intimate knowledge of the project reality. An advisor on MEAtherefore has to be able to master group / team steering techniques tofacilitate the elaboration of the system with the project staff.

For the purpose of simplification, three basic alternatives for organising theimplementation of MEA systems are described:

14.3.1.1 The project team carries out monitoring - evaluation - adjustment itself

Normally the execution of the system shouldbe a regular and integral part of the work ofthe project team. Thus, the learning effect onexperiences gained is safeguarded in anoptimum fashion, and the self steeringcapacities are enhanced in a way whichfosters participatory management. Such anapproach,

however, requires that the MEA is merged with the procedures and decision-making structures of the project. Procedures and respective formats(integrated in a organisational system) have to incorporate indicators andmilestones comprehensively and meaningfully. The amount of energy devotedto carrying out the MEA system would then, however, be substantial for everystaff member and would also make up a share of the total inputs available.

F


14.3.1.2 Establishing a monitoring - evaluation unit or employing a monitoring andevaluation specialist

This is recommendable in large,complex and / or multi-sectoralprojects. Such a specialist may initiallybe used to introduce the design of anMEA system in a project. If it is tobecome a permanent feature in aproject,

• the unit is to be kept as small aspossible;

• the backward and forward linkages (planning, adjustment) have to beclarified;

• the unit or the specialist has to make sure that it / he / she rendersservices to all sections of and persons in the project;

• the information requirements as well as the survey and assessmentmethods are to be defined in participation with other project staff.

14.3.1.3 Project external specialists for monitoring and evaluation

They could be contracted

• in order to introduce a MEAsystem

• if the project team is so fullyoccupied with work that themonitoring and evaluation of theproject impacts and frameconditions in particular can notbe done by the project personnel

• where it is conducive toperiodically have external

backstoppers in a project in order to remedy conceptual andoperational weaknesses and to overcome "routine-blindness" of theproject staff.


14.3.2 Other issues related to MEA

14.3.2.1 Variations of monitoring - evaluation -adjustment according to differentphases in the project life cycle

Phase of a Project Focus ofMEA

MEAPredominantlyCarried Out by

Orientation Phase:

project implementationstrategies are worked outwhich explicitly takeaccount of economic,political, institutional andecological frameconditions as well as thecircumstances of potentialgroup of envisagedbeneficiaries andimplementing agencies

• reactions ofdifferentgroups ofenvisagedbeneficiariestoconfidencebuildingmeasures

• actualcommitmentof potentialco-operatingagencies

• assessmentof changesof frameconditions

• the project itself

Innovation Phase:

socially, economically andecologically adjustedtechnical andinstitutional/organisationalsolutions are identified inclose collaboration withgroups of envisagedbeneficiaries and with theagencies in charge - thesolutions are tested underconditions which arerepresentative of theproject

• acceptanceof identifiedsolutionswith thedifferentgroups ofenvisagedbeneficiariesand with co-operatingagencies incharge

• the project inclose co-operation withbeneficiariesandimplementingagencies


Phases of a Project Focus of MEA MEAPredominantlyCarried Out by

Dissemination andIntegration phase:

the groups of envisagedbeneficiaries and agenciesin charge have adoptedthe new techniques,procedures and functionsin order to implement themon a demand-coveringscale, which puts thebeneficiaries in a positionto reach necessaryservices, markets andinformation and enableservice institutions toprovide the relevantrequirements

• access ofbeneficiariesto services,markets andinformation

• effectivenessandefficiency ofregularagencies inprovidingsupport tobeneficiaries

• the agencies incharge

• a self evaluationof groups ofbeneficiaries

Consultation andConsolidation Phase:

beneficiaries and agenciesin charge are able toimplement identifiedsolutions on their own andproject executing agenciesare enabled to flexiblyadjust solutions tochanging conditions, toidentify, develop andinitiate new solutions tonew problems

• co-operationofbeneficiariesandagencies incharge

• changingframeconditions

• higherauthoritiescharged withsupervisingimplementingagencies

14.3.2.2 Cost Implications of MEA

It is not possible to provide a general guideline for the "reasonable" cost ofMEA as this would depend on the type of project and the project phase. Butproject funding agencies consider an average of 4-10 % of the total projectcosts to be justifiable for MEA.

14 Quality Control: Monitoring, Evaluation & Adjustment...

Documents

Transcript of 14 Quality Control: Monitoring, Evaluation & Adjustment...