Navigating Participatory Monitoring and Evaluation: notes and lessons from the CSO African Peer Review Mechanism Monitoring Project (AMP) in South Africa

OgochukwuNzewi (PHD) 

Intro and context A discussion of notes and lessons

1. Asks how much systemization and rationality one can expect within a PM&E process?

2. Should legitimacy of PM&E process be ascribed based on the participatory component?

3. If so how about other legitimising standards such as validity and reliability of your methods, tools and instruments?

In July 2010, the African Peer Review Mechanism Monitoring Project (AMP) was launched in Johannesburg

AMP: To build capacity among CSOs for M&E of the implementation

of the APRM National Programmes of Action (NPoA) establish a core group of CSOs who have the capacity to track

the implementation of the APRM National Programmes of Action.

Experience and lessons sharing on recommendations

APRM in South Africa APRM established in 2003, lauded as “Africa’s Innovative

Thinking on Governance” voluntary accession by states A dual approach: self assessment = Country Self-Assessment

Report CSAR) ; external evaluation by a Country Review Mission (CRM) = Country Review Report (Country Review Report). Based on the report, country prepares and commits to a National Programme of Action (NPoA)

South Africa acceded to the APRM in 2004. 2006: Conducted and submitted its own Country Self-

Assessment Report (CSAR). 2007: the South African external review was completed

resulting in the Country Review Report (CRR).

PM&E Transformative PM&E (Latin America and parts of Asia)

sensitivity and affability to social problems PM&E objectives, principles and process determinants

are hinged on decisions from engagements with stakeholders

PM&E is seen as a useful good governance tool in monitoring and evaluation of development projects and programs as it establishes legitimacy to M and E findings

PM&E produces lessons for stakeholders and there is a greater sense of ownership in findings which motivate the development of action plans and encourages greater accountability for stakeholders and society at large

AMP as a PM&E process Implies: Process issues will focus on the critical

matters affecting participation as a point of departure in M&E

Who is monitoring, what is being monitored and weighing and negotiating interests and goals of stakeholders. Estrella (2002)

Who monitors? AMP Participatory framework

1. A central working team2. CSO clusters3. Government

4 Participatory phases1. scoping and planning workshop, 2. training workshop in which CSOs were invited to

propose what to monitor3. the third stage was the knowledge generation

stage which included data gathering, analysis and report writing

4. validation and ‘rating’

Who monitors? Lessons: Ownership: balancing the objectives of project with

stakeholders frames Commitment: Levels of commitment hinged on different

interest factors. Commitment from stakeholders is not always sustainable.

Representativeness: It is possible that participation is not exhaustive. Decisions on who should be involved still rests with the convener, however setting limits and making decisions on who should be involved is while not a bad thing in such a process, must be motivated from the outset.

Where ‘experts’ become outsider facilitators, and stakeholders become ‘experts’ there will be problems in maintaining expectations of high technical merit.

What to monitor Streamlining: a wide range of governance issues to

contend with. Result: CSO process outlined governance themes.

The appropriateness of the NPOA as a measuring yardstick: Ideally, NPOA but…

Vagueness of NPOA Indicators M&E Framework: expectations for PM&E in terms of

conventional M&E frameworks difficult to actualise Methodology: Data gathering and analysis: triangulation

of sources albeit generalized and arbitrary. Validation/rating

What to monitor

Lessons: What to monitor will always be a critical issue in a PM&E

process. While the decision to not deal with indicators was a collective one and it made the AMP process a lot more manageable as an M&E project, it presented considerable methodological concerns especially in the terms of the validity of results and the ratings.

Participatory Monitoring and Evaluation is prone to anecdotal evidence as against methodological data gathering.

However if seen as transformative PM&E, what the AMP process lacks in technical merit, it delivers in full dose as a strong advocacy instrument for reform. Perhaps in this regard, conventional M&E practice may fall short.

Negotiation and weighting of interests Four features that are associated with good PM&E

practice: participation, learning, negotiation and flexibility

Implies the dynamics of group behavior, which from the group theory standpoint diagnoses the potential struggle between groups and individuals.

Alexander George (Designing public policy) “some kind of bargaining process is likely to operate within the group even if members are unaware of it”

a major variable affecting group bargaining processes is the attitude and behaviour of the authority convening this group.

Negotiation contd Lessons:The convener/expert facilitates.

Facilitation involves establishing clear objectives for the project, mediation and an ability to negotiate terms and conditions among stakeholders in such a way as to maintain the essence the project objectives.

conclusion Certain schools in PM&E essentially see participation

as validating and legitimizing a M&E process. This is debatable. If that is so, like the AMP, can the relative success of guaranteeing some level of participation at every stage of its process produce a reliable report that can be used to benefit decision making in governance?

the lessons above show us that there is a weakness in the capacity of CSOs to undertake such M&E processes especially in terms of negotiating measuring instruments and indicators and in ensuring technical cleanness in the process

For full article go to: http://www.sabinet.co.za/abstracts/afrins/afrins_v41_n4_a3.html

Africa Insight 9vol 41) Issue 1