Márcia Paterno JoppertBrazilian Monitoring and Evaluation Network

October 2013

Multipaper session: The State of Evaluation Practice in the Early 21st Century in Latin

America: Realities and Challenges

A Brazilian portrait of the Monitoring and Evaluation field

Agenda

• Some contextual data about demand and supply in Brazilian evaluation field

• The practice • The importance of VOPEs for the mobilization and

articulation of a set of stakeholders around the development of M&E field

• The BMEN features, role and governance model

Some contextual data – Potential Demand in Brazil

Study on the Supply and Demand for Services in Evaluation (PRODEV/IDB-2010)

Demand•Currently, little demand by public sector (more internal initiatives)•International organizations (still) play leading role•Difficulty in finding professionals / organizations that meet expectations

Supply•Few specific courses (3 masters)•Little information about offer (universities, individual consultants, firms (-))•Where are the experiences?•Lack of theoretical framework•Lack of a public policy on evaluation, however successful institutionalization initiatives (Ministry of Social Development, General Court of Audits)

Study on the Supply and Demand for Services in Evaluation (PRODEV/IDB-2010)

The Practice

• Brazilian Federal Governemnt– We don’t know what is being evaluated and how

(quality) – We don’t know what are the results that the evaluation

studies are generating for the improvement of public policies and if they are being properly disseminated (transparency)

The Practice: CSO and PSI

More about practice – survey with 83 professionals – ago/13

Objects and scope

Outputs and outcomes

Outputs and outcomes

More about practice – survey with 83 professionals – Aug/13

Sectors

More about practice – survey with 83 professionals –Aug/13

The Process

• Evaluation Questions: stakeholders and beneficiaries rarely or never participate in definition

• Involvement in different evaluation stages: beneficiaries and auditors, almost never

• Data collection strategy: traditional tools (documental analysis, secondary data and auto applicable questionnaires)

• Communication Strategy: formal reports, executive summaries and meetings with decision makers

The importance of VOPEs for the mobilization and articulation of a

set of stakeholders around the development of M&E field

VOPEs Evolution

Source: IOCE

VOPEs’ Mapping

Source: IOCE

BMEN - Brief History

• 2003: Launch of the Brazilian Evaluation Network • Nov/2008 – The JPF, WB and IDB launch the Brazilian

M&E Network during the 4th Meeting of the Latin America and the Caribbean Monitoring and Evaluation Network, in Belo Horizonte/MG – 1st annual meeting

• From 2009 to 2013 - Annual Seminars and Meetings• 2011 - launch of the Brazilian M&E Journal (currently

in 3rd edition)

BMEN – Features, Role and Governance

• Informal initiative• 4,640 “members” (13.10.2013)

– people and institutions: academic, public managers, development funders, multilateral agencies, civil society organizations, private companies, control and auditing agencies, statistical bodies

– All the 27 Brazilian states– 10% from abroad

• Theoretical, methodological and sectorial plurality

BMEN – Features, Role and Governance

We articulate and mobilize stakeholders by:•Disseminating knowledge and information (annual seminar, publications, articles, news, etc..)•Promoting interaction and exchange of experience (annual seminar, networking, participation in other networks and events: RedLacMe, ReLAC, IOCE, EES, AEA)•Promoting and supporting capacity development initiatives (events, courses, workshops)•Supporting the demand (job opportunities, bank of professionals, contracting management)

BMEN – Features, Role and Governance

... And a Governance Model: •Plural Management Committee: representatives of the various stakeholders (academia, private sector, civil society, public managers, statistical agencies, funders)•Development of a Strategic Plan, containing: organizational model, results-oriented projects, potential sponsors and partners, planning mechanisms and M & A)•International Counseling (IOCE, CES Evalpartners)