The Market Facilitation Initiative (MaFI) – an initiative of The SEEP Network

“Systemic M&E”

Catalyzing a global dialogue to improve practice in measuring market system change

Concept note, version 31 July 20121. Background In recent years, pro-poor (or inclusive market development) initiatives have shown a major shift away from direct inputs and service provision towards the creation of appropriate conditions for public and private stakeholders to drive structural changes in the market system more easily and more cost-effectively. This paradigm shift was driven by the realization that only through a systemic or holistic approach to markets and a focus on stakeholder-driven solutions, can these initiatives reach sustainability and scale.

When considering markets from a systems perspective, it becomes crucial to recognize that markets are not predictable machines, but rather complex and dynamic systems that are constantly learning.1 For this reason, using linear and deterministic methods and tools to measure how markets change, and how we change them, is not only inappropriate but can lead to wasted resources and missed opportunities to achieve poverty reduction.

As emphasized by Dr. Jeanne Downing from USAID in her presentation to the market development community at the M4P Hub conference in 2011, there is growing consensus about systems thinking and market facilitation, there is however a recognition that evaluating projects implementing these approaches as projects is difficult. Current impact evaluations require that counterfactuals are established using control groups, treatment versus non-treatment groups are defined, differentiation is made between program participants and non-participants, and baselines and endlines are needed. In contrast, effective market development projects move after baselines have been completed, no clear distinctions are made between program participants and non-participants as projects aim to catalyze change beyond direct assistance, projects and activities evolve over time, and control groups become contaminated as spill-over effects are seen as an explicit success of effective market facilitation practices.

New thinking and practice is needed that is relevant to complex and evolving projects. Complex and adaptive systems must be monitored and measured using approaches that embrace complexity and facilitation principles, concepts and tools. In order to do this, we need a systemic approach to M&E2. To our knowledge, no systematic process of collective learning with a focus on M&E of inclusive market systems and with the participation of a global community of practitioners and donors has been attempted.

2. Project in brief

The project is designed to catalyze an international process of debate, dialogue and learning between donors and practitioners who are trying to support market systems to work better to reduce poverty at a large scale. It will achieve this through an intentional learning process comprised of in-person workshops, e-consultations, plenary presentations and discussions, and publication of an ongoing discussion document.

The project will assist donors and practitioners alike to improve understanding and practice of systemic approaches to M&E is and its application to inclusive market development programs worldwide.

1 These are often referred to as complex adaptive systems (CSAs).2 This is not a new idea per se. Experts such as Michael Quinn Patton have written extensively about the need for such types of approaches (which he refers to as “Developmental Evaluation”).

1

3. Project context – the conditions are ripe for success

The project in itself is relatively limited in terms of budget and duration but it will play a key, catalytic role in unlocking and cementing ongoing and long-term processes that have been promoted by USAID and the SEEP Network, mainly through one of its initiatives: The Market Facilitation Initiative (MaFI), in recent years. These processes are geared towards revolutionizing market development approaches and private sector engagement.

Three examples of ongoing processes that create fertile conditions for this project to succeed are:

● MaFI-festo: A distillation of discussions that MaFI members have had over the past several years on the changes required to make the international development cooperation system more efficient and effective through facilitation approaches and complexity science. One of the four issues proposed by the MaFI-festo is precisely “changing what and how we measure change”.Catalyzing the learning about systemic approaches to M&E will show the 250+ MaFI members and donors, such as DFID and SDC, that turning the MaFI-festo ideas into real change is possible. This will in turn unleash the energy that has been building in the past several years around the MaFI-festo principals and motivate key donors and hundreds of practitioners to contribute to the success of this project.

● Complexity Dialogues: A series of conversations promoted by MaFI members in different parts of the world to bring inclusive market development practitioners and complexity experts together to find concrete applications of complexity science in inclusive market development practice. The Complexity Dialogues are currently in Phase 1: the production of a learning agenda (a set of questions) built by MaFI members using the inputs of a 12-lesson video course by complexity expert Prof. Scott E. Page.The Complexity Dialogues are building a global movement of reflection and learning about complex systems in the context of inclusive market development practice that will enrich the discussions that will take place as a product of this project.

● USAID’s initiatives: Among all key donors, USAID is one of the thought leaders when it comes to innovative and cutting-edge policies and frameworks about complexity in development practice. For example, in Nov 2011, Dr. Tjip Walker from USAID’s Policy and Practice Learning Unit collaborated with two MaFI members to design and deliver a workshop on complexity and inclusive market development during the SEEP Annual Conference. Dr. Walker talked about the importance that USAID is giving to these issues, especially in highly volatile contexts like Afghanistan, and how much donors needs practitioners to demand more progress in this field. One month earlier, Dr. Walker had organized a 1-day workshop for USAID staff to learn about complexity issues and concepts.

Other senior USAID staff, such as Dr. Stacey Young and Dr. Jeanne Downing, are working with USAID colleagues and partners to promote complexity concepts and proposing ideas that support the construction and application of systemic approaches to M&E frameworks.

4. Potential consultants and experts

Potential leads managing this process and shaping the learning agenda:

2

● Marcus Jenal, Consultant involved in complexity sciences and market development. Co-editor of SEEP’s MaFI-festo. Active member of SEEP’s Market Facilitation Initiative (MAFI).

● Lucho Osorio-Cortes, International Market Systems Specialist at Practical Action Consulting and Coordinator of SEEP’s Market Facilitation Initiative (MaFI). Co-editor of SEEP’s MAFI-festo.

Mr. Jenal and Mr Osorio-Cortes will assist in writing discussion papers, mobilizing involvement and input from MaFI members and other relevant experts, co-facilitating online discussions, synthesizing the online discussions, supporting the participation of high level speakers at the SEEP’s Annual Conference plenary, and designing a practitioner learning agenda in moving these dialogues forward.

Potential experts to involve in online e-consultations, plenary presentations, in-person workshops, and input into discussion papers involve complexity scholars as well as development practitioners that are recognized as applying an innovative and forward looking approach to M&E using insights from the field of complexity and processes of social change.

5. Activities and outputs

This initial project will consist of a introductory paper, an e-consultation and a plenary session during the 2012 SEEP Annual Conference.

The e-consultations will bring practitioners, complexity experts and donors from all over the world together to discuss systemic approaches to M&E: what it is or should be, what principles should underpin it, how a general framework would look, and how to make it practical for inclusive market development practitioners and policy makers.

September 2012: A discussion paper that will inform and catalyze an online e-consultation.

Output 1: Initial discussion paper to catalyze initial dialogues, with a potential focus being a synopsis of how are current M&E practices may be undermining and hampering good inclusive market development and facilitation practices and how a new paradigm for M&E could look like. This paper could draw on existing literature by M&E experts such as David Snowden, Michael Quinn Patton, Don Snodgrass and Lucy Creevey.

Output 2: Online e-consultation with discussion synthesis

November 2012: At the beginning of November, select speakers will be invited to participate in one of the SEEP Annual Conference plenaries with an audience of over 200 professionals in-person and an internet audience far beyond this. They will use this space mainly to:

● send two clear messages to the development industry: “better ways of measuring impacts in market systems are possible” and “if we want the current M&E paradigm to improve we have to be in this together for the long run and have a joint, long-term vision”

● share the key results from the first round of discussions (the discussion paper and e-consultation)

● capture feedback from the audience to improve the discussion paper and identify missing issues

○ Proposed plenary: Rethinking the current M&E paradigm - measuring impacts in market systems: This plenary will explore cutting edge thinking and enquiry around ways to improve the current M&E paradigm and improved practice in measuring impacts in market systems. When considering markets from a systems perspective, it becomes crucial to recognize that markets are not predictable “machines”, but rather complex and dynamic “organisms” that

3

are constantly learning. Changes in complex and adaptive systems must be monitored and measured using approaches that embrace complexity and facilitation principles, concepts and tools rather than linear and deterministic ones. This plenary will bring economic development practitioners and complexity experts together to explore applications of complexity science in inclusive market development. The plenary will explore ideas that support the construction and application of systemic approaches to M&E of inclusive market development, the principles that should underpin this and how to make it practical for practitioners and policy makers.

Output 3: Plenary presentation and discussion published in online video format

Output 4: Presentation materials and discussion synthesis

6. What happens after the project? Main outcomes

● Increased levels of energy, awareness and commitment among inclusive market development practitioners and donors (including MaFI members) that will contribute to making the application of systemic approaches to M&E a reality in inclusive market development.

● A sense of possibility and practicality in the field of inclusive market development about systemic approaches to M&E – where it is currently perceived as a complicated or intractable issue, which it is not if we use the right paradigms and concepts. This can contribute to the adoption of systemic approaches to M&E by NGOs and donors.

● Increased reputation and legitimacy of the organizations leading this project. in the case of SEEP, reputation as a thought leader is essential for its capacity to mobilize and align the interests of its members

● Development of a strong foundation for learning programs that incorporate systems approaches to M&E to be applied in planning and evaluation phases by practitioners and donors.

4