Systemic impact evaluationEvaluating impacts of agricultural projects

The example of SOGUIPAH in Guinea

AFD - J. Delarue

What do we want ?

• Rigorously estimating impact (counterfactual approach)

• Accounting for impact heterogeneity

• Evaluating impacts on the lifetime of the project’s outputs

What is SOGUIPAH?

• An agroindustrial company in Guinea: SOGUIPAH (1987), produces rubber (exports) and palm oil (national market)

• Land allocation : 23 000 ha Farmers deprived of land

• SOGUIPAH helps other farmers develop perennial cropping systems : A model farmer : 1 ha oil palm, 2 ha rubber and 0.5 irrigated rice

SOGUIPAH

Evaluative questions

• What are the impacts of SOGUIPAH on the farmers’ income on the lifetime of the plantations?

Impacts of land allocation Impacts of out-grower scheme

• Who is impacted by the project? directly Indirectly

• Why is there such an impact?

Evaluation context and methodological challenges

An evaluation 15 years after the project started Very little baseline and follow-up data

Estimating impacts : a counterfactual approach

Counterfactual at village level

Counterfactuals for the different types of farms

Matching on observables +

their combination in a system

Agrarian system analysis : matching villages

History

Region

Agricultural land use

EcosystemSocial

relationships

Village WITH SOGUIPAH

Village WITHOUT SOGUIPAH

Farming system analysis : matching farms

Tools Labour

Land

Family

Cropping systems and breeding systems

Labour calendar

Land allocation

Cash calendar…

Baseline situation

WITH SOGUIPAH

WITHOUT SOGUIPAH

RESULTS :

5 farming systems before SOGUIPAH

11 trajectories with SOGUIPAH

7 trajectories without SOGUIPAH

Agrarian system analysis : activities and results

ACTIVITIES :

1. In depth interviews (focus groups, farms)

2. Landscape observations

3. Secondary data

… and triangulation

RESULTS :

2 comparable villages

Understanding interactions of SOGUIPAH with the agrarian system (land market, labour market, prices…)

Indentification of farming systems

Farming system analysis : activities and results

ACTIVITIES :

1. In depth interviews (recall method at farm level)

2. Field observations

3. Productivity analysis nowadays

… and triangulation

RESULTS :

Comparable farms before SOGUIPAH

Trajectories of farming systems with and without SOGUIPAH

Economic results with and without SOGUIPAH for a variety of cropping systems

What is the impact we want to estimate ??

income

1987 2005

A cross-section-based measure of average impact on the population is not what we’re looking for !

Impact ?

0

10 000 000

20 000 000

30 000 000

40 000 000

1987 1997 2007 2017 2027 2037

GNF

con

stan

t

Kpoo 3

Galaye 5

Mesure d'impact ex post

Projectionex ante

Modeling economic results with and without SOGUIPAH for one farming system

2005

Translating trajectories into a simple economic model based on financial analysisAssessing future impact based on structural trends

(yields) and a few assumptions (prices) Impact on the lifetime of the plantations

Accounting for heterogeneity

0

4 000 000

8 000 000

12 000 000

16 000 000

1987

1990

1993

1996

1999

2002

2005

2008

2011

2014

2017

2020

2023

2026

2029

2032

2035

2038

2041

GN

F c

on

stan

t

Guilamou 1

Galaye 1

Mesure d'impact ex post

Projection ex ante

0

10 000 000

20 000 000

30 000 000

40 000 000

1987 1997 2007 2017 2027 2037

GN

F c

on

stan

t

Kpoo 3

Galaye 5

Mesure d'impact ex post

Projectionex ante

0

4 000 000

8 000 000

12 000 000

16 000 000

1987

1990

1993

1996

1999

2002

2005

2008

2011

2014

2017

2020

2023

2026

2029

2032

2035

2038

2041

GN

F c

on

stan

t

Kpoo 4

Galaye 1

Mesure d'impact ex post Projectionex ante

0

1 000 000

2 000 000

3 000 000

4 000 000

5 000 000

6 000 000

7 000 000

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

2019

2021

2023

2025

2027

2029

2031

2033

2035

2037

2039

2041

2043

GN

F co

nsta

nts

Kpoo 5

Galaye 4

Mesure d'impactex post

Projection ex ante

Impacts are highly differentiated ! Average : + 200 €/y ranging from – 200 €/y to + 1530 €/y

Conclusions about the systemic impact evaluation

• A qualitative approach which allows the quantification of impacts

• A very simple economic model based on high quality data

• It assesses the impacts along the life time of the project’s outputs

• It takes into account the complexity of real world

• It provides an in-depth understanding of the project effects and allows formulating recommandations