http://www.madagasikara-

voakajy.org/index.php

Assessing the impact of

environmental management interventions on ecological

outcomes

Julia P. G. Jones1, Ranaivo Rasolofoson1,2, Edwin Pynegar1

1School of Environment, Natural Resources and Geography, Bangor University

2 Department of Food and Resources Economics, University of Copenhagen

“Those who cannot learn from history are doomed to repeat it”

Georges Santayana 1863-1952

Key message: social phenomena influence impact evaluation of environmental management interventions

Randomised Control Trial: units are allocated to the intervention group or control group randomly

Intervention Control

Random allocation

Randomised Control Trials: mainstay of much applied ecology

RCTSs are increasingly advocated for more complex environmental interventions

It is well recognised that impacts of an intervention may ‘spill over’ (from control to intervention units)

Such ‘biophysical spill-over’ may mask the impact of the intervention

Imitation of the intervention in control areas (social spill over) can mask the true impact of the intervention in an RCT

Important social phenomenon #1: interventions may be imitated in the control units (social ‘spill over’)

Pynegar, Gibbons, Asquith & Jones (submitted) What role should Randomised Control Trials play in providing the evidence base underpinning environmental management?

Intervention should have a direct impact on outcome of interest

Intervention aims to change behaviour

Change in behaviour should impact the outcome of interest

Interventions which aim to change behaviour are fundamentally different from those aiming to have a direct impact on outcome of interest….

John Henry effect (those in control group increase their effort)-will reduce apparent effect of the intervention

John Henry effect

Important social phenomenon #2: people behave differently when they are being experimented upon

Pynegar, Gibbons, Asquith & Jones (submitted) What role should Randomised Control Trials play in providing the evidence base underpinning environmental management?

Intervention Control

Pre-matching comparison

Post-matching comparison

Matching

Quasi-experimental approach: units in intervention group are “matched” with control units which are as similar as possible

Relative effectiveness of different approaches to Community Forest Management at delivering conservation outcomes in Madagascar?

CFM which do not allow commercial forestry

CFM which do allow commercial forestry

Matched based on:

Proximity to recent deforestation

Proximity to a road

Proximity to an urban centre

Quality of agricultural land

Altitude

Result: CFM which allows commercial logging appears to have increased deforestation, CFM which does not allow commercial logging appears to have reduced deforestation

Source: Rasolofoson, Ferraro, Jenkins & Jones (submitted) The effectiveness of Community Forest Management at slowing deforestation in Madagascar

BUT: We know CFM (and the types of CFM) were not allocated randomly, there will likely be reasons (social/political/historical) not captured in our matching variables

Important social phenomenon #3: True matching is challenging without social/institutional understanding

of process of allocation to the intervention

http://www.madagasikara-

voakajy.org/index.php

Conclusions

• Talk of a ‘gold standard’ for impact evaluation is unhelpful.

• Social phenomena influence evaluation of the impacts of ecological management interventions and perhaps deserve more attention.

http://www.madagasikara-

voakajy.org/index.php

Thanks (@juliapgjones)

Nigel Asquith Paul Ferraro James Gibbons Clinton Jenkins

Edwin Pynegar Ranaivo Rasolofoson