Integrating complexity into policy evaluation and enabling ... · and enabling better policy...
Transcript of Integrating complexity into policy evaluation and enabling ... · and enabling better policy...
![Page 1: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/1.jpg)
Integrating complexity into policy evaluation
and enabling better policy making:
Introducing the Centre for the Evaluation of
Complexity Across the Nexus (CECAN)
Dr Candice Howarth
Senior Research Fellow & Knowledge Integrator, CECAN
University of Surrey
5th European Environmental Evaluators Network Forum, Copenhagen, 15-16 Sept 2016
![Page 2: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/2.jpg)
Policies across the environment, food, energy, water nexus are entangled in complex ways
∙ Span multiple sectors
∙ Cross stakeholder impact and influence
∙ Conflicting timescales and cultures
Influenced by pressing issues such as climate change, loss of
biodiversity, poverty, food scarcity etc.
Innovative methods are needed to evaluate complex policy interventions
∙ Bring evidence & expertise from range of domains & stakeholders
∙ Cross sector and integrative
Exploring the nexus
![Page 3: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/3.jpg)
CECAN aims to transform the practice of
evaluation in the Nexus to make it fit for a
complex world
![Page 4: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/4.jpg)
Evaluation questions
Impact evaluation
What difference did the policy make?
Process evaluation
How was the policy delivered?
Economic evaluation
Did the benefits justify the costs?
![Page 5: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/5.jpg)
WHY IS
EVALUATION
DIFFICULT?
![Page 6: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/6.jpg)
Society is inter-connected…
![Page 7: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/7.jpg)
…multi-level
![Page 8: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/8.jpg)
What is a complex system?
Many independent, heterogeneous units,
…that interact,
… give rise to system behaviour as a whole
…which is more than the sum of its parts
Micro to macro systems – neutrons to nations
Multi-level, nested interactions
Emergence – behaviour of a system emerges
Feedback is important
![Page 9: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/9.jpg)
WHAT IS
CECAN DOING
![Page 10: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/10.jpg)
CECAN Vision
To integrate complexity into policy evaluation
and to enable more effective policy-making
To catalyse a step-change policy evaluation for
complexity and Nexus issues
![Page 11: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/11.jpg)
Developing methods for policy evaluation in
complex domains
Testing these methods with a range of case
studies
Supporting knowledge exchange and capacity
building
CECAN is..
![Page 12: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/12.jpg)
Co-Funders
![Page 13: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/13.jpg)
Partners
![Page 14: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/14.jpg)
ACTIVITIES
![Page 15: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/15.jpg)
CECAN acknowledges from the outset, the issue of
equifinality: there are many pathways to the same outcome
Developing:
• Timely methodological solutions that actively respond to
the fact that context matters and different approaches
may be needed (even for the same problem)
• Methodologies that are also suitable for a non-nexus
topics and can be re-purposed in other complex social
settings (e.g. health care, urban planning)
Methodological development
![Page 16: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/16.jpg)
Addressing methodological challenges
Policy design and evaluation
Scaling Causality
Context TimeInter-
disciplinarity
Framing Big data Cost
![Page 17: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/17.jpg)
Potential case studies
Regulating our Future' evaluating new approaches to food safety
regulation - Food Standards Agency
Lessons from evaluating complexity in the Transitional Arrangements
of Electricity Market Reform - BEIS
Procuring complexity-ready evaluation - lessons from Water
Abstraction Reform Agent Based Modelling evaluation - Defra
Evaluation strategies for Marine Plans in England - Defra
Understanding the effectiveness of enforcement interventions on
waste crime - Environment Agency
Programme impact and scheme interactions in the Rural
Development Programme for England - Defra
A full system analysis of the Renewable Heat Incentive - BEIS
![Page 18: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/18.jpg)
Example of methodologies to explore
Agent based modelling
Type of simulation where 'agents' (e.g. individuals, communities,
neighbourhoods, etc.) assigned rules about how they interact over time & space
Strengths
• Observing at a macro-level sum of micro-level level behaviours & interactions
• Handling non-homogeneous &inter-dependent behaviour
• allowing the effect of a policy intervention to be simulated &explored
Usage
• Modelling ex ante ‘what if?’ questions about policy options, and exploring ex post models as a way of understanding the causes of observed changes due to policy and to construct credible counterfactuals;
• Developing a user-friendly toolkit to help policymakers determine when and how an evaluation would benefit from agent based modelling
![Page 19: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/19.jpg)
Example of methodologies to explore
Involving citizens
The growth of crowd-funding, crowdsourcing and citizen science has shown the
opportunities for recruiting the public to assist with evaluation
Strengths
• Relatively low cost
• Large numbers of people with diverse expertise and interests can be involved
• Provides much greater quantity of data than would otherwise be available
• Increasing the transparency of the evaluation and associated policy intervention
Challenges
• Demands effective leadership
• Appropriate motivation to sustain contributions
• Support of technology to gather and organise the data
![Page 20: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/20.jpg)
Evaluation Policy and Practice Notes
Bayesian Updating for Contribution Claims
Qualitative Comparative Analysis (QCA)
Dependency Modelling
Agent Based Modelling
and more…
![Page 21: Integrating complexity into policy evaluation and enabling ... · and enabling better policy making: Introducing the Centre for the Evaluation of Complexity Across the Nexus (CECAN)](https://reader033.fdocuments.us/reader033/viewer/2022042418/5f3503a57682fe0e6b429fa1/html5/thumbnails/21.jpg)
Seminars, workshops and Fellowships
Monthly seminars in central London
Residential Workshops
One day courses
Fellowships
• Fellows – policy professionals,
academics, practitioners –work within the
Centre. Applications now open!