Public Value Innovation and Research Evaluation

Discussion by Karen MacoursINRA - Paris School of Economics

Economic models for assessing innovation

• Economic models can and have been used to assess impacts on many types of social outcomes, not just economic growth

• Even when theory might have it’s limits (complexity of incorporating certain social and distributional values), economic and econometric tools are well suited to help evaluate changes in social outcomes– E.g. work on “non-economic” outcomes : crime, on self-

commitment problems, on aspirations, …– E.g. work on distributional consequences (heterogeneity

of impacts) of new technologies

Public failure versus market failure

• Contrast seems to rely on narrow definition of market?

• E.g. unregulated markets that fail to account for externalities and distributional consequences: is indeed a market failure even in economic models.

Public value mapping• We learned:– To provide a social theory (i.e. public value theory)

basis for research evaluation– To connect assessments of research outputs and first

order impacts (e.g. RVM) with broad social impacts, both anticipated and unanticipated (PVM)

• Can provide a framework to guide evaluations of specific science policies (support) and their outcomes, including evaluations focused on “ends” (not means)

A bit more concretely• Let’s say we want to evaluate the value of agronomical

research• We might then to consider the social impacts of

technological innovations resulting from that research• Off course there is the difficulties with building blocks

versus final innovations, dynamic issues, scale, etc… • But no matter how complicated the larger picture: it

seems quite crucial to demonstrate the social impacts of the innovations

• Not necessarily narrowly in terms of productivity (though that again would be a good preliminary step), but in terms of welfare, food security, health outcomes, distribution of cost and benefits

Learning from development economics?

• Quantitative impact evaluations using RCTs or quasi-experimental methods, as practiced by development and labor economists provide an approach to look at these type of questions

• One’s a certain product of science (a technological innovation) exists, we can randomly allocate it among the targeted population to evaluate it’s benefits

• In medicine they figured this out a long time ago off course• But we add a toolbox that accounts for externalities

(spillovers) and for considering wider social impacts• Through careful experimental design and well-designed data

collection, based on a priori “theoretical” evaluation of potential outcomes (positive, negative and at different levels)

Some examples• Impact of access to improved seed varieties (DRC) on:

– Poor household’s food security and nutrition – Their investments in education and health – Social learning and changes in intra-village equalities– Intra-household dynamics and women empowerment– Aspirations and attitudes towards the future – Etc, etc…

• Impact of a new “vertical supply technology” (Cambodia) on :– Adoption of healthier practices, – Formation of new contract types,– Distribution of benefits (directly and indirectly) among different

villagers (poor and less poor), – Village social cohesion and trust, – Etc, etc…

Some concluding thoughts• Public value mapping very useful as overall framework • Economic tools and impact evaluations can be used to

provide strong causal evidence of key building blocks in that framework

• Important to do, as without evidence of the products of the investments in science, it’s hard to make a broader argument about the value of different types of science investments

• Such evaluations need to be done in real world context (go beyond the lab), exactly in order to shed light on social outcomes and broad welfare implications

• By testing for mechanisms underlying distribution of benefits (e.g. limits to adoption by certain type of farmers) can aim to improve social outcomes