Economic Impact Evaluation of the Economic and Social Data Service of UK

download Economic Impact Evaluation of the Economic and Social Data Service of UK

of 97

Embed Size (px)

description

In its attempt to evaluate the quality and impact of its research and data infrastructure investments the Economic and Social Research Council (ESRC) assigned to Charles Beagrie the assessment of the Economic and Social Data Service (ESDS) [annual operating budget £3.3 million; 23,000 active users]. This report provides an overview of methods and approaches explored in the study, issues encountered and our findings and recommendations.

Transcript of Economic Impact Evaluation of the Economic and Social Data Service of UK

  • 1. Economic Impact Evaluationof theEconomic and Social Data ServiceCharles Beagrie LtdandThe Centre for Strategic Economic Studies (CSES)University of VictoriaMarch 2012

2. 2ContentsExecutive Summary ................................................................................................................................... 41. Introduction ........................................................................................................................................ 91.1. Background ................................................................................................................................ 91.2. Aims and Objectives ................................................................................................................. 91.3. The ESDS .................................................................................................................................. 101.4. Conceptual Approaches in the Study ................................................................................. 122. Methods Used to Collect and Analyse Data ............................................................................. 142.1. Desk Research ......................................................................................................................... 142.2. Interviews ................................................................................................................................. 142.3. Online Surveys ......................................................................................................................... 153. Methods and Approaches used to Assess Economic Impact ................................................ 174. Economic Analysis ........................................................................................................................... 214.1. Weighting the Data ................................................................................................................ 214.2. Data Analysis ............................................................................................................................ 244.3. Investment and Use Value ..................................................................................................... 244.4. Contingent Value .................................................................................................................... 274.5. Consumer Surplus .................................................................................................................. 294.6. Net Economic Value ............................................................................................................... 294.7. Potential Efficiency Impact .................................................................................................... 304.8. Return on Investment in Social Science Data Infrastructure ........................................ 314.9. Summary of Economic Analysis ........................................................................................... 335. Illustrating Wider Benefits ............................................................................................................. 365.1. Benefits Identified in Desk Research .................................................................................. 365.2. Benefits identified in the Interviews .................................................................................... 375.3. Benefits Identified in the User Survey ................................................................................ 425.4. Benefits Identified in the Depositor Survey ...................................................................... 435.5. Summary of Benefits Analysis ............................................................................................... 436. The Case Studies ............................................................................................................................. 456.1. Case Study 1: E3MG: An Energy-Environment-Economy (E3) Model ........................ 456.2. Case Study 2: Knife Crime: A Review of Evidence and Policy ...................................... 476.3. Case study 3: Trends in Obesity ......................................................................................... 497. Conclusions ...................................................................................................................................... 517.1. Economic Impact and Benefits of the ESDS ...................................................................... 517.2. Economic Impact and Benefits of Social Science Research Data Service Infrastructure in General ................................................................................................................... 517.3. Effectiveness of the Evaluation Methodology and Approaches..................................... 528. Summary of Recommendations ................................................................................................... 568.1. Good Practice/Lessons for Maximising the Benefits of Research Data Service Infrastructure ........................................................................................................................................ 568.2. Approaches to Future Economic Impact Evaluations ..................................................... 56References ................................................................................................................................................. 58Appendix A: Findings from the Surveys .............................................................................................. 64Appendix B: Summary of Interviews ................................................................................................... 87 3. 3AcknowledgementsWe would like to thank: Peter Williams and Anna Palaiologk for their input as researchers to the study; Matthew Woollard, Lucy Bell and colleagues at the UK Data Archive for their collaboration and support, together with Keith Cole and colleagues at MIMAS and Vanessa Higgins, Angela Dale and colleagues and students at CCSR. Margaret Macadam at ESRC provided feedback and guidance throughout. Finally, our thanks go to all the interviewees and survey respondents who gave valuable time and input to the study.Neil Beagrie (Charles Beagrie Ltd) and John Houghton (CSES)March 2012 4. 4Executive SummaryBackgroundThe Economic and Social Research Council (ESRC) evaluates the quality and impact of its research and data infrastructure investments. The Councils Evaluation Committee operates a well-developed system for evaluating academic quality and impact, and in recent years has developed qualitative and quantitative approaches to assessing policy and practice impacts of the research projects it funds and for establishing its broader contribution to the economic and social well-being of the UK.Through this study, the ESRC is now looking to assess the economic benefits of its investment in the Economic and Social Data Service (ESDS). The ESDS is a distributed service based on a partnership between the Universities of Essex and Manchester, and is co- funded by ESRC and JISC. The annual operating budget for ESDS is around 3.3 million. There are around 23,000 active users of the ESDS, including users from outside the academic community. The overall aim of the ESDS is to promote wider and more informed use of data for research and teaching in the social sciences and to ensure that these data continue to be available over time.This report provides an overview of methods and approaches explored in the study, issues encountered and our findings and recommendations.Aims and terms of referenceThe aims of this study are to: (i) evaluate the economic benefits and impact of ESDS; and (ii) contribute to the further development of impact evaluation methods that can provide ESRC with robust estimates of the economic benefits of its data service infrastructure investments. This includes ESDSs economic impact on the creators and users of the data it provides access to. The study does not attempt to quantify policy impacts but it does include case studies to illustrate the broader context and provide understanding of the breadth of both quantifiable and qualitative benefits including areas such as policy.The terms of reference for this evaluation are to identify and assess the economic impact of the ESDS by: Applying economic valuation techniques where appropriate to derive a quantitative estimate of the economic benefits of the services and data provided by the ESDS; Presenting the results of the economic assessment within the context of a broader qualitative analysis of the benefits of the ESDS; Conducting case studies of the impact generated by research based on data accessed through the ESDS; Reflecting on the results and identifying best practice and lessons for:o maximising the benefits from research data infrastructure investments; ando developing approaches for future economic impact evaluations. 5. 5Economic impact of ESDSOur economic analysis included a range of approaches, starting with the most immediate and direct measures of value that are likely to represent lower bound estimates of the value of ESDS data and services and moving outwards to estimates of the wider economic benefits. They included: Investment and use value - with the amount of time and money spent producing/ obtaining ESDS data and services indicating the minimum value of ESDS; Contingent value - with the amount that users would be willing to pay to access ESDS data and services and/or willing to accept to forego access indicating the value of ESDS to them; Consumer surplus - with the total willingness to pay minus the cost of obtaining indicating the benefit they derive from ESDS; Net economic value - with the users' benefits derived minus the cost of providing ESDS data and services indicating the net