of 19

Embed Size (px)


EVALUATION OF SOCIAL IMPACT productive interactions in health (policy) research


  • 1. EVALUATION OF SOCIAL IMPACTproductive interactions in health (policy) researchHTAi conference Bilbao, 27-06-2012Jack Spaapen

2. 2Three questions (SIAMPI project)Q1. What is social impact? - Conceptual and practical problemsQ2. What are productive interactions? And what do they have to do with social impact?Q3. How to evaluate social impact: PI, intermediate impacts, social impact 3. 3Q1. WHAT IS SOCIAL IMPACT- Conceptual problems - Linear models vs network approaches - Temporality - Attribution / contribution - Overlap between various impacts - Positive / negative impacts- Practical problems - Reward systems - Indicators - Data collection 4. 4where is social impact? 5. 5research is contextualizedTopsector Health / Life sciences (Netherlands)New Dutch Science Policy Top sectors: energy, water,food/agriculture, health/life sciences, creative industry, Healthy aging (EU)Creative industry, new media, (serious) gamesNanotechnologyGreen meat production in 2020 in NLWater management (New Orleans)Social impact is the outcome of a joint effort in a network of relations: research, industry,government, NGOs, consumer-organisations public private partnershipsWhich data are relevant? Activities in the network coalitions, goals, interactions,intermediate results 6. 6New coalitions for Research, development and innovation- Triple helix, golden triangle: research, industry, society (government, NGOs, generalpublic), new collaborative arrangements: PPP (transdisciplinary research,transepistemic communities Knorr-Cetina, science 3.0 - Miedema)- Interdisciplinary input from research: natural science/technical fields, social sciences,humanities (technical knowledge, content, use/behavior)- But also input from other expertise: politics, law, economists, ethics, consumer-interests, etc.- Co-creation of new knowledge and practical solutions = innovation, research bydesign, iterative process (non-linear) -- Consensus about long term goals (healthy aging, clean energy), but in the meantimeshifting coalitions, different partners, different intermediate goals, different interests- Problems arise from not being used to work together (academics industry, notknowing what to formulate as research questions), institutional problems, political andcultural problems. 7. 7RESEARCH, DEVELOPMENT and INNOVATION (RDI)AS A LONG TERM NETWORK EFFORTThe network consists of a variety ofstakeholders working on a commonproblem; but goals and people shiftEverybody produces knowledge, everybody does research:transdisciplinary collaborationThe result is to be socially robust knowledge : scientifically reliable, socially valuable 8. 8 Q2. PRODUCTIVE INTERACTIONSWhat are productive interactions?- Relations in a network that produce something of value- Not only of economic value (knowledge-new technology-cash value)- Also socio-cultural, ethical, intellectual, technological, environmentalAnd what do they have to do with social impact?- They show what it takes to achieve impact- Guideline for indicators of intermediate impact 9. 9SIAMPI: productive interactions between science and societynetwork oriented Personal interactions : joint projects, advisory, consultancy, doublefunctions, mobility Media interactions : Texts : articles, books, catalogues, protocols, new diagnostics Artifacts : instruments, exhibitions, models, designs Support: contracts, subsidies, patenting, licensing, sharing of people andfacilities Instances of social impact 10. 10DATA AND INDICATORS: FOCUS ON INTERACTION AND INTERMEDIATE OUTCOME representing productive interactions between a variety of stakeholders = research, policy, industry, society Data collection: (i) personal interactions (i) interaction thru media (iii) financial or material support (iv) intermediate output (v) indications of social impact (vi) new products, procedures, etc 11. 11Q3. HOW TO EVALUATECONCEPTUAL [self evaluation reports] Mission orientation Network analysis (thru texts, people, organisations) Stakeholder approach (involvement from the start)PRACTICAL Network indicators - indications Intermediate indicators 3-5 years Impact indicators instances of impact 12. 12 HEALTH CASE: NIVEL and LUMCNIVEL public/private institute for health policy and primary health careLUMC academic department (public health, gynaecology, anatomy/embryology)NIVEL: top down organisation of network activities, stakeholder contacts actively organised to safeguard financial support and to enhance chance of implementation of results, include all types of stakeholdersLUMC: bottom up, incentive structure based on WoS, but on the movefrom traditional academic department to contextual research (topsector policy for life sciences and health research) 13. 13EXAMPLE OF CASE STUDIESSocial impacts related to research at NIVEL or LUMC are often too intricate to be attributed directly to or identified with specific groups.LUMC: Anatomy, stem cell projects. Results / impacts are only to be expected inthe very long run (10-15 years)In this case, we did see some impact in clinical practice of a neighbouring field,cardiology.NIVEL: the result of one project in Public Health pointed out that currently popular health centers specifically for elderly only raised health care demand but did not improve the health situation of elderly.Impacts (i) can be positive or negative; and (ii) take shape in various contexts: the academic group (also other groups!), the hospital, the policy domain (stem cells!) 14. 14 Type of Example Effects (success) interaction DirectConsultation rounds with stakeholders Adaptation in research agenda User groups, supervising boards (PPPAdaptation of research projects consortia) Presentations to health care professionals Sustaining relations with (conferences, meetings, post-academicstakeholders, knowledge transfer teaching) Collaboration in research Mutual adaptation in research projects IndirectAnnual plansMutual agreements with funding agencies over future research agenda Implementation plansMutual agreements with funding agency over future implementation of project results Reports & medical guidelines, Knowledge transfer, Response scientific publications (uptake of knowledge) by a wider variety of stakeholders than those involved in direct interactions Financial Contracts, Licences, Project grants Enabling completion of research projects Lump sum grants Enabling independent research 15. 15SOCIAL IMPACT OF NIVEL: reception of reports per domainAD PRINSGoogle searches for reports / PDFs of 8 NIVEL domains 16. 16 SOCIAL IMPACT OF LUMC: MEDICAL GUIDELINES FOR GPS AD PRINSGoogle searches for three guidelines of LUMC Dep. of Public Health and General Practice 17. Table 1: SIAMPI indicators for productive interactionspersonal interactionsinteraction betweenFinancial / materialbetween stakeholders stakeholders through media interaction betweenstakeholdersface-to-face meetings academic journals research contracts, public anddouble functions, other professional journals private, and mixed, national,mobility arrangementsnon academic journals internationalphone conferences popular media facility, instruments sharingemail exhibitions start upssocial mediaartefacts, models contribution in kind (people)videoconferencing films IPR arrangements, patents,public debate master theses, graduate licensesradio, tv, internet projects Professional trainingetc.standards, protocolsOther stakeholder interest social mediaetc. etc. 18. 18What is necessary to evaluate research in context o A network perspective, mutual learning, extended review (more than peers), new reward systems, administrators that dare to care o Development and use of new type of criteria and indicators that focus on process and intermediate results o Continuous feed back between stakeholders, impact through learning! o Use / acceptance of these methods at all levels of research (institute, national, EU) 19. 19Researchnano ictHealth caresshdomainCountry NL, France UK, NL, EU NL Esp, UKResearch type Frontier, basic, Basic, applied, TD strategic, applied,Basic, strategic,strategic policy appliedResearch mode Academic, in Open to partnership of Academic, open toAcademic, open tocollaboration with knowledge producerscollaboration with collaboration withindustry and usersindustry, govern-policy, institutions,ment, patientwider public,groups, professio- industrynalsProductivePublic Transport use, Consultation, colla- Informal links andInteractionsunderstanding, security, interactionboration, regula-advice, formalethical debates, between citizens and tions, proto-cols, research contract and collaborativepolicy making, government commercial projects,productsexchanges, PPPs, consultancy,post academiccultural eventstraining, patientorganizationsSocial Impact Health, safety,Transport use, Diagnostics, Policy tools andpublic security, interactiontreatments, safety,techniques,acceptance ofbetween citizens and general health,managementnano techgovernment policy advicemethods, cultural goods and services