Output and citation impact of interdisciplinary networks: Experiences from a dedicated funding...
1
11th International Conference on Science and Technology Indicators Creating Value for Users ▬ Leiden ▬ 8-11 September 2010 Output and citation impact of interdisciplinary networks: Experiences from a dedicated funding program Nadine Rons, [email protected] Research Coordination Unit, Research & Development Department, Vrije Universiteit Brussel (VUB), B-1050 Brussels, Belgium Introduction In a context of ever more specialized scientists, interdisciplinarity receives increasing attention as innovating ideas are often situated where the disciplines meet. In many countries science policy makers installed dedicated funding programs and policies. This induces a need for specific tools for their support. There is however not yet a generally accepted quantitative method or set of criteria to recognize and evaluate interdisciplinary research outputs. Interdisciplinarity also takes on very different forms, as distinguished in overviews from the first codifications (Klein, 1990) to the latest reference work (Frodeman et al., 2010). In the specific context of research measurement and evaluation, interdisciplinarity was discussed e.g. by Rinia (2007) and Porter et al. (2006). The present empirical study aims to contribute to the understanding and the measuring of interdisciplinary research at the micro level, in the form of new synergies between teams from different disciplines. Investigation of a specialized funding program shows how a new interdisciplinary synergy and its citation impact are visible in co-publications and co-citations, and that these are important parameters for assessment. The results also demonstrate the effect of funding, which is clearly present only after about three years. ‘Horizontal Research Actions’ Program Set up at the Vrije Universiteit Brussel in 2002. Collaborations joining expertise from different disciplines on topics proposed by the applicants. Two-phased selection procedure involving the Board of the Research Council as a selection committee and oral presentations of pre-selected projects (Rons, 2010). Funding spent primarily on researchers embodying the link between the disciplines. Project duration initially 2 years, in the majority of cases extended until 4 years. Program Evaluation Evaluation started end 2009, when the first four generations of applications could be followed for three years after start of funding. Author-centered approach, based on the applicants and their affiliated departments. Key Figures 4 generations of applications, per application on average 4 applicants from 3 departments. 36 applications including 3 resubmissions and 4 applications completely situated in Social Sciences and Humanities 29 distinct networks analyzed. 12 funded applications. 17 unfunded applications. Bibliometric Study Co-publications: defined as joint publications by applicants from different departments, monitored as an indicator of interdisciplinary output (a similar measure was used by Bordons et al., 1999). Co-citations: defined as publications citing applicants from different departments, monitored as an indicator for citation impact (obtained by combining the results of ‘Cited Reference Search’ operations performed per applicant). Bibliometric data source: on line Web of Science (SCIE, SSCI, AHCI and Proceedings indices). Applications completely situated in Social Sciences and Humanities (4 out of 36) excluded due to insufficient coverage. ! Absolute numbers of co-publications and co- citations should be used with caution, not being normalized for the interdisciplinary research areas. Survey Directed to the five earliest funded networks that could be followed for at least 4 full years after start of funding at the time of the survey. Seeking confirmation of the validity and adequacy of the data and indicators used in the bibliometric study, as well as the applicants’ opinions regarding stimulation and support of interdisciplinary research. Evaluation Results Output & Citation Impact The majority of funded applications (9/12) successfully generated both co-publications and co- citations, while a minority showed neither (2/12). About half of the unfunded applications (8/17) showed neither co-publications, nor co-citations. Despite not being funded, an important part of the unfunded applications (7/17) did generate co- publications and co-citations. Effect of Funding SUBSET : 8 newly activated networks (i.e. co- publications not yet present before application) that could be followed for at least 4 full years after start of funding: 4 networks funded for 4 years, and 4 unfunded networks The effect of funding is visible in more strongly rising co-publications and co-citations from the third year (Figure 1) Figure 1: Co-publications by and co-citations to partners of newly activated interdisciplinary networks Networks funded for 4 years (white) and unfunded networks (gray) Dashed border = with partner in social sciences and humanities 73% of co-citations are citing co-publications 13% of co-citations are itself co-publications Validation From a survey of the five earliest funded networks, all newly activated. The large majority of the co-publications represent a synergy of expertise related to the topic (23/28). The other co-publications concern rather an application of results from one discipline in another (2/28) or are not related to the topic (3/28). Overall the majority of the co-citations is related to the topic (83/130), with considerable differences between networks. The co-publications and co-citations traced in the Web of Science contain the majority of interdisciplinary output and impact items generated by the networks in relation to their topics (next to other contributions in proceedings, in Dutch, PhD’s and obtained funding). Conclusions The survey indicates that co-publications and co- citations are good representatives of output and impact of interdisciplinary synergies, and important parameters for their assessment. The presence of co-publications and co-citations can be used to indicate a program’s success in attracting, selecting and activating promising applications for interdisciplinary synergies. New interdisciplinary synergies on a certain topic may in addition lead to interdisciplinary combinations of knowledge on another topic, in or outside of the initial network. The added value of a funding program for new interdisciplinary synergies in terms of output and impact may become visible only after a sufficient number of applications could be followed for 3 or more years. References Bordons, M., Zulueta, M.A., Romero, F. and Barrigón, S. (1999). Measuring interdisciplinary collaboration within a university: the effects of the Multidisciplinary Research Programme. Scientometrics, 46(3):383-398. Frodeman, R., Klein, J.T. and Mitcham, C. (Eds.) (2010). The Oxford Handbook of Interdisciplinarity. Oxford: Oxford University Press. Klein, J.T. (Ed.) (1990). Interdisciplinarity: History, Theory, and Practice. Detroit: Wayne State University Press. Porter, A.L., Roessner, J.D., Cohen, A.S. and Perreault, M. (2006). Interdisciplinary research: meaning, metrics and nurture. Research Evaluation, 15 (3), 187–195. Rinia, E.J. (2007). Measurement and Evaluation of Interdisciplinary Research and Knowledge Transfer. Doctoral thesis, Leiden University. Rons, N. (2010, forthcoming). Interdisciplinary Research Collaborations: Evaluation of a Funding Program. In: Proceedings of the Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) and the Eleventh COLLNET meeting, Mysore, India, October 19-22, 2010.
-
Upload
nadine-rons -
Category
Education
-
view
36 -
download
0
Transcript of Output and citation impact of interdisciplinary networks: Experiences from a dedicated funding...
11th International Conference on Science and Technology Indicators Creating Value for Users ▬ Leiden ▬ 8-11 September 2010
Output and citation impact of interdisciplinary networks: Experiences from a dedicated funding program
Nadine Rons, [email protected] Research Coordination Unit, Research & Development Department, Vrije Universiteit Brussel (VUB), B-1050 Brussels, Belgium
Introduction
In a context of ever more specialized scientists, interdisciplinarity receives increasing attention as innovating ideas are often situated where the disciplines meet. In many countries science policy makers installed dedicated funding programs and policies. This induces a need for specific tools for their support. There is however not yet a generally accepted quantitative method or set of criteria to recognize and evaluate interdisciplinary research outputs. Interdisciplinarity also takes on very different forms, as distinguished in overviews from the first codifications (Klein, 1990) to the latest reference work (Frodeman et al., 2010). In the specific context of research measurement and evaluation, interdisciplinarity was discussed e.g. by Rinia (2007) and Porter et al. (2006). The present empirical study aims to contribute to the understanding and the measuring of interdisciplinary research at the micro level, in the form of new synergies between teams from different disciplines. Investigation of a specialized funding program shows how a new interdisciplinary synergy and its citation impact are visible in co-publications and co-citations, and that these are important parameters for assessment. The results also demonstrate the effect of funding, which is clearly present only after about three years.
‘Horizontal Research Actions’ Program
Set up at the Vrije Universiteit Brussel in 2002. Collaborations joining expertise from different
disciplines on topics proposed by the applicants. Two-phased selection procedure involving the
Board of the Research Council as a selection committee and oral presentations of pre-selected projects (Rons, 2010).
Funding spent primarily on researchers embodying the link between the disciplines.
Project duration initially 2 years, in the majority of cases extended until 4 years.
Program Evaluation
Evaluation started end 2009, when the first four generations of applications could be followed for three years after start of funding.
Author-centered approach, based on the applicants and their affiliated departments.
Key Figures
4 generations of applications, per application on average 4 applicants from 3 departments.
36 applications including 3 resubmissions and 4 applications completely situated in Social Sciences and Humanities 29 distinct networks analyzed.
12 funded applications. 17 unfunded applications.
Bibliometric Study
Co-publications: defined as joint publications by applicants from different departments, monitored as an indicator of interdisciplinary output (a similar measure was used by Bordons et al., 1999).
Co-citations: defined as publications citing applicants from different departments, monitored as an indicator for citation impact (obtained by combining the results of ‘Cited Reference Search’ operations performed per applicant).
Bibliometric data source: on line Web of Science (SCIE, SSCI, AHCI and Proceedings indices).
Applications completely situated in Social Sciences and Humanities (4 out of 36) excluded due to insufficient coverage.
! Absolute numbers of co-publications and co-citations should be used with caution, not being normalized for the interdisciplinary research areas.
Survey
Directed to the five earliest funded networks that could be followed for at least 4 full years after start of funding at the time of the survey.
Seeking confirmation of the validity and adequacy of the data and indicators used in the bibliometric study, as well as the applicants’ opinions regarding stimulation and support of interdisciplinary research.
Evaluation Results
Output & Citation Impact
The majority of funded applications (9/12) successfully generated both co-publications and co-citations, while a minority showed neither (2/12).
About half of the unfunded applications (8/17) showed neither co-publications, nor co-citations.
Despite not being funded, an important part of the unfunded applications (7/17) did generate co-publications and co-citations.
Effect of Funding
SUBSET: 8 newly activated networks (i.e. co-publications not yet present before application) that could be followed for at least 4 full years after start of funding: 4 networks funded for 4 years, and 4 unfunded networks
The effect of funding is visible in more strongly rising co-publications and co-citations from the third year (Figure 1)
Figure 1: Co-publications by and co-citations to partners of newly activated interdisciplinary
networks
Networks funded for 4 years (white) and unfunded networks (gray)
Dashed border = with partner in social sciences and humanities
73% of co-citations are citing co-publications 13% of co-citations are itself co-publications
Validation
From a survey of the five earliest funded networks, all newly activated.
The large majority of the co-publications represent a synergy of expertise related to the topic (23/28). The other co-publications concern rather an application of results from one discipline in another (2/28) or are not related to the topic (3/28).
Overall the majority of the co-citations is related to the topic (83/130), with considerable differences between networks.
The co-publications and co-citations traced in the Web of Science contain the majority of interdisciplinary output and impact items generated by the networks in relation to their topics (next to other contributions in proceedings, in Dutch, PhD’s and obtained funding).
Conclusions
The survey indicates that co-publications and co-citations are good representatives of output and impact of interdisciplinary synergies, and important parameters for their assessment.
The presence of co-publications and co-citations can be used to indicate a program’s success in attracting, selecting and activating promising applications for interdisciplinary synergies.
New interdisciplinary synergies on a certain topic may in addition lead to interdisciplinary combinations of knowledge on another topic, in or outside of the initial network.
The added value of a funding program for new interdisciplinary synergies in terms of output and impact may become visible only after a sufficient number of applications could be followed for 3 or more years.
References
Bordons, M., Zulueta, M.A., Romero, F. and Barrigón, S. (1999). Measuring interdisciplinary collaboration within a university: the effects of the Multidisciplinary Research Programme. Scientometrics, 46(3):383-398.
Frodeman, R., Klein, J.T. and Mitcham, C. (Eds.) (2010). The Oxford Handbook of Interdisciplinarity. Oxford: Oxford University Press.
Klein, J.T. (Ed.) (1990). Interdisciplinarity: History, Theory, and Practice. Detroit: Wayne State University Press.
Porter, A.L., Roessner, J.D., Cohen, A.S. and Perreault, M. (2006). Interdisciplinary research: meaning, metrics and nurture. Research Evaluation, 15 (3), 187–195.
Rinia, E.J. (2007). Measurement and Evaluation of Interdisciplinary Research and Knowledge Transfer. Doctoral thesis, Leiden University.
Rons, N. (2010, forthcoming). Interdisciplinary Research Collaborations: Evaluation of a Funding Program. In: Proceedings of the Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) and the Eleventh COLLNET meeting, Mysore, India, October 19-22, 2010.
