A new role for libraries in research assessments
-
Upload
wouter-gerritsma -
Category
Education
-
view
2.147 -
download
2
description
Transcript of A new role for libraries in research assessments
![Page 1: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/1.jpg)
A new role for libraries in research assessments
Wouter Gerritsma
Ticer International Summer School 2012
![Page 2: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/2.jpg)
Contents
Introduction
Research assessments
CRIS and repository
Advanced bibliometrics
Alternatives
Why in the library
How can you start?
![Page 3: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/3.jpg)
Introducing Wageningen UR Library
Wageningen University & Research Centre is the results of the merger between Wageningen University and the former DoA research institutes
Life Sciences is our domain
Some 6500 researchers and 10000 students
Strong international focus
Ranked 75th worldwide, 2nd in the Netherlands, 17th in Life Sciences ranking of THE WUR 2001-12 ranking
![Page 4: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/4.jpg)
Bibliometrics at Wageningen UR Library
Since the 1990s' few citation analysis with SciSearch on Dialog and DIMDI
2001: Web of Science
●Collection analysis
●First exercises with bibliometrics
2004: Essential Science Indicators
●Bibliometric analysis for graduate school WIAS
2008: Start of implementation of bibliometric services on our (metadata-)repository Wageningen Yield
![Page 5: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/5.jpg)
National research assessments elsewhere
Australia: The Excellence in Research for Australia (ERA) http://arc.gov.au/era
UK: Research Excellence Framework http://www.ref.ac.uk
●The pilot exercise concluded that citation information is not sufficiently robust to be used formulaically or as a primary indicator of quality in the REF; but there is scope for such data to inform and enhance the process of expert review.
![Page 6: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/6.jpg)
Research assessment in the Netherlands
Supervised by VSNU/QANU
●6 year cycle for external peer reviews
●After 3 year midterm review
●Unit of analysis (in Wageningen): Graduate schools
Citation analyses are not stipulated in the current Standard Evaluation Protocol (SEP).
●This has become mandatory at Wageningen UR, als at the social sciences department and for the research institutes
![Page 7: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/7.jpg)
SEP Criteria
Quality (including international academic reputation and PhD training)
Productivity (the relationship between input and output)
Societal relevance (including valorisation)
Vitality and feasibility (the ability to react adequately to important changes in the environment).
![Page 8: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/8.jpg)
Metis, our CRIS
Metis is a Current Research Information System (CRIS)
Data entry at chair group level
Quality control by the library
●Locating full text (uploading to e-depot)
●Maintenance journal lists
●Document type assignation
Compulsory output registration
●Research assessments only on metis registered pubications
Information on all labour relations of faculty and staff
Information on all projects
![Page 9: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/9.jpg)
Wageningen Yield
Wageningen Yield (WaY) is the bibliographic database of all output of Wageningen UR and OA repository
Comprehensive for the university since 1976, for the research institutes since 1996
Metadata records as well as full text
Ca. 180,000 metadata records
Ca. 37,000 full text OA publications
Since 2003 fed by metis, ca. 90,000 records
Synchronization each night with updates from metis
![Page 10: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/10.jpg)
Klik op het pictogram als u een afbeelding wilt toevoegen
![Page 11: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/11.jpg)
Collecting citation data from Web of Science
WoS record Identifier (UT)
Thomson Reuters Article Match Retrieval service http://wokinfo.com/directlinks/amrfaq/
API
●Input DOI or ISSN/Journal Title/Volume/Start Page
●Return: UT
●Input: UT
●Return: Times Cited
![Page 12: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/12.jpg)
Full screen image with title
Klik op het pictogram als u een afbeelding wilt toevoegen
![Page 13: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/13.jpg)
How are we able to compare numbers?
Scientist Z. Math has a publication from 2001 with 17 citations
Scientist M. Biology has a publication from 2007 with 32 citations
![Page 14: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/14.jpg)
Baselines for Mathematics
![Page 15: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/15.jpg)
Baselines for Molecular Biology
![Page 16: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/16.jpg)
A quantitative example
Bouma, J, Bulte, EH, & DP van Soest (2008) Trust, Trustworthiness and Cooperation: Social Capital and Community Resource Management. J. Env. Ec. & Mngmt 56(2)155-166.
●Cited 12 times
Journal of Environmental Economics and Management from journals menu in ESI:
●Economics & Business
Baseline data for Economics & Business (from ESI)
●Article from 2008: Average: 3.62 Citations; Top 10%: 9 citations; Top 1% 27 citations
RI = 12 / 3.62 = 3.31
![Page 17: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/17.jpg)
Advanced bibliometric indicators
Follow Moed (1995) as closely as possible; but.....
Web of Science is used for citation data
●We can’t make corrections for self citations
Essential Science Indicators for baseline data (World average, Top 10% and Top 1%)
●Limited number of research fields (22)
BUT:
We can determine the representativeness of the citation analysis!
![Page 18: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/18.jpg)
Representativeness
Publication type #Pubs
Refereed articles 324
Non-refereed articles 7
Books 1
Refereed book chapters 36
Non-refereed book chapters 13
PhD Theses 45
Conference papers 137
Total Academic Publications 563
![Page 19: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/19.jpg)
Representativeness
Publication type #Pubs WoS Repr.
Refereed articles 324 288 89%
Non-refereed articles 7
Google Scholar 1
Refereed book chapters 36
Non-refereed book chapters 13
PhD Theses 45
Conference papers 137
Total Academic Publications 563
![Page 20: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/20.jpg)
Slice and dice any way you want
![Page 21: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/21.jpg)
Standard citation reports are created for any set of publications
![Page 22: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/22.jpg)
Accountability is of utmost importance!
![Page 23: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/23.jpg)
Register once use many
![Page 24: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/24.jpg)
Register once use many
![Page 25: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/25.jpg)
Register once use many
![Page 26: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/26.jpg)
Register once use many
![Page 27: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/27.jpg)
Matching Way and WoS
WoS: 9577 articles
WaY: 10933 articles
Missing in Way: 807 articles
Missing from WoS: 1159 articles
1161 peer reviewed articles not in ISI journals
![Page 28: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/28.jpg)
Sources of citation data
Web of Science
Scopus
Google Scholar
Microsoft Academic
SciFinder; Psychinfo
Citeseer and other open access repositories
Various Altmetrics initiatives
![Page 29: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/29.jpg)
Differences in citations reflected in h-index
![Page 30: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/30.jpg)
Differences in citations reflected in h-index
![Page 31: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/31.jpg)
Differences in citations reflected in h-index
![Page 32: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/32.jpg)
Web of Science & Incites
Citation data (includes also citations from other databases on Wok)
API to download citation data
Baselines from ESI
"New" product InCites
![Page 33: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/33.jpg)
Scopus & Strata
Citation data obtainable through an API
Benchmarking with SciVal Strata, no API yet
Not yet fully developed
![Page 34: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/34.jpg)
Google Scholar
Give them a few more years
Coverage?
Ghost citations
Content duplication
Benchmarking?
![Page 35: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/35.jpg)
Altmetrics http://altmetrics.org/manifesto/
Quickly developing
●PLoS article level metrics
●ScienceCard
●Total-Impact
●Readermeter
●Microsoft Academic Search
●etc.
Wouters, P. & R. Costas (2012). Users, narcissism and control. Utrecht, NL: SURFfoundation. http://www.surffoundation.nl/en/publicaties/Pages/Users_narcissism_control.aspx.
![Page 36: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/36.jpg)
Why in the library?
Library is the functional manager of CRIS / repository because of wide experience with bibliographic metadata
Library manages contracts with publisher(s) of external databases that are being used
Library has experience in developing and maintaining large databases
Library has ample experience in searching complicated databases such as Web of Science
![Page 37: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/37.jpg)
Advantage of CRIS/repository combination
Improvements in publication lists, etc. recorded
Knowledge of, and experience with bibliometric analyses is better institutionalized
Clarity / transparency for researchers
Analysis of a single unit of the institute offers advantages for whole institute
Better understanding of our own researchers
●We know where they publish
●We know what they cite
●We know something about their impact
![Page 38: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/38.jpg)
Raising library awareness
Improvement of the (metadata) quality in the repository
Quality has lead to compulsory registration for research assessments
Presentations for research groups during the preparation for peer reviews
Presentations based on detailed studies of single groups
Library gives advice on elements for publication strategies for groups and individuals
●there is a huge demand for these workshops
![Page 39: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/39.jpg)
Closing the circle
With the coupling of publications with WoS
We have gained insight in the relation
●Research group – Researchers – Publications – Reference list
●It is feasible to assign journal usage at faculty level, or more detailed (chair groups)
Journal title Total AFSG ASG ESG Imares PSG SSG
NATURE 2584 511 341 753 93 989 59
PNAS 2467 787 325 166 20 1225 29
SCIENCE 2303 529 239 594 52 970 99
APPLIED AND ENVIRONMENTAL MICROBIOLOGY 2257 1320 257 139 12 696 27
PLANT PHYSIOLOGY 1597 379 4 58 0 1296 2
JOURNAL OF BIOLOGICAL CHEMISTRY 1543 931 223 13 6 379 8
![Page 40: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/40.jpg)
But where should you start?
Courses on bibliometrics
●CWTS http://www.socialsciences.leiden.edu/cwts/
●ESSS http://www.scientometrics-school.eu/
Conferences
●ISSI http://www.issi-society.info/aboutconf.html
●STI http://2012.sticonference.org/
Journals
●Scientometrics
●Journal of Informetrics
Books
●Moed, H.F. (2005). Citation analysis in research evaluation. Dordrecht, Springer.
![Page 41: A new role for libraries in research assessments](https://reader037.fdocuments.us/reader037/viewer/2022110114/546fcfb4af79591b498b4839/html5/thumbnails/41.jpg)
Thank you!
On the Web:@wowterwowter.netwww.slideshare.net/wowter
http://tinyurl.com/7r67fmm