Research Information and Analytics at Cambridge Insight over measurement Research Information...
-
Upload
blaise-joshua-davis -
Category
Documents
-
view
215 -
download
0
Transcript of Research Information and Analytics at Cambridge Insight over measurement Research Information...
Research Information and Analytics at CambridgeInsight over measurement
Research Information Office, Academic Division
Dr Owen RobersonResearch Information [email protected]@orobersonORCiD: 0000-0001-6938-5162
Introduction – my environment
Research Information
team
Research Office
Grant reporting(e.g. Researchfish)
HESA (return on statistics)
REF return
HRCRISGrantsDSpace
Academics
Administrators/Co-ordinators
University Library
Senior Academic Committees
Research Strategy Office
Research Information Landscape
• What is Research Information?
• Publications
• Grants
• Conferences
• Patents, software, compositions, performances, exhibitions…amongst others.
Not BIG DATA but WIDE DATA.
What it’s like being a researcher
I like doing research!
Are you doing anything interdisciplinary?
What were your recent publications?
Where did you publish them?
Is the data open too?
How did you pay the APC?
Where is your Research heading?
Are they Open Access?
Was it well received?
Did it have any impact?
How many grants have you applied for so far?
How many grant applications were successful?
How are you going to sustain your grant income profile?
Where is your grant money?
What did you do with the grant income you received?How did you cost your overheads?
Does it have an international dimension?
Did it have any Impact?
How many students are you teaching?How many students are you supervising?
Did it make a spinoff?
Does it have any IMPACT?
What’s your next project?
WILL IT HAVE ANY IMPACT?
So what makes a good analytics approach?
You can generate statistics on all of these things, but be cautious of the tail wagging the dog.
• What is your goal as an analytics team?
• Once you have an analysis, what can you actually do? (How many carrots and sticks do you have?)
• What are the implications of using the sticks?
What can we do, What should we do?
What Research Information is NOT, and the perils of metricide
• What is a KPI in the context of Research?
• How sure are you that you’re measuring what you think you’re measuring?
• Are you presenting it fairly?
• If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?
What Research Information is NOT, and the perils of metricide
• What is a KPI in the context of Research?
• How sure are you that you’re measuring what you think you’re measuring?
• Are you presenting it fairly?
• If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?
What is a KPI in the context of research?
There are none that work.
This is a game where the rules and goals change continuously.
•Donald Schon’s Reith Lecture: “The Loss of the Stable State”, as part of the series Change and Industrial Society, 1970. http://www.bbc.co.uk/programmes/p00h65rn#play
What Research Information is NOT, and the perils of metricide
• What is a KPI in the context of Research?
• How sure are you that you’re measuring what you think you’re measuring?
• Are you presenting it fairly?
• If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?
How sure are you that you’re measuring what you think you’re measuring?
• Impactful
• REFability
• EPSRC CompliancyHTTP://XKCD.COM/552/
What Research Information is NOT, and the perils of metricide
• What is a KPI in the context of Research?
• How sure are you that you’re measuring what you think you’re measuring?
• Are you presenting it fairly?
• If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?
What Research Information is NOT, and the perils of metricide
http://figshare.com/articles/What_aren_t_you_seeing_Advance_in_visualisation_reserach_/1464961
Once you’ve got a metric, are you presenting it fairly?
What Research Information is NOT, and the perils of metricide
• What is a KPI in the context of Research?
• How does calculating something inappropriate affect the conversation?
• How sure are you that you’re measuring what you think you’re measuring?
• Are you presenting it fairly?
• If a measure is biased, are there ways of creating measures that counterbalance the bias, and if not, are you asking the right question?
Scientocentricity, alienation, and bad incentives
HTTP://XKCD.COM/435/
Also…
NEVER USE JOURNAL IMPACT FACTORS
If you’re going to do it, do it well
Research Information must:•Provide insight•Be explained in its context•Be aware of its limitations
•Good places to start:1
• Have a statement about your position re DORA• Be aware that the watchmen are being watched
too..
____1. Thanks to Jane Tinkler, whose original presentation can be found here.
Making it work: enter once, use multiple times
HTTP://XKCD.COM/927/
Research Information Systems @ Cambridge
• With apologies to Harry Beck….
Core I: Symplectic meets institutional data
Core II: Further mapping of internal data to Symplectic
Core III: REF – Research Assessment
Post REF: Altmetrics and ORCID
Post REF: Reporting tools, CV & biosketch generation
Closing the gap: Open Access
Bridging the gap: Dimensions & rich grant meta data
From Integration to Innovation – VIVO and network analysis
And what all this means…
Social Media Impact and Comms Strategy
Publication Strategy
HEFCE ComplianceGrant Capture, RCUK
Compliance
Academic Workload
Genuinely new and useful information
researchers want to hear
Network Analysis: finding connections
This shows collaborations between REF2014 UoAs; indicating that interdisciplinary research is already a key
element of research practice at Cambridge.
It is possible to explore the connections between members of a defined group and consider not only the direct connections
between them (through collaboration or publication), but who their collaborators collaborate with, and what other connections
they may have made.
Colours indicate University departments. Points or nodes represent researchers. Connections or edges
represent an instance of co-authorship.
Roberson, Owen; Wastl, Juergen (2015): Beyond Serendipity: Networks of Research at Cambridge. figshare.http://dx.doi.org/10.6084/m9.figshare.1431829
Glossary
Altmetric.com – Provider of article-level metrics focussing mainly on social media platforms, reliant on DOIs in the main, to track articles through internet sources. Also covers Wikipedia entries and some policy documents. Digital Science portfolio.
CASRAI – Not-for-profit seeking to reduce the administrative burden on academics, by supporting single entry, multiple use of data.
Dimensions – Provider of publically available grants data, including abstracts and semantic searches. Medical and US/UK focussed at the moment, but growing constantly. Digital Science portfolio.
DOI – Digital Object Identifier – unique string used to identify an online object. Operates like a URL, and is public. Mainly applied to articles in the research space, use will probably grow in the near future.
DORA – San Francisco Declaration on Research Assessment: an open declaration from researchers and institutions that commits its signatories to the improvement of research assessment metrics and the avoidance of Journal Level Metrics.
ORCiD – a unique identifier for a researcher that operates like a DOI for an individual.
ResearchFish – the Company and software used by a number of funders, but crucially by RCUK, to track research outcomes for the grants its participants have funded. Mandatory annual returns. Database uses the grant as its principal component.
RIMS – Research Information Management System – a database containing metadata about publications. Key offerings in the field are Symplectic Elements, PURE and ePrints, as well as a host of institution-specific, home-grown solutions. Most solutions use the academic as the principal component.
Symplectic – Company that produces the Elements system, the RIMS that we use at Cambridge. Digital Science portfolio.
VIVO – Open Source ontology of researchers and their research outputs originally developed at Cornell. Can be driven by a RIMS, used by Melbourne for “Find the Expert”.
Further Reading and Links:
https://responsiblemetrics.org/about/
http://orcid.org/
http://www.vivoweb.org/
http://casrai.org/
http://figshare.com/articles/What_aren_t_you_seeing_Advance_in_visualisation_reserach_/1464961
http://www.research-information.admin.cam.ac.uk/
http://figshare.com/articles/Beyond_Serendipity_Networks_of_Research_at_Cambridge/1431829
Donald Schon’s Reith Lecture: “The Loss of the Stable State”, as part of the series Change and Industrial Society, 1970. http://www.bbc.co.uk/programmes/p00h65rn#play