Mindy ThunaScience Liaison

Pam KingScholarly Communication Liaison

This work is licensed under:

http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_US

You are free to:Share — copy and redistribute the material in any medium or formatAdapt — remix, transform, and build upon the material

Under the following terms:Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

NonCommercial — You may not use the material for commercial purposes.

ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

Altmetrics in Higher Ed Settings metrics in context - What our researchers are saying

Agenda• Impact Metrics• Altmetrics• What our researchers have to say• Bringing it back to libraries

IMPACT METRICS

Impact Metrics are……calculations/algorithms that quantify the impact of research or scholarly activity.

Research impact defined“Impact is usually demonstrated by pointing to a record of the active consultation, consideration, citation, discussion, referencing or use of a piece of research.”

http://blogs.lse.ac.uk/impactofsocialsciences/introduction/

Basic Premise

Number of citations

Impact

Reach and significanc

e

Three main providers• Scopus

• Web of Science

• Google Scholar

Three Levels• Journal Level

• Article Level

• Author Level

Journal Level Metrics

Web of Science

Impact Factor (2 & 5 year)

Cited half-life

Eigenfactor®

Score

Article Influence® Score

Scopus SCImago Journal Rank (SJR)

Source Normalized Impact per Paper (SNIP)

Google Scholar

h5-index h5-median

Journal Impact FactorAverage number of times articles published in a two year (or five year) period have been cited

A = total number of times ALL articles published in 2 (or 5) year period were cited in WofS indexed journals during the next year

B = total number of "citable items" (usually articles, reviews, proceedings or notes; not editorials and letters-to-the-editor) published in 2 (or 5) year period

Impact factor = A/B

Journal Impact Factor Debate

• Vary across disciplines• e.g., variants in time-to-publication

• Review journals have higher impact• fewer articles per journal, cited more

• Calculations easily skewed• heavy editorial/letter/opinion content• high/low percentage of review articles

Declared Dissent – 10,419

• “Impact factors declared unfit for duty”Stephen Curry, LSE Blog

• “Do not resuscitate: the journal impact factor declared dead”

Brendan Crabb, The conversation

• “Just say no to impact factors”Ismael Rafols and James Wilsdon, the guardian

DORA: San Francisco Declaration on Research Assessment (Dec. 2012)

Author Level Metrics

Web of Science

h-index

Scopus h-index

Google Scholar

h-index

i10-index

h index is NOT perfect

Paper 1

Paper 2

Paper 3

Paper 4

Paper 5

Author A 3 cites 6 cites 100 cites

4 cites 1 cite

Author B 3 cites 6 cites 100 cites

Author C 400 cites

150 cites

3 cites 6 cites

h index = 3

Article Level MetricsCitation Counts - how often a particular journal article is being cited by other articles

ALTMETRICSALT-METRICSALTERNATIVE METRICS

Web metrics• Webometrics - link analysis, web log file analysis,

1990s• Bibliometric use – downloads, citations• Google web page ranking• Web 2.0 ->scientometrics 2.0 (Priem &

Hemminger 2010)

Open Access

• Producing a highly complex information environment; touches all disciplines• Many formats, e.g. websites, nanopubs, blog posts,

code, images• versions, e.g. preprints, postprints, “same article

different places”, version of record

Identification SystemsAuthors• International Standard Name Identifier (ISNI)• ORCID (subset of ISNI)• Names Project (JISC) • Author Claim, RePEc Author Service• arXiv Author ID• ResearchID (Thomson Reuters)• Scopus Author ID (Elsevier)• Gravatars (Auttomatic)

Objects• DOIs, PMIDs URLs, URIs

Altmetrics

… are calculated from newer data sources, e.g. Wikipedia, Mendeley, Twitter, Facebook, Weibo, blogs

… report the impact of a wider range of research outputs, e.g. presentation slides, data sets, articles, code

Altmetrics aggregators/tools/ services

• PLoS One Article Level Metrics (ALM)• ImpactStory • Altmetric • PlumX • and others …

PLoS Article Level Metrics (ALMs)

ImpactStory

30

Altmetric (altmetric.com)

PlumX

The Debate“Article-Level Metrics: An Ill-Conceived and Meretricious Idea” vs “Broaden your horizons: impact doesn’t need to be all about academic citations”

(Jeffrey Beall vs Euan Adie, Aug. 2013)

Moving forward

“to move out of its current pilot and proof-of-concept phase”

• Sloan foundation grant for NISO initiative - creation of community-based standards; definitions, calculations. Data classifications and data sharing practices

WHAT OUR RESEARCHERS HAVE TO SAY

Our Ongoing Study“Research Impact Metrics: Use and Understanding by Post-Tenure Faculty”

What do you think of when you hear the term impact metrics?

“… I don't think that impact factors at the journal level necessarily captures the importance of the paper... I jokingly say that out of all the things I have published, the paper that I expect will have the longest life is the one that's got the lowest impact factor.”

Biology

What do you think of when you hear the term impact metrics?

“Nothing I don’t understand what your talking about. No clue. I understand ‘impact’ I understand those words separately. But not together.” Mathematical and Computational Sciences

“Means nothing to me – zero.” Political Science

What do you think of when you hear the term impact metrics?

“I have a picture of that book that you find at the Robarts library on the 4th floor that is a collection of binders in colourful colours and when you open it, it looks like the phone book with all the list of all the articles and researchers so that is my idea of impact metrics.”

Language Studies

Have you considered/do you use any alternative metrics?“No. No trust in them.” Mathematical and Computational Sciences

“… I give a lot of relevance to those metrics and I will tell you why…There was an event in the year 1900. It was some popular festivity in England and they were running a contest. There was a cow or a bull and people had to guess it's weight and the person who gets closes to the weight wins the cow, or something like that. And someone kept a record of the answers and it is very interesting. It was very wide-ranging so let's say the weight of the cow was 500 kg. So the range was from 300-1000. The most accurate answer was the average of all the answers that people gave…So what is that telling you. It is telling you many things…It is telling you that this phase in which we are moving into in which content is user generated, there is a lot of proof there. You have to remember, I am a mathematician, I don't care what people think, we build truth from the bottom up…”

Mathematical and Computational Sciences

BRINGING IT BACK TO LIBRARIES

Bringing it back to libraries• Traditional metrics• Libraries provide the support• Libraries pay for the metrics• Libraries purchase the research output

• Altmetrics change the rules• Now what happens?• It all depends on what libraries want to measure….

• tweets and blogs about our buildings and spaces and our service

• Bookmarks of websites and libguides• Likes on Facebook• Downloads of librarian publications

Thoughts?