Post on 21-May-2020
Horizon 2020 Policy Support
Facility
Different types of altmetrics (Part 1/2) 07 / 04 / 2017
Kim Holmberg
kim.j.holmberg@utu.fi
http://kimholmberg.fi/
Page 2 Horizon 2020 Policy Support Facility
Content
Part 1 / 2
Key points from the kick-off meeting and the
questionnaire
Objectives of the working meeting
Background information (what are altmetrics)
Part 2 / 2
What is impact?
Benefits and challenges of different types of altmetrics
Altmetrics data aggregators
Page 3 Horizon 2020 Policy Support Facility
Key points from the kick-off meeting
Great diversity in the approaches to and the degree of
adoption of open science exist between the member
states.
It is also clear that although there are some ongoing
discussions and a keen desire to learn more about
altmetrics, none of the member states are using
altmetrics for research evaluation and only few
examples of other types of use were mentioned.
Page 4 Horizon 2020 Policy Support Facility
Key points from the questionnaire (1/5)
Question 1: Are you aware of ways in which altmetrics
are being used to assess research in your country? At
governmental/institutional/individual level?
Altmetrics are not used for research evaluation at any
level in the member states. However, member states are
aware of altmetrics and some examples of use of them
do exist.
Page 5 Horizon 2020 Policy Support Facility
Key points from the questionnaire (2/5)
Question 2: Are you aware of any other ways in which
altmetrics are being used in your country?
Besides from some national journals and national or
institutional repositories altmetrics are not widely used
in the member states for other purposes either.
In some of the member states researchers themselves
add altmetrics to their CVs and homepages.
Page 6 Horizon 2020 Policy Support Facility
Key points from the questionnaire (3/5)
Question 3: What kind of change do you wish to see in
your country in regard to open science and altmetrics?
Learn about what altmetrics could do to present the
impact/attention (both scientific and societal) of locally
relevant research published in a national language
Clear national policies and operational action plans to
realize open science principles in research and education
Development of a broader and more appropriate set of
indicators and article level metrics
A cultural change towards open science
Page 7 Horizon 2020 Policy Support Facility
Key points from the questionnaire (4/5)
Question 4: What are your specific expectations to get
out of this MLE in regards to altmetrics?
Learn about what altmetrics can be used for and what
they should not be used for and with that
Learn about open science policies and regulations and
about the experiences from different countries
Learn about approaches to ensure open research data,
and exchange and re-use of it
Learn about good practice examples of open science
training and education programs
Learn about successful reward and evaluation systems
Page 8 Horizon 2020 Policy Support Facility
Key points from the questionnaire (5/5)
Question 5: Any failures or success stories regarding
open science initiatives and/or (alt)metrics that you
wish to share?
The success stories mentioned revolve around the
promotion of open science via institutional Research
Information Systems (CRIS)
In some cases some altmetrics have been displayed at
article level (in for instance repositories or researchers’
profiles)
No failures were mentioned (with the exception of a
conservative attitude towards more traditional metrics)
Page 9 Horizon 2020 Policy Support Facility
Objectives of the working meeting
The objectives for the 1st Working Meeting include:
1. Discussion about what impact means for different
stakeholders and how different types of impact could
and should be responsibly measured.
2. Learn about altmetrics in general and more
specifically about the possibilities and challenges with
using them for research evaluation
Page 10
Background information A (very) short introduction to altmetrics (”alternative metrics”
Altmetrics the study and use of scholarly
impact measures based on
activity in online tools and
environments
Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto,
C.R. (Eds.). Beyond Bibliometrics: Harnessing
Multidimensional Indicators of Scholarly Impact. MIT
Press, 2014.
“…altmetrics presents an
alternative to the current
practice of relying only on
citation counts and
journal impact factors for
the quantitative analysis
of impact by introducing
new complementary
approaches and
sources of data.”
Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level
discussion and metrics. Learned Publishing, 26(1).
“No one can read everything. We rely on
filters to make sense of the scholarly
literature, but the narrow, traditional filters
are being swamped. However, the growth of
new, online scholarly tools allows us to
make new filters; these altmetrics reflect the
broad, rapid impact of scholarship in this
burgeoning ecosystem. We call for more
tools and research based on altmetrics.”
Altmetrics: a manifesto - http://altmetrics.org/manifesto/
Search
Altmetrics data providers
Page 17 Horizon 2020 Policy Support Facility
Altmetrics data providers
Including but not limited to:
Policy documents published online
Mainstream media (news sites)
Social reference managers (e.g., Mendeley, CiteULike)
Peer-review platforms (e.g., Publons, PubPeer, F1000)
Wikipedia
Blogs (curated list of scientific blogs)
Social media (e.g., Twitter, Facebook, Google+, LinkedIn)
Other sources such as YouTube, Reddit and Pinterest
Page 18 Horizon 2020 Policy Support Facility
Associations between altmetrics and citations
“Statistically significant associations were found
between higher metric scores and higher citations…
(Twitter, Facebook wall posts, research highlights, blogs,
mainstream media and forums)...”
“…the coverage of all the altmetrics except for Twitter
seems to be low and so it is not clear if they are
prevalent enough to be useful in practice.”
Thelwall M, Haustein S, Larivière V, Sugimoto CR (2013) Do Altmetrics
Work? Twitter and Ten Other Social Web Services. PLoS ONE 8(5): e64841.
doi:10.1371/journal.pone.0064841
Altmetrics are not alternatives to citations but rather complements
Measure of
attention,
engagement
or influence
Altmetric.com 2014 Top 10 scientific articles (Source: http://www.altmetric.com/top100/2014/)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0
50
100
150
200
250
300
total_posts_blog
total_posts_policy
total_posts_fbwall
total_posts_wikipedia
total_posts_gplus
total_posts_linkedin
total_posts_tweet
Engineering, Altmetric (2016): Altmetric Top 50 2016. figshare. https://doi.org/10.6084/m9.figshare.c.3590951.v2. Retrieved: Apr 04, 2017
Stories behind the numbers?
Measure of societal impact
on different audiences (?)
Usage and demographics of social media platforms. Pew Research, http://www.pewinternet.org/2016/11/11/social-
media-update-2016/
Mapping interactions,
contexts and networks
“… instead of emulating the indicators used for scientific impact --based on outputs or citation approaches--, societal impact assessment should develop a different methodological approach --based on interaction approaches.”
Robinson-Garcia, van Leeuwen, & Ràfols (2017). Using altmetrics for contextualised mapping of societal impact: from hits to networks. SSRN. Available at:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2932944
Holmberg, K., Bowman, T.D., Haustein, S., & Peters, I. (2014). Astrophysicists’ Conversational Connections on Twitter. PLoS ONE,
vol. 9, no. 8: e106086. DOI:10.1371/journal.pone.0106086. Available
at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0106086
Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I. (2014). Astrophysicists on Twitter: An in-depth analysis of
tweeting and scientific publication behavior. Aslib Journal of Information Management, vol. 66, no. 3, pp. 279-296. DOI:10.1108/AJIM-
09-2013-0081. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/AJIM-09-2013-0081.
Astrophysicists
on Twitter –
results from two
studies
Communities of attention and
conversational networks
Communities of attention and
conversational networks
Publications and tweets per day: -0,339*
Citation rate and tweet per day: -0,457**
Astrophysicists on Twitter
Altmetrics can be… … a measure of attention,
engagement or influence
… a measure of societal impact
on different audiences
… used to map interactions,
contexts and networks
… used to tell the narratives
behind the numbers
But…
More research is needed!