Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and...
Transcript of Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and...
![Page 1: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/1.jpg)
Horizon 2020 Policy Support
Facility
Different types of altmetrics (Part 1/2) 07 / 04 / 2017
Kim Holmberg
http://kimholmberg.fi/
![Page 2: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/2.jpg)
Page 2 Horizon 2020 Policy Support Facility
Content
Part 1 / 2
Key points from the kick-off meeting and the
questionnaire
Objectives of the working meeting
Background information (what are altmetrics)
Part 2 / 2
What is impact?
Benefits and challenges of different types of altmetrics
Altmetrics data aggregators
![Page 3: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/3.jpg)
Page 3 Horizon 2020 Policy Support Facility
Key points from the kick-off meeting
Great diversity in the approaches to and the degree of
adoption of open science exist between the member
states.
It is also clear that although there are some ongoing
discussions and a keen desire to learn more about
altmetrics, none of the member states are using
altmetrics for research evaluation and only few
examples of other types of use were mentioned.
![Page 4: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/4.jpg)
Page 4 Horizon 2020 Policy Support Facility
Key points from the questionnaire (1/5)
Question 1: Are you aware of ways in which altmetrics
are being used to assess research in your country? At
governmental/institutional/individual level?
Altmetrics are not used for research evaluation at any
level in the member states. However, member states are
aware of altmetrics and some examples of use of them
do exist.
![Page 5: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/5.jpg)
Page 5 Horizon 2020 Policy Support Facility
Key points from the questionnaire (2/5)
Question 2: Are you aware of any other ways in which
altmetrics are being used in your country?
Besides from some national journals and national or
institutional repositories altmetrics are not widely used
in the member states for other purposes either.
In some of the member states researchers themselves
add altmetrics to their CVs and homepages.
![Page 6: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/6.jpg)
Page 6 Horizon 2020 Policy Support Facility
Key points from the questionnaire (3/5)
Question 3: What kind of change do you wish to see in
your country in regard to open science and altmetrics?
Learn about what altmetrics could do to present the
impact/attention (both scientific and societal) of locally
relevant research published in a national language
Clear national policies and operational action plans to
realize open science principles in research and education
Development of a broader and more appropriate set of
indicators and article level metrics
A cultural change towards open science
![Page 7: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/7.jpg)
Page 7 Horizon 2020 Policy Support Facility
Key points from the questionnaire (4/5)
Question 4: What are your specific expectations to get
out of this MLE in regards to altmetrics?
Learn about what altmetrics can be used for and what
they should not be used for and with that
Learn about open science policies and regulations and
about the experiences from different countries
Learn about approaches to ensure open research data,
and exchange and re-use of it
Learn about good practice examples of open science
training and education programs
Learn about successful reward and evaluation systems
![Page 8: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/8.jpg)
Page 8 Horizon 2020 Policy Support Facility
Key points from the questionnaire (5/5)
Question 5: Any failures or success stories regarding
open science initiatives and/or (alt)metrics that you
wish to share?
The success stories mentioned revolve around the
promotion of open science via institutional Research
Information Systems (CRIS)
In some cases some altmetrics have been displayed at
article level (in for instance repositories or researchers’
profiles)
No failures were mentioned (with the exception of a
conservative attitude towards more traditional metrics)
![Page 9: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/9.jpg)
Page 9 Horizon 2020 Policy Support Facility
Objectives of the working meeting
The objectives for the 1st Working Meeting include:
1. Discussion about what impact means for different
stakeholders and how different types of impact could
and should be responsibly measured.
2. Learn about altmetrics in general and more
specifically about the possibilities and challenges with
using them for research evaluation
![Page 10: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/10.jpg)
Page 10
Background information A (very) short introduction to altmetrics (”alternative metrics”
![Page 11: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/11.jpg)
![Page 12: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/12.jpg)
![Page 13: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/13.jpg)
Altmetrics the study and use of scholarly
impact measures based on
activity in online tools and
environments
Priem, J. (2014). Altmetrics. In Cronin, B. & Sugimoto,
C.R. (Eds.). Beyond Bibliometrics: Harnessing
Multidimensional Indicators of Scholarly Impact. MIT
Press, 2014.
![Page 14: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/14.jpg)
“…altmetrics presents an
alternative to the current
practice of relying only on
citation counts and
journal impact factors for
the quantitative analysis
of impact by introducing
new complementary
approaches and
sources of data.”
Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level
discussion and metrics. Learned Publishing, 26(1).
![Page 15: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/15.jpg)
“No one can read everything. We rely on
filters to make sense of the scholarly
literature, but the narrow, traditional filters
are being swamped. However, the growth of
new, online scholarly tools allows us to
make new filters; these altmetrics reflect the
broad, rapid impact of scholarship in this
burgeoning ecosystem. We call for more
tools and research based on altmetrics.”
Altmetrics: a manifesto - http://altmetrics.org/manifesto/
Search
![Page 16: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/16.jpg)
Altmetrics data providers
![Page 17: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/17.jpg)
Page 17 Horizon 2020 Policy Support Facility
Altmetrics data providers
Including but not limited to:
Policy documents published online
Mainstream media (news sites)
Social reference managers (e.g., Mendeley, CiteULike)
Peer-review platforms (e.g., Publons, PubPeer, F1000)
Wikipedia
Blogs (curated list of scientific blogs)
Social media (e.g., Twitter, Facebook, Google+, LinkedIn)
Other sources such as YouTube, Reddit and Pinterest
![Page 18: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/18.jpg)
Page 18 Horizon 2020 Policy Support Facility
Associations between altmetrics and citations
“Statistically significant associations were found
between higher metric scores and higher citations…
(Twitter, Facebook wall posts, research highlights, blogs,
mainstream media and forums)...”
“…the coverage of all the altmetrics except for Twitter
seems to be low and so it is not clear if they are
prevalent enough to be useful in practice.”
Thelwall M, Haustein S, Larivière V, Sugimoto CR (2013) Do Altmetrics
Work? Twitter and Ten Other Social Web Services. PLoS ONE 8(5): e64841.
doi:10.1371/journal.pone.0064841
![Page 19: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/19.jpg)
Altmetrics are not alternatives to citations but rather complements
![Page 20: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/20.jpg)
Measure of
attention,
engagement
or influence
![Page 21: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/21.jpg)
Altmetric.com 2014 Top 10 scientific articles (Source: http://www.altmetric.com/top100/2014/)
![Page 22: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/22.jpg)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0
50
100
150
200
250
300
total_posts_blog
total_posts_policy
total_posts_fbwall
total_posts_wikipedia
total_posts_gplus
total_posts_linkedin
total_posts_tweet
Engineering, Altmetric (2016): Altmetric Top 50 2016. figshare. https://doi.org/10.6084/m9.figshare.c.3590951.v2. Retrieved: Apr 04, 2017
![Page 23: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/23.jpg)
![Page 24: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/24.jpg)
![Page 25: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/25.jpg)
Stories behind the numbers?
![Page 26: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/26.jpg)
Measure of societal impact
on different audiences (?)
Usage and demographics of social media platforms. Pew Research, http://www.pewinternet.org/2016/11/11/social-
media-update-2016/
![Page 27: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/27.jpg)
Mapping interactions,
contexts and networks
![Page 28: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/28.jpg)
“… instead of emulating the indicators used for scientific impact --based on outputs or citation approaches--, societal impact assessment should develop a different methodological approach --based on interaction approaches.”
Robinson-Garcia, van Leeuwen, & Ràfols (2017). Using altmetrics for contextualised mapping of societal impact: from hits to networks. SSRN. Available at:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2932944
![Page 29: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/29.jpg)
Holmberg, K., Bowman, T.D., Haustein, S., & Peters, I. (2014). Astrophysicists’ Conversational Connections on Twitter. PLoS ONE,
vol. 9, no. 8: e106086. DOI:10.1371/journal.pone.0106086. Available
at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0106086
Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I. (2014). Astrophysicists on Twitter: An in-depth analysis of
tweeting and scientific publication behavior. Aslib Journal of Information Management, vol. 66, no. 3, pp. 279-296. DOI:10.1108/AJIM-
09-2013-0081. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/AJIM-09-2013-0081.
Astrophysicists
on Twitter –
results from two
studies
![Page 30: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/30.jpg)
Communities of attention and
conversational networks
![Page 31: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/31.jpg)
Communities of attention and
conversational networks
![Page 32: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/32.jpg)
Publications and tweets per day: -0,339*
Citation rate and tweet per day: -0,457**
Astrophysicists on Twitter
![Page 33: Different types of Horizon 2020 altmetrics (Part 1/2 ... · impact/attention (both scientific and societal) of locally relevant research published in a national language Clear national](https://reader034.fdocuments.us/reader034/viewer/2022042221/5ec71307f6fd5611092a5b23/html5/thumbnails/33.jpg)
Altmetrics can be… … a measure of attention,
engagement or influence
… a measure of societal impact
on different audiences
… used to map interactions,
contexts and networks
… used to tell the narratives
behind the numbers
But…
More research is needed!