Open Science Lecture: Altmetrics

1

b

b

www.know-center.at

Altmetrics

Open Science – Technologie und Wissenschaft

Wien, 13. Jänner

Dr. Peter Kraker

2

AN EXPONENTIALLY

EXPANDING UNIVERSE

Part 1

3

The Hitchhiker‘s Guide to the Galaxy

3 Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

4

How to Stay on Top of the Universe


5

How to Stay on Top of Scientific Literature


6 6 Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics

“One of the diseases of this age is the

multiplicity of books;

they doth so overcharge the world that it is not

able to digest the abundance of idle matter that

is every day hatched and brought forth into the

world.“

Attributed to Barnaby Rich in 1613


Price (1963)

8

SCIENTOMETRICS:

THE QUANTITATIVE

ANALYSIS OF SCIENCE

Part 2

9

Eugene Garfield: Pathways through Science


Science Citation Index (Garfield 1955)

Web of Science/Web of Knowledge

An index of incoming citations

Outgoing citations

Incoming citations

10 10

Garfield et al.

(1964)

Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics


Van Eck and Waltman

(2010)

13

Evaluative Scientometrics: Impact Factor


A measure to quantify the relative importance of a scientific

journal

The average number of citations in a given year y to papers of a

journal in the years y-1 and y-2

𝐼𝐹𝐽𝑜𝑢𝑟𝑛𝑎𝑙 𝑋, 2015 =

𝐶𝑖𝑡𝑎𝑡𝑖𝑜𝑛𝑠 𝑖𝑛 2015 𝑡𝑜 𝑎𝑟𝑡𝑖𝑐𝑙𝑒𝑠 𝑝𝑢𝑏𝑙𝑖𝑠ℎ𝑒𝑑 𝑏𝑦

𝐽𝑜𝑢𝑟𝑛𝑎𝑙 𝑋 𝑖𝑛 2013 𝑎𝑛𝑑 2014

𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑎𝑟𝑡𝑖𝑐𝑙𝑒𝑠 𝑝𝑢𝑏𝑙𝑖𝑠ℎ𝑒𝑑 𝑏𝑦 𝐽𝑜𝑢𝑟𝑛𝑎𝑙 𝑋

𝑖𝑛 2013 𝑎𝑛𝑑 2014


Source: Thomson Reuters


Journal X

Paper 1 Citations 15






Impact Factor 15.5

Median 15 2

Journal Y

100

2

1

2

1

2

18.0

Rank 2 1

16

Criticisms of the Impact Factor


Source: Amin & Mabe (2000)

17

Antidotes


„Blockbuster“ papers can skew the IF

Use robust statistics (e.g. the median)

Report variation in the data (CWTS Journal Indicators)

Publication and citation behavior varies wildly between fields

Source-normalised impact factor (SNIP)

The IF window is too short

3- and 5-year Impact Factors

18

Criticisms of citation-based metrics


Citations take very long to appear in meaningful quantities

Citation-based metrics are dependent on the corpus that is used

for calculation

Citations are mostly assigned to articles, leaving other products

of research out of the picture (data, code etc.)

Citation-based metrics are often intransparent and irreproducible

19

ALTMETRICS:

A NEW HOPE

Part 3

20

Setting the Stage for Alternative Metrics


Open Science and the publication of more research products

Increased use of online services in the scientific community

Source: figshare

21

Altmetrics


Altmetrics: metrics based on data generated in online systems

Seeing academic literature through the eyes of the readers (Rowlands

& Nicholas 2007)

Usage data (downloads, views, readership)

Links, likes and shares

Blogs and comments

Promises of altmetrics

Assess publications quicker and on a broader scale

Consider all outputs of research, not just papers

The altmetrics manifesto: http://altmetrics.org

http://altmetrics.org/

http://altmetrics.org/


http://zenodo.org/record/34079

http://zenodo.org/record/34079


http://www.altmetric.com/details/4822452




Source:

https://impactstory.org/Carl

Boettiger

https://impactstory.org/CarlBoettiger





http://openknowledgemaps.org

Kraker et al.

(2014)

http://openknowledgemaps.org/


http://stellar.know-center.tugraz.at/umap

Evolution of a Knowledge Domain

Kraker et al.

(2014)




28

The Emerging Open Digital Science Ecosystem

Bibliographic data – bibliometric data – full texts – open source software

29

CAVEATS

Part 4


Kraker & Lex

(2014)

33

The Leiden Manifesto: Ten Principles for Research

Metrics (Hicks et al. 2015)

1) Quantitative evaluation should support qualitative, expert

assessment

…

4) Keep data collection and analytical processes open,

transparent and simple

5) Allow those evaluated to verify data and analysis

…

9) Recognize the systemic effects of assessment and indicators

10) Scrutinize indicators regularly and update them

34

Further Reading

Blog: http://blog.wikimedia.de/2015/09/15/freies-wissen-und-

wissenschaft-teil-4-ueber-die-notwendigkeit-fuer-mehr-offenheit-

und-transparenz-in-der-qualitaetssicherung-und-evaluierung-

open-peer-review-und-open-metrics/

Publication: Kraker, P., & Lex, E. (2015). A Critical Look at the

ResearchGate Score as a Measure of Scientific Reputation.

In ASCW’15 Workshop at Web Science 2015.

doi:10.5281/zenodo.35401

Source Code: https://github.com/pkraker/Headstart

http://blog.wikimedia.de/2015/09/15/freies-wissen-und-wissenschaft-teil-4-ueber-die-notwendigkeit-fuer-mehr-offenheit-und-transparenz-in-der-qualitaetssicherung-und-evaluierung-open-peer-review-und-open-metrics/


















































https://github.com/pkraker/Headstart



35

b

Thank you for your attention!

Peter Kraker

[email protected]

Twitter: @PeterKraker

Open Science Lecture: Altmetrics

Science

Transcript of Open Science Lecture: Altmetrics