SCHOLARLY METRICS BOOTCAMP

Rachel Borchardt and Andrea Michalek

USAIN Preconference

May 4, 2014

What will we cover?

The what and why’s of metrics Bibliometrics overview and discussion Hands-on training with bibliometrics tools Tips for bringing it back home Break Altmetrics overview and discussion Hands-on training with altmetrics tools Bringing it all together

But first..

What are your objectives for the workshop?

What are metrics?

Synonyms/narrower terms: Bibliometrics Scientometrics Impact measures Research metrics Altmetrics

“ … a set of methods to quantitatively analyze academic literature.” (Wikipedia)

Why metrics?

We want to identify high-quality scholarship, but

No one can read everything!

Bibliometrics Impact Quality

Used to evaluate publications, scholars, labs, departments and more

Basic metric categories

Article level Times cited

Journal level Impact Factor SJR, SNIP Google metrics

Author level H-index

(note – this is not a comprehensive list, but covers the most frequently used metrics, and what we’ll be talking about today)

Now, let’s dive into these metrics!

Article level metrics

Article-level Metric: Times Cited Most ubiquitous metric “Basic building block” of bibliometrics Captured by many databases, including

Web of Science Scopus Google Scholar

No one source is definitive and results can vary

Example: Elizabeth Ainsworth Web of Science – 45 documents Scopus – 58 documents Google Scholar – ‘About 17,300 results’

Database/resource coverage will determine times cited as well as number of publications

Lesson: often, an accurate times cited requires an extensive search/comparison

We will explore these resources a bit later!

Web of Science Scopus Google Scholar

Journal-level metrics

Journal-level metric: Impact Factor Developed by Eugene Garfield in 1955 Most widely known and used metric Based on Web of Science citations Distributed exclusively by Journal

Citation Reports (also owned by Thomson Reuters)

2-year and 5-year impact factor metrics

(2-year) Impact Factor Formula

Putting Impact Factor in context Preset disciplines can be chosen and

displayed by impact factor to determine highest impact factor journals

Some high impact factors by discipline

Agricultural Engineering – 4.75, Bioresource Technology

Agriculture, Dairy and Animal Science – 3.494, Genetics Selection Evolution

Agriculture, Multidisciplinary – 2.906, Journal of Agricultural and Food Chemistry

Agricultural Economics and Policy – 2.212, Food Policy

Multidisciplinary Sciences – 38.597, Nature Environmental Studies – 14.472, Nature Climate

Change

Lesson: impact factor varies. Greatly.

So, what’s a ‘good’ impact factor?

An often-asked question with no great answer

Context can help determine this answer Reviewers, particularly outside one’s

discipline, should always have context to avoid misunderstanding impact factor

Discipline specificity remains a major problem with context

Remember: no metric is perfect or absolute!

Let’s explore Journal Citation Reports!

http://library.uvm.edu/research/ Search by Title J, Journal Citation Reports

View a group of journals by subject category or Search for a specific journal

Journal-level metric alternatives to Impact Factor

SJR/SNIP Google Metrics (many others have been proposed, but

in my perspective, are not widely used)

SJR/SNIP

SJR = SCImago Journal Ranking SNIP = Source Normalized Impact per Paper Based on Scopus citations SJR uses an algorithm similar to Google’s to

weigh citations (but otherwise similar to impact factor)

SNIP is based on SJR, but attempts to ‘normalize’ scores between disciplines

Both available at scimagojr.com or via Scopus (Elsevier)

SJR/SNIP Rankings

Similar to Impact Factor, discipline context helps understand SJR/SNIP in comparison

Journal rankings only available on scimagojr.com

Let’s explore SCImago!

http://www.scimagojr.com

This is a freely available resource! (great for libraries/schools with small budgets)

Note that multiple journals can be directly compared in a chart Select “Compare”, then “Journals” Search for up to 4 journals, hit “Compare”,

then select the metric for comparison

Google Scholar Metrics H5-index and H5-median metrics

H5-index = X number of articles in the journal that have been cited X times in the past 5 years H5-index of 10 means that 10 articles in the past 5

years have been cited 10+ times each (remember this – we’ll come back to the H-index

concept!)

H5-median is the median number of citations for all of the articles that fit the H5-index criteria H5-median of 14 means that those 10 articles were

cited a median of 14 times (median = middle number)

Google Scholar rankings

Yet again, different discipline categories! Life Sciences and Earth Sciences

Let’s explore Google Scholar Metrics!

http://scholar.google.com Select “Metrics” along the top Note that a custom list of journals can be

created with a keyword search Try searching for ‘dairy’ or ‘soil’

Author-level metrics

Author-level metric: H-index

Remember H5-index?

H-index is used to measure an author’s output over time

Again, X number of publications that have been cited X or more times

H-index highly dependent on the citation rate of the author’s discipline(s) Just like other measures, context is key Also, like other measures, the metric is only as good as the

source

Where to get H-index?

Web of Science, Scopus, and Google Scholar all have author profiles with an H-index.

However, an incomplete citation / times cited means the H-index is also based on incomplete information.

Let’s practice - Sarah Taylor Lovell Can you find the H-index for Sarah Taylor

Lovell in Web of Science, Scopus and Google Scholar?

Web of Science

To access Web of Science: http://library.uvm.edu/research/ Search by Title W, Web of Science

To find Sarah Taylor Lovell: Author Search in dropdown Lovell, ST Look for matching article – her name is hyperlinked! Clicking on name will pull up (mostly) accurate list

of publications Create Citation Report

Scopus

To access Scopus: http://www.scopus.com

To find Sarah Taylor Lovell: Author Search Lovell, Sarah Taylor Select all relevant results View Citation Overview

Google Scholar

To access Google Scholar: http://scholar.google.com

To find Sarah Taylor Lovell: Search for “Sarah Taylor Lovell” Click on user profile at top

Note: H-index only available for authors with a Google Scholar profile

Lesson: like times cited, an accurate H-index requires extensive search/comparison

Calculating H-index

If citations are incomplete, how do we get an accurate H-index?

First, if they’re close, it may not be worth sweating the details..

Large gaps of coverage between databases may substantiate use of Downloading records from individual

databases Compiling records (usually with Excel) Publish or Perish software

Publish or Perish

Free software program Downloadable at

www.harzing.com/pop.htm Calculates H-index and more Based on Google Scholar data

We won’t explore PoP, but it’s a great tool for downloading/analyzing Google Scholar citations

Bringing metrics back to your institution

What bibliometrics should I recommend to my scholars? Factors to consider

Institutional culture What’s widely used at the institution? What’s the popular perception of metrics?

What do you own / have access to? Resource’s discipline coverage

Social science-oriented scholars may be better served by Scopus; many scientists use Web of Science

Comprehensive results requires more Time Energy Skill

Ways you can help scholars: big picture

Get to know metrics needs at institution Keep an eye on trends in academia and

at your institution Be aware of new products and assess

what you have to ensure it continues to meet user need (so, be a librarian!)

Remember that no metric is perfect, context is key

Ways you can help: concrete activities

Workshops One-on-one consultations Department/lab/admin/etc.

presentations Repository of past works / templates Online research guide (many exist, so

feel free to borrow good ideas!)

Personal plug: Look for my book coming this fall from ACRL Press!

Bibliometrics and altmetrics handbook for librarians

Questions / discussion / hands-on practice