Building Your Academic Profile (Part 2):

Understanding the Role of Citation Analysis & Impact Factors in Your Academic Career

Sheryl Adam | Dean Giustini | UBC Librarians

Related workshops: Web of Science/Journal Citation Reports

Scholar, Blogger, Tweeter, Author: Building Your Academic Profile

Measuring research impact:introduction to citation metrics in scholarship

Sheryl Adam | Dean Giustini | UBC Librarians

I. Introduction

II. Overview of issues | tools

III. Practice time for Google scholar | Web of Science

IV. Discussion / Q & A

Agenda – 1 hr

• Overview of journal impact vs. article impact metrics vs. scholar impact

"citation metrics“

• Pros & cons | how to calculate impact factors for journals (& scholars)

• How to manage your indexed publications (or unindexed)

• Leave workshop knowing a bit about Web of Science & Google scholar

Goals

In the digital age it's increasingly "Get cited or perish"

Departments track citation counts for individuals/subgroups

Noticeable increase in requests for citation metrics from faculty, especially for tenure and promotion & graduate students

Emphasis on quantitative data & evaluating output to make decisions in academic units

Citation counts used in tenure & funding decisions by institutions

Publish or Perish?

• To examine humanities-social sciences vs. sciences viewpoints

• To consider a broad range of “usage” and “metrics”

• To discuss what shapes success (or lack thereof)

• Does “value” shift across disciplines?

• connection between impact | quality | importance?

• Tools, how they deal with citations, articles-authors & books

• Which tools cover books? • Monographs still benchmark in humanities/social science

Why look at this issue?

See journal impact factors based on "usage" (measured by citations, primarily to journal articles, but also some book chapters)

Rank or visualize impact of publications, scholars & journals

Discover new research areas and trends by mapping

See which journals are most important, and publish in those journals

Why evaluate ‘impact’?

Citation, journal & author metrics are part of field called bibliometrics

Bibliometrics is "discipline of measuring performance of a researcher, collection of articles, journal, research discipline or institution“. It involves "application of statistical analyses to study patterns of authorship, publication, and literature use" (Lancaster 1977)

What is being counted? # papers published per individual, institutional aggregated # of citations, cites per paper, cites per journal, cites per book, cites per book chapter …& so on

Other metrics: number of patents, number of graduate students, conference papers, monographs, etc.

How is citation analysis done?

Web of Science and Scopus (UBC does not subscribe to Scopus)

For example, determine researcher’s output of articles from 2005 to present

Book chapters (see other tools) find social scientists’ work

Small part of humanities scholars’ work

Look at journal impact factor (IF) in Journal Citation Reports

Determine "research impact"/ figure out H-index

Google scholar can be used

Research impact in STEM

Web of Science and Scopus

• index only a small part of social scientists’ work , a very small part of a humanities scholar’s output of journal articles: and a limited number of books

To determine HSS research impact, you need to use tools that find citations in books and to books, including GoogleBooks, HathiTrust and other tools .

For example, in Google Books, the book to journal citations ratio, roughly: Philosophy 4:1 |Sociology, Fine Arts 3:1 | Psychology 1.5:1 | Physics .001:1

Example(s): Jon Beasley-Murray; W. Peter Ward

Research impact in Hum-SocSci

Prepare for confirmation, promotion, tenure

Know who is citing your work; find collaborators

Exciting to see how others use your research

Get ego boost when someone reads your paper

Generate new ideas or reframe existing

Why do you need impact?

Journal impact factor (Web of Science , Eigenfactor)

Individual scholar impact: h-index (Web of Science, also Google Scholar)Individual scholar impact: i10-index (Google Scholar)

Individual article impact: Citation count(Web of Science; Google Scholar)

InCites - individual and institutional data

Article acceptance rate- low acceptance rate = high status journal?

Traditional Impact Measures

Google Scholar : only source with broad coverage, some limitations

Web of Science : “Gold Standard”

Eigenfactor.org : non – commercial alternative to Web of Science

Discipline specific citation sources, e.g. Citeseer (computer science), RePEc (Economics)

More info about discipline-specific sources at http://en.wikipedia.org/wiki/Google_Books and http://hlwiki.slais.ubc.ca/index.php/Citation_metrics_%E2%80%93_Where_to_find_them%3F

Where is citation data?

• All major & mid-size academic publishers/societies• Elsevier, Springer, Wiley, Science, Nature, APS, ACS, IEEE, ACM, IOP,

OUP, Sage, JSTOR, Taylor & Francis, Highwire, LWW…

• Most smaller publishers & journals

• All major public A&Is (open search tools)• PubMed, INIST, ADS, DTIC, ERIC, AGRIS, WHO…

• Google Books, US/CAN/EU/WO Patents

• Repositories: arXiv, PubMed, RePec, SSRN

Google scholar covers?

• Simple, open way to track citations to author’s articles • A form of user-generated evaluation

• Determine citation metrics• Do it for most articles in most journals• Create automatically maintained profile

• Public profiles appear in any GS search results display

• Takes ~10 mins to set up feature (need a Google mail account)

• Automatically finds all (most?) papers by an academic• Sometimes, even ones forgotten (or, hidden)

Google scholar citations feature

All have data errors; in fact, data is less complete & accurate than you'd think

Web of Science relies on journals but some areas (e.g., philosophy) incomplete

Impact metric formulas are based on science citation patterns

most articles cited within a few years of publication

HSS disciplines are cited over longer periods, e.g., relevant longer

Years to zero cites: Biomed= 7-8 | Soc=12.5 | History, Philosophy=39 years

Disciplines are divided in ways that don’t make sense to scholars

Canadian journals are missing from lists; critical to publish about Canada, etc.

Gaming the system; manipulated by journal editors; self-citations

Critique of impact factors

Stray citations are common; different “author initials” lead to confusion

WoS has poor aggregation of minor variations of same titles

80% of citations to “Baden-Fuller” entered as "Badenfuller“

80-90% of citations to “van Raan” entered as “Vanraan”

Both receive 100-150 citations to correct name even though their names were correctly referenced (incorrectly entered by WoS)

Require formal request to WoS to merge variations with master record

Language bias: WoS has limited coverage of non-English sources (note origins in science) e.g. For example, journals in CAIRN not included

Critique of impact factors

Google scholar is useful for individuals | CAUTION: some counts are inflated

Web of Science – InCites are useful for universities (UBC does not have Scopus)

H-index is convenient summary of quantity & impact (productivity & citedness)

H-index of 7 = 7 papers have at least 7 citations each

total # citations is a fair way to assess individual impact

adjust for co-authorship & age (citations in recent years count more)

focus on citations “per paper” may discourage new scholars

adjust by years (cites/year) age of papers (AWCR; age-weighted citation rate) and # of authors (AWCRpA)

Sources & use of data for impact

What else can scholars do to get more citations to their publications?

Will publishing in an open access journal help to improve my citation counts?

Do you promote your research? Are you aware of the major communication channels in your field? If not, why not?

Speak to a subject specialist librarian

Q & A

Thank you~

Instructors: Sheryl Adam & Dean Giustini

Resources for workshop

• February 19th powerpoints & handout • Citation metrics – Where to find them? • Workshop: Graduate Student Workshop Series Citation Metrics

See also Altmetrics | Author impact metrics | Bibliometrics | Citation management | Utopia Documents | Web of Science vs. Scopus | Webometrics