Post on 22-Mar-2019
Making a difference: The impact of scholarly publication
in addictionJudit H. Ward, Director of Information Services,
Paul Candon, Managing Editor, Journal of Studies on Alcohol and Drugs,
Center of Alcohol Studies, Rutgers, the State University of New Jersey
ELISAD Conference, Budapest, October 8-10, 2009
Agenda
• Background• Performance indicators• The Project• Conclusions• Q&A
Disclosure of bias and authorship: Prepared by the Center of Alcohol Studies Library staff for the Journal of Studies on Alcohol and Drugs, both part of the Center of Alcohol Studies, Rutgers University
Background: Center of Alcohol Studies
• Interdisciplinary research center at Rutgers University (New Jersey) since 1962
• Research in • Biological sciences (biochemistry and
neuropharmacology) • Psychology (clinical and experimental) • Sociology
• Education and Training• Continuing professional education• Summer school of alcohol studies
• More at: alcoholstudies.rutgers.edu
Background: Information Services
• Library and Archives• one of the world’s largest and most comprehensive
English-language collections on alcohol abuse• mission: to organize, preserve, and disseminate
information for scholars and clinicians in the field
• Research done by• librarian, library school student, library school
professor, editor, researcher, author, publisher
Background: Journal of Studies on Alcohol and Drugs
• The oldest substance-abuse journal published in the United States
• Multidisciplinary: epidemiological, psychological, psychiatric, biological, medical, social, and legal aspects of alcohol and drug use, abuse, and dependence
• Online access to the journal's archives dating to 1975 for subscribers
• One of the most cost-effective of all the psychology and substance-abuse journals
• Name change: QJSA 1940, JSA 1975, JSAD 2007Please see poster for more information
The Project: Significance
Interdisciplinary research Googlization
e-only journals BUDGET CRISIS
international collaboration preprints
NEW MODELS OF SCHOLARLY COMMUNICATION
NEW ACCESS TO CURRENT RESEARCHworking papers
open access new technologiesdiscussion forums
NEW FORMS OF SCHOLARLY PRODUCTS
Facebook Twitter Blogs
The Project: Impact• Why now? Another research with unimportant
discoveries? Mr. Turner and the “merformance”test?
• Direct impact– Unveil JSAD trends, corroborate anecdotal evidence,
verify assumptions, fix errors– Understand how audience (scholars, librarians,
competitors) views JSAD via available tools and indicators
– Learning opportunity in bibliometrics, research methods, group decision making, collaboration, and communication
• Long-term impact– Still a work in progress
Performance Indicators: Terminology
• Working definition: quantitative and qualitative statistical information used to assess performance
• Metrics-based research includes– Bibliometrics: "the application of mathematics and
statistical methods to books and other media of communication“ coined by Alan Pritchard in Journal of Documentation (1969) 25(4):348-349
– Scientometrics: science in general, not just publications– Informetrics: includes all information objects– Webmetrics, cybermetrics: focuses on web relationships
• Literature review nearly impossible without focus– Each slide has abundant literature
Performance Indicators: Users
• Scholars• AU: monitor own performance in research context,
peers’ performance, tenure and reappointment• JN: what to read and where to publish, invitations
(reviewer, editor, board member)
• University administrators• Performance of unit or individual scholar/educator• Hiring and tenure decisions• Budget and financial support: fund allocations,
policies
Performance Indicators: Librarians
• Evaluating JN performance for collection development: subscriptions, cancellations, weeding, formats
• Quantitative analysis: basis and/or justification for decisions
• Organization of information for patrons• Research assistance for patrons: scope, impact
factor, trends, based on history of science• LIS research: complementing qualitative methods
with quantitative (similarly to social sciences)• WorldCat for books & journals, Ulrich’s Periodical
Directory
Performance Indicators: Publishers
• Monitor trends and make decisions• Identify patterns, structure and dynamics,
predictions• Interested in
• authors and institutions who contribute• the most highly cited authors and articles• types of articles drive the impact factor• percentage of papers with no citations• geographical distribution• impact of special issues and supplements• expected citation rates• ranking among peer journals
The Project: Research Design• Participants: Judit Ward, Paul Candon, James
Cox, Molly Stewart, Nicole Frisone • Goal: Identify citation patterns of articles published
in JSAD over time• Methods: Using a variety of citation analysis
software, compute and analyze citations for particular periods
• Communication: shared workspace, F2F, informal
• Data storage– Electronic: refdesk computer, RefShare– Print: at refdesk in folders organized by topics
The Project: Questions • MEASURES?
– Citation analysis software: Which ones? How many? Proprietary vs. free? Benchmarking? Growth? Productivity? Mapping?
• CRITERIA?– AU? TI? Citation? Depth? Cited articles vs citing articles? Co-
citation?– Publishing period: X-year cycle? Selected years? Selected
issues? Selected articles? Number of issues or articles in period– CAS authors only? Institution? RU? CAS? Collaborations?
• DATA?– Top 10? Top 25? 0 citation?– Subject categories: DB driven? Compare apples and oranges?– Textual analysis? Frequency? Classification?
• ETC?
Citation Maps
Cited
Cited
Cited
Cited
Cited
Cited
ARTICLEARTICLE
CitingCiting
CitingCiting
Impact: 1-gen Forward Citation
Web of Science, (Thomson Reuters, 2009)
More Impact: 2-gen Forward Citation
Web of Science, (Thomson Reuters, 2009)
1-gen Backward/Forward Citation
Web of Science, (Thomson Reuters, 2009)
The Project: Breakdown• JSA, JSAD representation and metrics• A few representative articles from JSA or
JSAD first authored by CAS researchers• One highly-cited article by a CAS author as
benchmark• One outstanding CAS author as benchmark• Compare findings• Assess results in the context of addictions
publications
The Project: Workflow
Institute for Scientific Information (ISI)
• Founded by Eugene Garfield in 1960
• Market leader • Most comprehensive
data• Widely used &
accepted• Web of Knowledge• Web of Science• Current Contents
• Citation Indexes (Humanities, Social Sciences, Science)
• Journal Citation Report
• ScienceWatch• Impact Factor• Eigenfactor• ISI Highly Cited• ResearcherID
Journal Citation Reports®
• journal performance metrics for over 7,500 scholarly and technical journals
• statistical information based on citation data
• includes virtually all areas of science, technology, and social sciences
• helps assess influence in the research community
• shows impact in the context of journal and field
Ranking of Addiction Journals
2008 Journal Citation Reports® (Thomson Reuters, 2009)
Impact FactorNumber of citations received in current year by
papers published in the journal in the previous two years
DIVIDED BYNumber of papers published in the journal in the
previous two years
2008 Journal Citation Reports® (Thomson Reuters, 2009)
Eigenfactor™ and Article Influence™
• Eigenfactor™: the number of times articles from the journal published in the past five years have been cited in the JCR year
• EF Category : Psychology– 18th out of 62
• JCR Category:Substance Abuse -4th out of 23
• Article Influence™ Score: relative importance of the journal on a per-article basis.
• =Eigenfactor Score divided by the fraction of articles published by the journal (sum total of articles from all journals is 1)
• mean Article Influence Score =1.00.
• score less than 1.00 => each article in the journal has below-average influence
• score greater than 1.00 =>each article in the journal has above-average influence
use citation data from JCR to assess and track the influence of a journal in relation to other journals
Free and searchable at www.eigenfactor.org
Cost effectiveness2nd out of 23
Impact Factor - Eigenfactor• Both: a ratio of number of citations to total
number of articles• Eigenfactor Score: citations to journals in
both sciences and social sciences • Eigenfactor Score: eliminates self-citations
(references from one article in a journal to another article from the same journal is discounted)
• Eigenfactor Score: weights each reference according to the amount of time researchers spend reading the journal
Example: JSA scores and trends
JSAD in Journal Citation Reports®
2008 Journal Citation Reports® (Thomson Reuters, 2009)
Journal Citation Reports®
Citations from the journal (per cited year)
Citations to the journal (per cited year)
distribution by cited year of citations to items published in the journal
distribution by cited year of citations from current-year items in the journal
2008 Journal Citation Reports® (Thomson Reuters, 2009)
IncitesThe top 20 journals in Social Sciences
ScienceWatchExample: Top papers/journals in Special Topics Underage/College Drinking
Free resource! See also Top institutions, Top authors, etc.
Example: AU Performance
Source: ISI (BUT 248 listed in ISIHighlyCited) instead of 114 results
ISIHighlyCited.comSM
• free database of researchers• the most highly cited within each category for
the period 1981-1999• top individual researchers in each of 21
subject categories– biographical information: appointments, honors,
memberships– Complete listing of publications with links to
bibliographic information or full text• membership is an honor
H-index (Hirsch index, Hirsch number)
based on two factors: cited papers and the number of their citations in other publications
“A scientist has index h if h of [his/her] Np papers have at least h citations each, and the other (Np - h) papers have at most h
citations each” (Hirsch, 2005, PNAS, 102, p. 16569)a scholar with an index of 27 has published 27 papers each of
which has been cited by others at least 27 times
• problem: each DB uses it own data
Publish or Perish: AU
Publish or Perish: AU
Publish or Perish: JSA
Publish or Perish: JSAD
SCOPUS: JN Profile
SCOPUS: AU Profile
SCOPUS: AU
SCIMago: Data from SCOPUS
SCIMago: Data from SCOPUS
Going with the flowPresence on the Internet
Performance 2.0?
Summary: Types of problemsJournal• Name change JSA>JSAD, records not updated (JCR, PubMed, WorldCat,
Scopus, etc.)
Authors • No name authority file, or ignored• Name variations (Helene Raskin White: H. White, H.R. White)• Transliteration (Vaschillo, Vashchillo)
Institution• Center of Alcohol Studies within Rutgers University• Name variations: Rutgers, the State University of New Jersey
Articles ( see example on next slide)• White, H.R. and Labouvie, E.W. 1989. Towards the Assessment of
Adolescent Problem Drinking. Journal of Studies on Alcohol 50:30-37.
Example: the most cited article
Conclusions
• No single source is sufficient• Context of the field: multidisciplinary,
evolving• Comparison, corroboration • Multiple perspectives of diverse users• Soliciting feedback from patrons,
subscribers• Practice critical thinking in evaluation