OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare •...
Transcript of OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare •...
![Page 1: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/1.jpg)
11TH ANNUAL MEETING OF ISMPP 1
1 1 T H A N N U A L M E E T I N G O F I S M P P
OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC APPROACHES
TO MEDICAL PUBLICATIONS April 27–29, 2015
Hyatt Regency Crystal City Arlington, VA, USA
![Page 2: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/2.jpg)
11TH ANNUAL MEETING OF ISMPP 2
1 1 T H A N N U A L M E E T I N G O F I S M P P
Innovation in Metrics: Capturing the full impact of publications
![Page 3: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/3.jpg)
11TH ANNUAL MEETING OF ISMPP 3
![Page 4: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/4.jpg)
11TH ANNUAL MEETING OF ISMPP 4
Key questions in metrics
•The performance of: •My agency (quality of service, cost/value) •My team (quality, efficiency, productivity) •Overall publication plan (volume, reach, understanding, impact) •Individual publications (reach, understanding, impact)
What do you want to measure?
Why do you want to measure it?
What is the best way to measure it?
What are you going to do with that information?
![Page 5: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/5.jpg)
11TH ANNUAL MEETING OF ISMPP 5
Key questions in metrics
What do you want to measure?
• To show how I performing against comparators • To understand if I should be doing things differently • To defend spend/budget and demonstrate return on investment
Why do you want to measure it?
What is the best way to measure it?
What are you going to do with that information?
![Page 6: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/6.jpg)
11TH ANNUAL MEETING OF ISMPP 6
Key questions in metrics
What do you want to measure?
Why do you want to measure it?
• What are the best components of a publication analysis? • The impact factor measures the ‘success’ of the journal, not the
‘impact’ of the article; what article level metrics are important? • How do / can we measure the impact of congress activities as
well journal publications?
What is the best way to measure it?
What are you going to do with that information?
![Page 7: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/7.jpg)
11TH ANNUAL MEETING OF ISMPP 7
Key questions in metrics
What do you want to measure?
Why do you want to measure it?
What is the best way to measure it?
• Defend (retrospective) • Apply (prospective)
What are you going to do with that information?
![Page 8: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/8.jpg)
11TH ANNUAL MEETING OF ISMPP 8
“NOT EVERYTHING THAT CAN BE COUNTED COUNTS, AND NOT EVERYTHING THAT COUNTS CAN BE COUNTED”
Jane Grimson. In Blockmans W et al, 2014 Bibliometrics: Use and Abuse in the Review of Research Performance
![Page 9: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/9.jpg)
11TH ANNUAL MEETING OF ISMPP 9
1 1 T H A N N U A L M E E T I N G O F I S M P P
Publication Metrics:
Practical Application and Experience
![Page 10: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/10.jpg)
11TH ANNUAL MEETING OF ISMPP 10
INTRODUCTIONS • Moderator
Sarah Feeney, BMedSc Head of Scientific Direction, Complete Medical Communications
• Panel
Bhakti Kshatriya, PharmD Global Scientific Communications, Novartis Pharmaceuticals Corporation Tom Rees, MSc, PhD, CMPPTM
Scientific Strategy Advisor, PAREXEL International Mike Buschman Co-Founder, Plum Analytics
![Page 11: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/11.jpg)
11TH ANNUAL MEETING OF ISMPP 11
The opinions expressed in this presentation are those of the faculty and do not necessarily represent the opinions of their employers or ISMPP. Neither the faculty nor the facilitator have received honoraria for their participation in this workshop.
DISCLAIMER
![Page 12: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/12.jpg)
11TH ANNUAL MEETING OF ISMPP 12
OBJECTIVES
•Understand the process of developing publication plan metrics
•Be conversant in the various types of newer publication metrics
•Be educated on which metrics are of most importance to various stakeholders
![Page 13: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/13.jpg)
11TH ANNUAL MEETING OF ISMPP 13
1 1 T H A N N U A L M E E T I N G O F I S M P P
PUBLICATION METRICS – PHARMA INDUSTRY PERSPECTIVE
Bhakti Kshatriya, PharmD
Global Scientific Communications Novartis Pharmaceuticals Corporation
![Page 14: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/14.jpg)
11TH ANNUAL MEETING OF ISMPP 14
DISCLAIMER
• This presentation is made at the request of International Society for the Medical Publication Professionals (ISMPP). The presenter, although an employee of Novartis Pharmaceuticals Corporation, is presenting on her own behalf. The information provided and any views expressed during this presentation are those of the presenter and may not be attributed to Novartis Pharmaceuticals Corporation.
![Page 15: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/15.jpg)
11TH ANNUAL MEETING OF ISMPP 15
WHY ESTABLISH METRICS? • Establish common goals among team members and management
• Helps prioritize publication activities • Facilitate discussion of new publication concepts
─ Does concept align with goals?
• Help ensure appropriate resources for planned publication activities
• Support requests for new budget and/or head count
• Measure progress and identify areas for improvement
![Page 16: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/16.jpg)
11TH ANNUAL MEETING OF ISMPP 16
CREATING METRICS
![Page 17: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/17.jpg)
11TH ANNUAL MEETING OF ISMPP 17
FACTORS AFFECTING METRICS Internal Factors • Product life cycle • SOPs & policies • Pub team effectiveness • Pub team resources
– Data generation – Data analyses – Content generation
(authors, writers, contributors, budget)
External Factors • Number of competitors • Author perceptions • Policies at author's
institution • New regulations • Evolving publication
guidelines and journal policies
![Page 18: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/18.jpg)
11TH ANNUAL MEETING OF ISMPP 18
Publication Metrics
• Volume • Timing • Quality • Topics ~ literature gap analysis
![Page 19: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/19.jpg)
11TH ANNUAL MEETING OF ISMPP 19
PUBLICATION METRICS: VOLUME
• Number of manuscripts submitted & published • Number of abstracts/posters/oral
presentations at key scientific meetings • Planned vs actual priority submissions/
acceptances • Number of abstracts/manuscripts to regional
meetings/journals
![Page 20: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/20.jpg)
11TH ANNUAL MEETING OF ISMPP 20
Number of Manuscripts Submitted
PUBLICATION METRICS: VOLUME
• Metric can also be reported by drug/indication • Similar metric can be analyzed for number of manuscripts
published
200
150
100
0
50
Q1 Q2 Q3 Q4
Primary/secondary manuscripts Review articles All Other (editorial, short reports, etc.)
![Page 21: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/21.jpg)
11TH ANNUAL MEETING OF ISMPP 21
PUBLICATION METRICS: VOLUME
Volume by Type of Publication (journal & congress)
1200
900
600
0
300
2008 2009 2010 2011 2012 YTD
Manuscripts & Review Articles Abstracts Posters/Oral All Other
![Page 22: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/22.jpg)
11TH ANNUAL MEETING OF ISMPP 22
PUBLICATION METRICS: PLAN VS ACTUAL
40
30
20
15
10
5
0 Actual
Num
ber o
f Pub
licat
ions
Planned Submissions Actual Submission per Plan Actual Submission not Planned for Quarter 35
25
Planned
Q3 Q2 Q4 Q1 Actual Planned Actual Planned Actual Planned
![Page 23: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/23.jpg)
11TH ANNUAL MEETING OF ISMPP 23
• FDA and EudraCT guidelines – Timing of publications for communication of
clinical trial results • Life cycle management
– May set very specific and/or accelerated timelines for key manuscripts (e.g., new indications, etc.)
• Timelines – When measuring timelines, need to define start
and end date of publication upfront (e.g., end date could be submit, accept, publish)
PUBLICATION METRICS: TIMING
![Page 24: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/24.jpg)
11TH ANNUAL MEETING OF ISMPP 24
0
Weeks 10 20 30 40 50 60
Drug A Drug B Drug C
MANUSCRIPT SUBMISSION TIMELINESS
![Page 25: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/25.jpg)
11TH ANNUAL MEETING OF ISMPP 25
PUBLICATION METRICS: QUALITY
• Impact factor (for key publications) – Visibility of journal or article
• Acceptance in first-choice journal • Number of times the article or scientific
statement is cited in other journal articles
![Page 26: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/26.jpg)
11TH ANNUAL MEETING OF ISMPP 26
PUBLICATION METRICS: JOURNALS
6
5
4
3
2
1
0
1 2 3 4 5
Drug A Drug B Drug C
Journal values La
ncet
Am
J M
ed S
ci
Dia
bet M
ed
Dia
bete
s D
iabe
tes
Car
e D
iabe
tolo
gia
Am J
Med
C
urr M
ed R
es O
pin
Dia
bete
s, O
besi
ty &
Met
abol
ism
D
iabe
tes
Res
Clin
Pra
ct
Dia
betic
Med
icin
e En
docr
inol
ogy
Endo
cr J
In
t J C
lin P
ract
Po
stgr
adua
te M
edic
ine
Ann
Phar
mac
othe
rapy
B
est P
ract
ice
Res
Clin
End
o M
etab
C
lin T
her
Dia
bete
s Te
ch T
her
Endo
cr P
ract
J
Am P
harm
Ass
oc
J C
lin P
harm
acol
J
Man
age
Car
e Ph
arm
Tr
ansp
lant
Inte
rnat
iona
l Tr
ansp
lant
atio
n C
an J
Clin
Pha
rmac
ol
Car
diov
ascu
lar D
iabe
tolo
gy
Dru
gs
Dru
gs T
oday
Ex
pert
Opi
n In
vest
ig D
rugs
Ex
pert
Opi
n Ph
arm
acot
her
Hor
m M
etab
Res
. In
t J C
lin P
harm
acot
her
Leuk
Rec
M
ol C
ell E
ndo
Phar
mac
othe
rapy
Va
sc H
ealth
Ris
k M
anag
Pr
escr
ire In
t
![Page 27: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/27.jpg)
11TH ANNUAL MEETING OF ISMPP 27
ACCEPTANCE RATE COMPARED WITH JOURNAL AVERAGE
100
80
60
0
30
Overall Journal Acceptance Rate Product-Specific Acceptance Rate (2006-2009)
90
70
50 40
20 10
BJU
Inte
rnat
iona
l
Euro
pean
Uro
logy
Inte
rnat
iona
l Jou
rnal
of
Uro
logy
Inte
rnat
iona
l U
rogy
neco
logy
J
Jour
nal o
f Se
xual
Med
icin
e
Neu
rour
olog
y &
Uro
dyna
mic
s
The
Jour
nal
of U
rolo
gy
Uro
logy
Num
ber o
f Ar
ticle
s
![Page 28: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/28.jpg)
11TH ANNUAL MEETING OF ISMPP 28
EMERGING PUBLICATION METRICS
• Congress activities – QR codes – number of downloads
• Journal articles – – Online journals
• Number of downloads • Number of clicks through to the article • Number of page views • Number of unique visitors
– Article appearing in the top 25 articles • Specific to journal/publisher
– Number of times cited (Scopus) – Novel and innovative ways of presenting publications
![Page 29: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/29.jpg)
11TH ANNUAL MEETING OF ISMPP 29
SUMMARY
• Metrics vary and there are more than one that may apply to a given situation
• Metrics are dependent upon a product’s life cycle
• Metrics should have purpose and measure success/effectiveness of the publication plan or team performance
• Emerging metrics may provide new opportunities for measure of publication reach & quality
![Page 30: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/30.jpg)
11TH ANNUAL MEETING OF ISMPP 30
1 1 T H A N N U A L M E E T I N G O F I S M P P
UNDERSTANDING MODERN IMPACT METRICS
Mike Buschman
Co-Founder, Plum Analytics April 29, 2015
![Page 31: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/31.jpg)
11TH ANNUAL MEETING OF ISMPP 31
Altmetrics? Article Level Metrics?
![Page 32: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/32.jpg)
11TH ANNUAL MEETING OF ISMPP 32
Our Approach to Metrics
•Citations •Usage •Captures •Mentions •Social Media
ALLmetrics
![Page 33: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/33.jpg)
11TH ANNUAL MEETING OF ISMPP 33
Journal citations are the current state of
Scholarly Measure 33
Journal Impact Factor
3.161
![Page 34: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/34.jpg)
11TH ANNUAL MEETING OF ISMPP 34
Samples of PlumX Metrics Sources
• Amazon • Bit.ly • CrossRef • Delicious • Dryad • dSpace • EBSCO • ePrints • Facebook • figshare
• Github • Goodreads • Google+ • Medwave • Mendeley • PLOS • PubMed
Central • Reddit • Research
Blogging
• Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat (OCLC) • YouTube
![Page 35: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/35.jpg)
11TH ANNUAL MEETING OF ISMPP 35
All Types of Research Artifacts
• Articles • Blog posts • Book chapters • Books • Cases • Clinical Trials • Conference Papers • Data Sets • Figures • Grants
• Interviews • Letters • Media • Patents • Posters • Presentations • Reports • Source Code • Theses /
Dissertations • Videos • Web Pages
![Page 36: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/36.jpg)
11TH ANNUAL MEETING OF ISMPP 36
Metrics Categories
USAGE (clicks, downloads, views,
library holdings, video plays)
CAPTURES (bookmarks, code forks, favorites,
readers, watchers)
MENTIONS (blog posts, comments, reviews, Wikipedia links)
SOCIAL MEDIA (+1s, likes, shares, tweets)
CITATIONS (PubMed Central, Scopus,
patents)
![Page 37: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/37.jpg)
11TH ANNUAL MEETING OF ISMPP 37
USAGE
• Is anyone reading our work?
• Did anyone watch our videos?
• Usage is the #1 stat researchers want to know after Citation counts
• PlumX is the only product that includes Usage
clicks, downloads, views, library holdings,
video plays
![Page 38: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/38.jpg)
11TH ANNUAL MEETING OF ISMPP 38
• Captures indicate that someone wants to come back to the work
• Early indicator of future citations
bookmarks, favorites, readers, watchers
CAPTURES
![Page 39: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/39.jpg)
11TH ANNUAL MEETING OF ISMPP 39
• This category measures people truly engaging with your research
• Automatically uncover the conversations about your research
• Discover feedback, opinions, etc.
blog posts, comments, reviews,
Wikipedia links MENTIONS
![Page 40: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/40.jpg)
11TH ANNUAL MEETING OF ISMPP 40
• Social media measures how well a researcher is promoting their work
• This is especially important for early career researchers to measure and understand
• Track the buzz and attention around your research output
+1s, likes, shares, tweets
SOCIAL MEDIA
![Page 41: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/41.jpg)
11TH ANNUAL MEETING OF ISMPP 41
• Citations still a standard of long-term impact
• Including citations enables side-by-side analysis with other metrics categories
PubMed Central, Scopus, patents
CITATIONS
![Page 42: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/42.jpg)
11TH ANNUAL MEETING OF ISMPP 42
5 Categories in Action – Example Article
Article level metrics are the building blocks
for PlumX
![Page 43: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/43.jpg)
11TH ANNUAL MEETING OF ISMPP 43
2015 Article
![Page 44: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/44.jpg)
11TH ANNUAL MEETING OF ISMPP 44
2014 Article
![Page 45: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/45.jpg)
11TH ANNUAL MEETING OF ISMPP 45
2013 Article
![Page 46: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/46.jpg)
11TH ANNUAL MEETING OF ISMPP 46
Researcher Profile
![Page 47: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/47.jpg)
11TH ANNUAL MEETING OF ISMPP 47
Analytics – document sets and researchers
![Page 48: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/48.jpg)
11TH ANNUAL MEETING OF ISMPP 48
Journal Comparisons
![Page 50: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/50.jpg)
11TH ANNUAL MEETING OF ISMPP 50
1 1 T H A N N U A L M E E T I N G O F I S M P P
INTERPRETING ARTICLE METRICS
Tom Rees, MSc, PhD, CMPPTM
Scientific Strategy Advisor, PAREXEL International
![Page 51: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/51.jpg)
11TH ANNUAL MEETING OF ISMPP 51
Why do we need metrics?
What do we want to achieve
Have we achieved it?
(…and what are the subtle influences on metrics that could trip us up?)
![Page 52: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/52.jpg)
11TH ANNUAL MEETING OF ISMPP 52
A framework for thinking about metrics
Allen HG, et al. PloS one 8.7 (2013): e68914.
![Page 53: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/53.jpg)
11TH ANNUAL MEETING OF ISMPP 53
The best articles achieve higher metrics
Bornmann, Lutz (2015): figshare. http://dx.doi.org/10.6084/m9.figshare.1344583 Retrieved 20:54, Apr 18, 2015 (GMT)
![Page 54: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/54.jpg)
11TH ANNUAL MEETING OF ISMPP 54
But the type of article influences metrics differently
Bornmann, Lutz (2015): figshare. http://dx.doi.org/10.6084/m9.figshare.1344583 Retrieved 20:54, Apr 18, 2015 (GMT)
![Page 55: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/55.jpg)
11TH ANNUAL MEETING OF ISMPP 55
Different topics are picked up differently
Costas R, et al.(2015):. figshare. http://dx.doi.org/10.6084/m9.figshare.1335773 Retrieved 20:04, Apr 19, 2015 (GMT)
Mendeley
![Page 56: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/56.jpg)
11TH ANNUAL MEETING OF ISMPP 56
Document characteristics affect metrics differently
00.10.20.30.40.50.60.70.80.9
1
Pages References Title length Countries
Cor
rela
tion
(r-sq
uare
d)
Citations Blog posts Tweets News
Haustein S, et al. PLoS ONE 2015;0(3): e0120495
![Page 57: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/57.jpg)
11TH ANNUAL MEETING OF ISMPP 57
Open access increases scholarly activity more than social media
Alhoori, Hamed, et al. "On the Relationship between Open Access and Altmetrics." iConference 2015 Proceedings (2015).
![Page 58: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/58.jpg)
11TH ANNUAL MEETING OF ISMPP 58
Even within open access, choice of journal influences metrics differently
Rees TJ, et al. Poster presentation ISMPP 8AM 2012
![Page 59: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/59.jpg)
11TH ANNUAL MEETING OF ISMPP 59
Video abstracts don’t affect views but does that matter?
Rees TJ, et al. Poster presentation ISMPP 11AM 2015
![Page 60: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/60.jpg)
11TH ANNUAL MEETING OF ISMPP 60
Few researchers currently use social media to discover recommended research papers
0% 20% 40% 60%
Frontiers
Academia.edu
ResearchGate
BioMedExperts
Google+
Mendeley
0% 20% 40% 60%
% of full sample* % of service users
* Extrapolation Rees TJ, et al. Oral presentation ISMPP European Meeting 2014
![Page 61: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/61.jpg)
11TH ANNUAL MEETING OF ISMPP 61
And social media is not just a young-person thing
0102030405060708090
18-24 25-34 35-44 45-54 55-64 65+
%
Age (years)
(NPG), Nature Publishing Group (2014): NPG 2014 Social Networks survey. figshare. http://dx.doi.org/10.6084/m9.figshare.1132584Retrieved 11:36, Apr 24, 2015 (GMT)
![Page 62: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/62.jpg)
11TH ANNUAL MEETING OF ISMPP 62
All that tweets is not gold…
![Page 63: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/63.jpg)
11TH ANNUAL MEETING OF ISMPP 63
Dam that Social Networking…
![Page 64: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/64.jpg)
11TH ANNUAL MEETING OF ISMPP 64
People are consuming this information!
Feeny S, et al. ISMPP 10AM 2014
![Page 65: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/65.jpg)
11TH ANNUAL MEETING OF ISMPP 65
Conclusion
• Evaluating publication impact is complex, and so requires multiple measures
• Understanding what different metrics indicate is critical to their interpretation
• We are still limited to measuring what we can, rather than what we want to
• Remember, Activity ≠ Impact!
![Page 66: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/66.jpg)
11TH ANNUAL MEETING OF ISMPP 66
1 1 T H A N N U A L M E E T I N G O F I S M P P
Innovation in Metrics: Capturing the full impact of publications
![Page 67: OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC ......Blogging • Scopus • SlideShare • SourceForge • Stack Overflow • Twitter • USPTO • Vimeo • Wikipedia • Worldcat](https://reader033.fdocuments.us/reader033/viewer/2022052022/60378729b25d0e35617f6e75/html5/thumbnails/67.jpg)
11TH ANNUAL MEETING OF ISMPP 67
1 1 T H A N N U A L M E E T I N G O F I S M P P
OPTIMIZING SCIENTIFIC VALUE: SMART AND SYSTEMATIC APPROACHES
TO MEDICAL PUBLICATIONS April 27–29, 2015
Hyatt Regency Crystal City Arlington, VA, USA