Post on 18-Jan-2016
Academic Publishing is Evolving…
How Should We ‘Evaluate’Scientific Publications Today?
Pete Binfield Co-Founder and Publisher PeerJ
Samuel Merritt - 10/30/2013
@ThePeerJhttps://peerj.com
@p_binfieldpete@peerj.com
Academic Publishing is Evolving…
• What do we mean when we say ‘Evaluate’?• Evaluating ‘Impact’ or ‘Reception’ or ‘Reach’ or ‘?’• Providing Subjective Opinions & Evaluations• Evaluating ‘Integrity’
• And at what level of granularity?• The journal?• The article?• The paragraph?
Academic Publishing is Evolving…
#1. Evaluating ‘Impact’ or ‘Reception’ or ‘Reach’ or ‘Interest’
or ‘Readership’ or, or, or
Academic Publishing is Evolving…
• Online-only, peer-reviewed, open access journals• covering a very broad subject area• selecting content based only on ‘technical
soundness’ (or similar) • with a business model which allows each article to
cover its own costs
Open Access ‘MegaJournals’
Academic Publishing is Evolving…
Q2 13
Q1 13
Q4 12
Q3 12
Q2 12
Q1 12
Q4 11
Q3 11
Q2 11
Q1 11
Q4 10
Q3 10
Q2 10
Q1 10
Q4 09
Q3 09
Q2 09
Q1 09
Q4 08
Q3 08
Q2 08
Q1 08
Q4 07
Q3 07
Q2 07
Q1 07
Q4 06
0
1000
2000
3000
4000
5000
6000
7000
8000
Year Pubs Notes2007
1,200 Larger than ~ 95% of all journals
2008
2,800 Largest OA journal in world
2009
4,400 3rd largest journal in world
2010
6,750 Largest journal in world
2011
13,800 ~1.4% of PubMed output in that year
2012
23,500 ~2.4% of PubMed output in that year
2013
~31,000
>3% of the literature
PLOS ONE Quarterly Output
Academic Publishing is Evolving…
Known MegaJournals (Oct 2013)
Academic Publishing is Evolving…
In addition, if we allow for narrow scope ‘megajournals’ then we should also include:
• All of the “Frontiers in…” Series (part of Nature)
• All of the “BMC Series” (~ half of BMC)
• ~ 1/3 of Hindawi’s current output
All of these titles refuse to pre-judge what the audience should be reading (other than determining that the content should join the literature).
Academic Publishing is Evolving…
An OA future containing MegaJournals
PLoSONE
SAGEOpen
PeerJ
ALL
OTHER
OA
JOURNALS
etc.etc.
Academic Publishing is Evolving…
The Effect of the ‘MegaJournal’
• Rapidly Approaching ~10% of all published content, spurring new developments
• Require, and have given rise to, Article-Level Metrics
• Publish Negative Results, Replication Studies, Incremental Articles
• Dramatic Improvement to the Speed of the Ecosystem
• Dramatic Improvement to the Efficiency of the Ecosystem
Academic Publishing is Evolving…From “Article- ‐Level Metrics, A SPARC Primer” - http://www.sparc.arl.org/sites/default/files/sparc-alm-primer.pdf
Academic Publishing is Evolving…Screenshot from ~ Nov 2009 but Way Back Machine has examples from April 2008
Academic Publishing is Evolving…
PLOS ALMs
Academic Publishing is Evolving…
PeerJ ALMs
Academic Publishing is Evolving…http://www.altmetric.com/details.php?doi=10.7717/peerj.182
Academic Publishing is Evolving…
Academic Publishing is Evolving…
• Rapidly Approaching ~10% of all published content, spurring new developments
• Require (and have stimulated) Article-Level Metrics
• Publish Negative Results, Replication Studies, Incremental Articles
• Dramatic Improvement to the Speed of the Ecosystem
• Dramatic Improvement to the efficiency of the way the ecosystem currently ‘filters’ content
The Effect of the ‘MegaJournal’
Academic Publishing is Evolving…
“rejected from at least six journals (including Nature, Nature Genetics, Nature Methods, Science) and took a year to publish before going on to be my most cited research paper (150 last time I looked)” – Cameron Neylon
Academic Publishing is Evolving…http://grigoriefflab.janelia.org/rejections
Academic Publishing is Evolving…
http://blog.rubriq.com/2013/06/03/how-we-found-15-million-hours-of-lost-time/
“…in a recent report Kassab and his colleagues estimated that Elsevier currently rejects 700,000 out of 1 million articles each year.”
http://poynder.blogspot.co.uk/2013/10/media-research-analyst-at-exane-bnp.html
Academic Publishing is Evolving…
#2. ‘Subjective’ Opinions & Evaluations(i.e. contextual, human evaluations)
Academic Publishing is Evolving…
PeerJ Q&A
Academic Publishing is Evolving…
PMC Commons
Academic Publishing is Evolving…
#3. Evaluating ‘Integrity’
Academic Publishing is Evolving…
Academic Publishing is Evolving…
Academic Publishing is Evolving…
Academic Publishing is Evolving…
Academic Publishing is Evolving…
Open Peer Review would makethis problem disappear.
Overnight
Academic Publishing is Evolving…
• Atmospheric Chemistry and Physics - Reviewers comments published on pre-pub discussion site. Reviewer names optional.
• Biology Direct - Reviewer comments published, and reviewers named
• BMJ Open - All reviewers named, all reports public• eLife - Decision letter published with articles with author
approval. Reviewers anonymous, but editor named. • EMBO journal - Review process file published with
articles. Reviewers anonymous, editor named. • F1000Research - All reviewers named, all reports
public. • Frontiers journals - Reviewers named, but reports not
public• GigaScience - Pre-publication history published with
articles, and reviewers named. (encouraged, opt-out)• Medical journals in the BMC series - Pre-publication
history published with articles, and reviewers named (encouraged).
• PeerJ - Peer review history published with articles with author approval. Reviewers encouraged to sign report.
Journals Practicing Open Peer-Review
Academic Publishing is Evolving…
• ~40% of PeerJ Reviewers name themselves.• ~80% of PeerJ Authors reproduce their peer review history
Academic Publishing is Evolving…
• What do we mean when we say ‘Evaluate’?• Evaluating ‘Impact’ or ‘Reception’ or…• Providing ‘Subjective’ Opinions & Evaluations• Evaluating ‘Integrity’
• And at what level of granularity?• The journal?• The article?• The paragraph?
Academic Publishing is Evolving…
Thank You
Pete BinfieldCo-Founder and Publisher
@p_binfieldpete@peerj.com @ThePeerJ