Peer review models and what are the trends today

Moderated by Valda VinsonPresented by Emma Veitch and Larry Sawers

Meet The Editors: Publishing HIV Research

Choice in Peer Review Models

Emma VeitchPLOS Medicine, PLOS ONE

2

3

Peer review at PLOS journals

Lots of choice for authors and readers!

4

What’s the evidence for open vs closed peer review?

• Closed peer review criticized for lack of accountability

• Randomized trial requiring reviewers to be identified to author: “no important effect on quality…, the recommendation regarding publication, or the time…, but significantly increased the likelihood of reviewers declining to review.”

• van Rooyen S et al, Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. BMJ 1999;318:23-7.

• Another RCT: telling reviewers their reviews may be posted on the web associated with high refusal rate and > time to review, but no effect on quality.

• Van Rooyen et al, Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ 2010; 341 doi: 10.1136/bmj.c5729

• So evidence of benefit from open peer review is mixed

• BMJ studies concluded open peer review preferable for ethical reasons, although trials suggested there could be barriers to implementation

5

Peer review models at PLOS journals

• All PLOS journals have optional open/closed peer review• ie, reviewer can choose whether to sign• On average just under half choose to sign on PLOS

Medicine• We’d love to go OPEN but are not yet convinced of the

benefits – and recognise impact on turnaround time • We’ve experimented with posting reviews openly – with

reviewer consent at the end of the process – on PLOS ONE• No complaints but is time consuming and selective• Academic editors are named on papers they handle• PLOS is committed to experimenting with alternative models,

more later…

Consider impact?

Professional Editors? Academic Editors:

✓ ✓Collaborate with

professional editors to make decisions

✓Run entire peer review and decision-making

process

✓Run peer review and

decision-making process, with assistance available from professional editors

Other types of peer review model at PLoS

7

PLOS ONE, the “MegaJournal”

• Any work that is scientifically and ethically sound within biomedicine can be published

• Rapid editorial decision making based on scientific soundness

• Judge impact + significance after publication

• Editorial decision making by members of editorial board, supported by journal staff and in-house editors

• Authors choose with their feet – aka, their papers!

• Submissions currently at around 3500/month, up from around 500/month in January 2010

8

Open Access and Alt-metrics

• Evaluate impact after publication

• Accesses• Citations• Twitter feed• Annotations and

comments

9

• The Atlantic July 24• http://www.theatlantic.com/

technology/archive/2012/07/the-great-sieve-this-is-what-browsing-scientific-research-looks-like/260259/#.UA8Seo10auY.twitter

• “Fifty-three million-odd page views for some 37,000 papers is no joke….their open model is attracting tons of readers and discussion -- both on the content of their articles, and, now, the shape of their sieve”

10

PLOS journal experimentation with peer review

11

PLOS journal experimentation with peer review

Reviewer comments, link to wikipedia entry for PLOS CompBio “Topic Page” paper

12

PLOS Comp Bio “Topic Page” on Wikipedia

13

Peer Review: The Bottom Line• Probably not enough data on peer review models

(especially from good randomized trials) to know what works and why

• PLOS is experimenting with social media/interaction, making metrics available and variants on online open peer review

• We need to do much more to collect good data on outcomes associated with different types of peer review, and reviewers + editors preferences and behaviours

14

My Competing Interests

• I’m a full time employee of PLOS

• My salary’s not linked to the number of articles I handle or publication fees that are paid by authors

• Other than PLOS salary I don’t have any other sources of income (unfortunately)

• I’ve had some reimbursement for local travel costs (and things like conference dinners) for involvement in publishing initiatives such as those of the EQUATOR group, which develops guidelines for how to better report scientific studies

• More details are at http://www.plosmedicine.org/static/editorsInterests.action#eveitch