Badges to Acknowledge Open Practices

21
Badges to Acknowledge Open Practices Andrew Sallans Center for Open Science http://cos.io IASSIST June 2014

description

Presented at IASSIST 2014 meeting, Toronto, Canada.

Transcript of Badges to Acknowledge Open Practices

Page 1: Badges to Acknowledge Open Practices

Badges to Acknowledge Open Practices

Andrew SallansCenter for Open Science

http://cos.io IASSIST

June 2014

Page 2: Badges to Acknowledge Open Practices

Openness is a core value of scientific practice

Page 3: Badges to Acknowledge Open Practices

Challenges: Perceived normsNorms

Communality Open Sharing

Universalism Evaluate research on own merit

Disinterestedness Motivated by knowledge and discovery

Organized skepticism Consider all new evidence, even against

one’s prior work

Quality

Counternorms

Secrecy Closed

Particularlism Evaluate research by reputation

Self-interestedness Treat science as a competition

Organized dogmatism Invest career promoting one’s own

theories, findings

Quantity

Page 4: Badges to Acknowledge Open Practices

Anderson, Martinson & DeVries, 2007

Page 5: Badges to Acknowledge Open Practices

Challenges

● Perceived norms (Anderson, Martinson & DeVries, 2007)

● Motivated reasoning (Kunda, 1990)

● Minimal accountability (Lerner & Tetlock, 1999)

● I am busy (Everyone)

Page 6: Badges to Acknowledge Open Practices

How might this change?

Page 7: Badges to Acknowledge Open Practices

Top-down● Committees● Journal Editors● Federal Agencies● University

administrators

Bottom-up● Exposing users to

good practices● Training sessions● Workshops● Spreading the

message through active publishing

● Going to where the user is now

● Consultants serving as “Shock troops”

Focus on simple interventions that researchers can actually adopt.

Page 8: Badges to Acknowledge Open Practices
Page 9: Badges to Acknowledge Open Practices

How to get a badge...1. Disclosure --- author provides public statement that they’ve met badge criteria

2. Peer Reviewed --- independent review of author’s public statement and meeting of criteria

Page 10: Badges to Acknowledge Open Practices

What are the criteria?● Persistent path to the data, materials, pre

registration, etc.● Sufficient information for an independent

person to reproduce the results● Other more specialized items...

Page 11: Badges to Acknowledge Open Practices

Who endorses these badges?

Page 12: Badges to Acknowledge Open Practices

Which journals are adopting?

Page 13: Badges to Acknowledge Open Practices
Page 14: Badges to Acknowledge Open Practices
Page 15: Badges to Acknowledge Open Practices

Going further…baking badges

Page 16: Badges to Acknowledge Open Practices
Page 17: Badges to Acknowledge Open Practices
Page 18: Badges to Acknowledge Open Practices
Page 19: Badges to Acknowledge Open Practices

What comes next?● Improve the badge issuing workflow● Refine integration into peer review process● Gain adoption of more journals/organizations● Measure effect

Page 20: Badges to Acknowledge Open Practices

Lastly, this is a community effort...Ben B. Blohowiak | Johanna Cohoon | Lee de-Wit | Eric Eich | Frank J. Farach | Roger Giner-Sorolla | Fred Hasselman | Alex O. Holcombe | Macartan Humphreys | Melissa Lewis | Brian A. Nosek | Jonathan Peirce | Andrew Sallans | Jeffrey R. Spies | Chris Seto | Sara Bowman

Note: bold are non-COS

Page 21: Badges to Acknowledge Open Practices

Thanks!Find out more and get involved here:https://osf.io/tvyxz/

Email me: [email protected] Twitter: @asallans