Ideals and Norms in Scholarship
-
Upload
paul-groth -
Category
Science
-
view
545 -
download
1
Transcript of Ideals and Norms in Scholarship
![Page 1: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/1.jpg)
Ideals and Norms in
Scholarship
Paul Groth
Elsevier Labs
April 20, 2015
![Page 2: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/2.jpg)
Warning:
my perspective
not comprehensive
still thinking
Responsible Science, Volume I: Ensuring the Integrity of the
Research Process. 1992
Panel on Scientific Responsibility and the
Conduct of Research, National Academy of
Sciences, National Academy of
Engineering, Institute of Medicine
Smarter people have thought more:
![Page 3: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/3.jpg)
Ideal
“a principle to be aimed at”
![Page 4: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/4.jpg)
Ideal
“a principle to be aimed at”
Norm
“a standard or pattern, esp. of social behavior, that is typical or expected of a group”
![Page 5: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/5.jpg)
WHY NOW?
![Page 6: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/6.jpg)
Reproducibility Crisis
“Amgen, recently dispatched 100 Amgen
scientists to replicate 53 landmark experiments
in cancer—the kind of experiments that lead
pharmaceutical companies to sink millions of
dollars to turn the results into a drug. In March
Begley published the results:They failed to
replicate 47 of them.”
“Good Scientist! You Get a Badge” Slate Aug 14. 2012
Original Paper: http://doi.org/10.1038/483531a
![Page 7: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/7.jpg)
![Page 8: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/8.jpg)
Figure 1. Papers published and retracted per year since 1973.
Steen RG, Casadevall A, Fang FC (2013) Why Has the Number of Scientific Retractions Increased?. PLoS ONE 8(7): e68397.
doi:10.1371/journal.pone.0068397
http://127.0.0.1:8081/plosone/article?id=info:doi/10.1371/journal.pone.0068397
![Page 9: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/9.jpg)
![Page 10: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/10.jpg)
![Page 11: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/11.jpg)
IS IT A PROBLEM FOR
COMPUTER SCIENCE?
![Page 12: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/12.jpg)
http://reproducibility.cs.arizona.edu/v2/in
dex.html
![Page 13: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/13.jpg)
![Page 14: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/14.jpg)
http://cs.brown.edu/~sk/Memos/Examining-Reproducibility/
![Page 15: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/15.jpg)
![Page 16: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/16.jpg)
![Page 17: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/17.jpg)
![Page 18: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/18.jpg)
IDEAL
![Page 19: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/19.jpg)
![Page 20: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/20.jpg)
The R Dimensions
Research Objects facilitate research that is
reproducible, repeatable, replicable, reusable,
referenceable, retrievable, reviewable, replayable,
re-interpretable, reprocessable, recomposable,
reconstructable, repurposable, reliable,
respectful, reputable, revealable, recoverable,
restorable, reparable, refreshable?”
@dder 14 April 2014
sci method
access
understand
new use
social
curation
Research Object
Principles
![Page 21: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/21.jpg)
WHY NOT?
![Page 22: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/22.jpg)
Cost of documentation
http://www.indoition.com/en/services/costs-
prices-software-documentation.htm
![Page 23: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/23.jpg)
People try to use the science only if
the science looks important
Why? – it’s expensive to r* science – see the next slides
![Page 24: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/24.jpg)
24Yolanda GilUSC Information Sciences Institute [email protected]
Measuring Time Savings with
“Reproducibility Maps” [Garijo et al PLOS CB12]
2 months of effort in reproducing published method (in PLoS’10)
Authors expertise was required
Comparison of ligand binding sites
Comparison of dissimilar protein structures
Graph network generation
Molecular Docking
Work with D. Garijo of UPM and P. Bourne of UCSD
![Page 25: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/25.jpg)
“The Reproducibility Project: Cancer Biology has been funded through a
$1.3mm grant from the Laura and John Arnold Foundation,”
http://validation.scienceexchange.com/#/cancer-
biology
![Page 26: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/26.jpg)
NORMS
![Page 27: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/27.jpg)
Remember: science is a social system
![Page 28: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/28.jpg)
Publishing is central
• “But Ivan Oransky, founder of the blog
Retraction Watch, says data-sharing isn’t
enough. The incentive structure in science
remains a problem, because there is too
much emphasis on getting published in top
journals,…”
http://www.washingtonpost.com/national/health-science/the-new-scientific-revolution-reproducibility-at-last/2015/01/27/ed5f2076-9546-11e4-927a-
4fa2638cd1b0_story.html
![Page 29: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/29.jpg)
Publishers Guidance
• Committee on Publishing Ethics
– http://publicationethics.org
– How to handle authorship
• http://publicationethics.org/files/u2/2003pdf12.pdf
• Society of Scholarly Publishing
– Ethical principles of scholarly publishing
![Page 30: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/30.jpg)
Guidance for Authors
• Conflict of Interest
• Confidentiality
• Privileged Information/Transparency
• Reporting standards and research conduct
• Data Access and Retention
• Originality and Plagiarism
• Multiple, Redundant or Concurrent Publication
• Acknowledgement of Sources
• Authorship of the Paper
• Fundamental errors in published works
http://www.stm-assoc.org/2013_05_21_STM_Ethical_Principles_for_Scholarly_Publication.pdf
![Page 31: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/31.jpg)
Two norms sum it up
• Be transparent*
• Embrace the iteration inherent in science
* If we encourage transparency, well then we should be constructive
![Page 32: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/32.jpg)
Co-rex-ions
http://whatsinjohnsfreezer.com/2014/05/10/co-rex-ions/
“My message from that
experience is to get out
in front of problems like
this, as an author.”
![Page 33: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/33.jpg)
http://ivory.idyll.org/blog/replication-i.html
![Page 34: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/34.jpg)
Registered Reports
• “conducting the peer review prior to data
collection and analysis”
![Page 35: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/35.jpg)
![Page 36: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/36.jpg)
“Science’s incremental, self-correcting
process can be carried out quite effectively
by publishing new papers that correct and
improve on old ones, rather than dismantling
the older papers themselves.”
http://whatsinjohnsfreezer.com/2014/05/10/
co-rex-ions/
![Page 37: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/37.jpg)
Constructive criticism
“I like to think of the Braintrust
as Pixar’s version of peer
review, a forum that ensures we
raise our game – not by being
perspective but by offering
candor and deep analysis”
-- Ed Catmull p. 93
![Page 38: Ideals and Norms in Scholarship](https://reader033.fdocuments.us/reader033/viewer/2022042817/55a5f30c1a28abd53d8b472f/html5/thumbnails/38.jpg)
Conclusion
• Scientists are people too
• Ideal as a guiding light
• Discussion
– What are other norms?
– Is this a correct characterization of ideals?