Reproducible Computational Experiments Using MADAGASCAR...
31
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions Reproducible Computational Experiments Using M ADAGASCAR Software Package Sergey Fomel Bureau of Economic Geology Jackson School of Geosciences The University of Texas at Austin May 15, 2008 MADAGASCAR Software Package 2nd Annual Scientific Software Days
Transcript of Reproducible Computational Experiments Using MADAGASCAR...
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Reproducible ComputationalExperiments Using
MADAGASCAR Software Package
Sergey Fomel
Bureau of Economic GeologyJackson School of GeosciencesThe University of Texas at Austin
May 15, 2008
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
The Magic of Science
I Verification and ValidationI ReproducibilityI Peer Review
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
The Magic of Computer Science
I
I
Abandoning the habit of secrecy in favor ofprocess transparency and peer review wasthe crucial step by which alchemy becamechemistry. In the same way, it is beginning toappear that open-source development maysignal the long-awaited maturation ofsoftware development as a discipline. Eric S.Raymond, TAUP, 2004
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
The Magic of Computational ScienceWithin the world of science, computation isnow rightly seen as a third vertex of a trianglecomplementing experiment and theory.However, as it is now often practiced, onecan make a good case that computing is thelast refuge of the scientific scoundrel [...]Where else in science can one get away withpublishing observations that are claimed toprove a theory or illustrate the success of atechnique without having to give a carefuldescription of the methods used, in sufficientdetail that others can attempt to repeat theexperiment? Randall J. LeVeque, ICM, 2006
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
(Hale, 1984)
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
(Hale, 1984)
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
The Magic of Computational Science
An article about computational science in ascientific publication is not the scholarshipitself, it is merely advertising of thescholarship. The actual scholarship is thecomplete software development environmentand the complete set of instructions whichgenerated the figures.Jon B. Buckheit and David L. Donoho,WaveLab, 1995
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Stanford Exploration Project
I 1 proposal, 60 PhDsI Electronic booksI (Schwab, Karrenbach, and Claerbout, 2000)
The purpose of reproducible research is tofacilitate someone going a step further bychanging something. The first step thatsomeone will want to make is to be sure thatyour work is reproducible before they changeand improve upon it. Jon F. Claerbout, 1991
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Follow Up
I Computational Harmonic Analysis (Donoho et al)I Computational Wave Propagation (LeVeque)I Computational Biostatistics (Gentlemen et al)I Summarized in CiSE, Jan/Feb 2009
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Lessons
I Reproducibility = MaintenanceI Community is importantI Tools are important
1. Number crunching2. Visualization and experiment setup3. Publications and presentations
I Computational experiments are tests
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR PackageI Publicly available since 2006I Version 0.9.6I Open source, open community, open scienceI Vladimir Bashkardin, Jules Browaeys, Cody
Brown, Will Burnett, Maria Cameron, SergeyFomel, Gilles Hennenfent, Trevor Irons, JimJennings, Long Jin, Yang Liu, Doug McCowan,Henryk Modzelewski, Colin Russell, Paul Sava,Jeffrey Shragge, Eduardo Filpo Silva, Ioan Vlad,Jia Yan
I http://rsf.sourceforge.net
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Tools
1. Number crunchingI Main programs (C, Fortran, C++, etc)I 500 modules
2. Visualization and experiment setupI Data processing flows (Python/SCons)I 600 scripts, 1,700 tests
3. Publications and presentationsI Books and papers (LATEX and Python/SCons)I 100 papers
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
MADAGASCAR Tools
1. Number crunchingI Main programs (C, Fortran, C++, etc)I 500 modules
2. Visualization and experiment setupI Data processing flows (Python/SCons)I 600 scripts, 1,700 tests
3. Publications and presentationsI Books and papers (LATEX and Python/SCons)I 100 papers
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Outline
The Magic of Science
History of Reproducible Research
Computational Experiment Example
Conclusions
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Conclusions
I Reproducible computational experimentsI Reproducibility = maintenanceI MADAGASCAR software package
MADAGASCAR Software Package 2nd Annual Scientific Software Days
The Magic of Science History of Reproducible Research Computational Experiment Example Conclusions
Help Needed
I Automatic testingI Efficient parallelizationI Visualization standardsI Graphical User InterfaceI . . .
http://rsf.sourceforge.net
MADAGASCAR Software Package 2nd Annual Scientific Software Days
