Neuroimaging Data Provenance Using the LONI Pipeline Workflow Environment
description
Transcript of Neuroimaging Data Provenance Using the LONI Pipeline Workflow Environment
Neuroimaging Data Neuroimaging Data Provenance Using the LONI Provenance Using the LONI Pipeline Workflow Pipeline Workflow EnvironmentEnvironmentAllan MacKenzie-Graham IPAW2008Allan MacKenzie-Graham IPAW2008Arash PayanArash PayanIvo DinovIvo DinovJohn Van HornJohn Van HornArthur W. TogaArthur W. Toga
22
Provenance in NeuroimagingProvenance in Neuroimaging
Tools used and data described Tools used and data described must be adequately described must be adequately described and documentedand documented•Determining data qualityDetermining data quality•InterpretationInterpretation•ReproducibilityReproducibility•ReusabilityReusability•InteroperabilityInteroperability
22
33
First Provenance ChallengeFirst Provenance Challenge
33(Moreau et. al, 2007)
44
Provenance SystemsProvenance Systems
44(Moreau et. al, 2007)
55
Goals of the LONI Goals of the LONI Provenance SystemProvenance System
DescriptionDescription•DataData•ProcessingProcessing
ReproducibilityReproducibility•Across platformsAcross platforms•Across compilationsAcross compilations•Across software versionsAcross software versions
Ease of useEase of use55
66
Neuroimaging Data Neuroimaging Data ProvenanceProvenance
Neuroimaging data provenanceNeuroimaging data provenanceProjectProjectSubjectSubject
SpeciesSpeciesAgeAgeSexSex
AcquisitionAcquisitionScannerScannerOrientationOrientationWeightingWeightingField StrengthField StrengthTRTRTETETITI
77
Provenance EditorProvenance Editor
77
88
LONI PipelineLONI Pipeline
88
99
LONI Pipeline ModuleLONI Pipeline Module
99
1010
Workflow ProvenanceWorkflow Provenance <connections><connections> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /><connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /> <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" />/> <connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /><connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /> <connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /><connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /> <connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /><connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /> <connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /><connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /> <connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /><connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /> <connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /><connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /> <connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /><connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /> <connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /><connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /> <connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /> <connection source="Options.Output_0" sink="InitializeStats.Options_0" /><connection source="Options.Output_0" sink="InitializeStats.Options_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /> <connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /><connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /> <connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /><connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /> <connection source="ComputeDeformationTensors.Output_0" <connection source="ComputeDeformationTensors.Output_0" sink="ComputeVoxelwiseStatistics.SynchParameter_0" />sink="ComputeVoxelwiseStatistics.SynchParameter_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /><connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /> <connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /><connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /> <connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /> <connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /><connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /> <connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /><connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /><connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /> </connections></connections>
1010
1111
Executable ProvenanceExecutable ProvenanceExecutable provenanceExecutable provenance
EnvironmentEnvironmentOptionsOptionsInput filesInput filesOutput filesOutput filesBinary provenanceBinary provenanceBinary configurationBinary configurationConfiguration optionsConfiguration optionsSystem configurationSystem configuration
ArchitectureArchitectureOperating systemOperating systemCompilerCompilerLibrariesLibraries
Script provenanceScript provenanceShellShellScriptScriptBinary provenanceBinary provenance
1212
Alignlinear ProvenanceAlignlinear Provenance<binary name="alignlinear" version="5.2.5" package="AIR" uri="<binary name="alignlinear" version="5.2.5" package="AIR" uri="http://www.loni.ucla.edu/Software/Software_Detail.jsp?software_id=8" " compilationTime="2006-04-21T11:58:00">compilationTime="2006-04-21T11:58:00">
<systemConfig><systemConfig>
<architecture name="x86" vendor="Intel" model="T2400" processorFlags="" /><architecture name="x86" vendor="Intel" model="T2400" processorFlags="" />
<OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" <OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" kernelVersion="9.3.0" />kernelVersion="9.3.0" />
<compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" /><compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" />
</systemConfig></systemConfig>
<configOptions><configOptions>
<config name="AIR_CONFIG_GROUPS" value="1" /><config name="AIR_CONFIG_GROUPS" value="1" />
<config name="HAVE_LIBM" value="1" /><config name="HAVE_LIBM" value="1" />
<config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" /><config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" />
<config name="AIR_CONFIG_REP16" value="1" /><config name="AIR_CONFIG_REP16" value="1" />
<config name="AIR_CONFIG_VERBOSITY" value="0" /><config name="AIR_CONFIG_VERBOSITY" value="0" />
<config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" /><config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" />
<config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" /><config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" />
<config name="AIR_CONFIG_THRESHOLD1" value="7000" /><config name="AIR_CONFIG_THRESHOLD1" value="7000" />
<config name="AIR_CONFIG_OUTBITS" value="16" /><config name="AIR_CONFIG_OUTBITS" value="16" />
<config name="AIR_CONFIG_REQ_PERMS" value="2" /><config name="AIR_CONFIG_REQ_PERMS" value="2" />
<config name="AIR_CONFIG_THRESHOLD2" value="7000" /><config name="AIR_CONFIG_THRESHOLD2" value="7000" />
<config name="AIR_CONFIG_PIPES" value="1" /><config name="AIR_CONFIG_PIPES" value="1" />
<config name="AIR_CONFIG_DECOMPRESS" value="1" /><config name="AIR_CONFIG_DECOMPRESS" value="1" />
<config name="AIR_CONFIG_PATHS" value="1" /><config name="AIR_CONFIG_PATHS" value="1" />
<config name="STDC_HEADERS" value="1" /><config name="STDC_HEADERS" value="1" />
</configOptions></configOptions>
</binary></binary> 1212
1313
ReproducibilityReproducibility
Across PlatformAcross Platform•ICA workflowICA workflow
Across compilationsAcross compilations•MDA workflowMDA workflow
1313
1414
Independent Components Independent Components AnalysisAnalysis
1515
Different Architectures Different Architectures Yield Different ResultsYield Different Results
1616
Minimum Deformation AtlasMinimum Deformation Atlas
1717
Different Compilation Different Compilation Options Yield Different Options Yield Different
ResultsResults
1818
Complex Neuroimaging Complex Neuroimaging WorkflowWorkflow
1818
1919
Future DirectionsFuture Directions Community involvementCommunity involvement
• provenance.loni.ucla.eduprovenance.loni.ucla.edu Make LONI Pipeline aware of provenance Make LONI Pipeline aware of provenance filesfiles• Read in provenance fileRead in provenance file• Display executable provenanceDisplay executable provenance• Append provenance informationAppend provenance information• Write out provenance fileWrite out provenance file
Visualize provenance filesVisualize provenance files• Interface similar to LONI PipelineInterface similar to LONI Pipeline• Invoke LONI Pipeline to recreate file or Invoke LONI Pipeline to recreate file or modify processingmodify processing
Provenance DatabaseProvenance Database• Database of workflowsDatabase of workflows
2020
AcknowledgementsAcknowledgements Arthur W. TogaArthur W. Toga
• Director, Laboratory of Neuro ImagingDirector, Laboratory of Neuro Imaging Arash PayanArash Payan
• Lead Developer, LONI PipelineLead Developer, LONI Pipeline Ivo D. DinovIvo D. Dinov
• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging
John D. Van HornJohn D. Van Horn• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging