Bill St. Arnaud CANARIE Inc – canarie Bill.st.arnaud@canarie
3D Slicer - CANARIE - Advancing Canada's knowledge and ... · PDF file3D Slicer: Open-source...
Transcript of 3D Slicer - CANARIE - Advancing Canada's knowledge and ... · PDF file3D Slicer: Open-source...
Csaba Pinter
3D Slicer:Open-source Platform for Medical Image Visualization and Analysis
Laboratory for Percutaneous Surgery, School of Computing, Queen’s University, Kingston, ON, Canada
Thank you, CANARIE!
- 2 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
+
Medical research and the“valley of death”
- 3 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Nature 453, 840-842 (2008) | doi:10.1038/453840a
• “A chasm […] between biomedical researchers and the patients who need their discoveries”
• “The clinical and basic scientists don't really communicate”
• In software terms:the difficult path from Matlab scripts to clinical software
Basic research
(bench)
Clinical routine
(bedside)
Right tool for the job
Research tool
“Should it be done?”
Robust and usable
enough for clinical
evaluation, flexible,
open, portable,
community supported
Technological prototype
Clinical tool
“Can it be done?”
Innovative,
not robust,
usually single developer
Patient ready
FDA approved,
company supported,
closed source
www.senkox.comwww.unusuallocomotion.comJalopnik.com
- 4 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
- 5 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Background for 3D Slicer
• Medical image analysis and visualization platform
• Since 1997
• $50M in funding(>2,000 years of labor)
• Professionally engineered
• 1,000+ analysis functions
• Application framework: customizable, extensible
• Completely free (BSD)• Multi-platform
Fedorov, et al. "3D Slicer as an image computing platform for the Quantitative Imaging Network." Magnetic resonance imaging 30.9 (2012): 1323-1341.
Visualization1. 2D (slice) and 3D views,
chart views
2. Configurable layout
3. Multi-modality imagefusion (foreground, background, label map)
4. Transforms, vector and tensor field visualization
5. Surface and volumerendering
6. Time sequence data
5
4
1 2, 3
6
- 6 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Registration
• Manual: translation, rotation in 3D
• Automatic: rigid, deformable, with various similarity metrics, initialization methods, optimizers, masking, etc.
• Extensions: structure-based registration, Elastix, etc.
- 7 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Segmentation• Manual (paint, draw, scissor, threshold, etc.)
• Semi-automatic (region-growing, fill between slices, etc.)
• Automatic (atlas-based, robust statistics, etc.)
- 8 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Large user community
250 000+ downloads over the past 4 years:
500 downloads per week in 20122000 downloads per week in 2017
- 9 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Community support
• Extensive documentationwww.slicer.org/wiki/Documentation/Nightly
• Tutorials: both basic and specializedwww.slicer.org/wiki/Documentation/4.6/Training
• User and developer forumhttps://discourse.slicer.org
• Weekly Slicer clinic / hangout
• Bootcamps, hackfests, project weeks
- 10 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Visualization tutorial
Programming tutorial
Project week
• Twice a year• Bring your own project, work
with experts• Meetings, training• Upcoming:
June 26-30, Catanzaro Lido, Italy
- 11 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
- 12 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Slicer for translational research
3D Slicer: a cross platform system fortranslating innovative algorithms intoclinical research applications
What does a researcher need ?
• Easily deployable
• Extensible and reconfigurable rich utility libraries
• Stable base
What does a user expect ?
• Easy install and upgrade
• “Standard” clinical behavior
• Advanced functionality
• Consistent interface
Courtesy R. Kikinis
3D Slicer in clinical use
Tracking peritumoral
white matter fibers
Diagnosis of
Different Tumors
in Lung Cancer
MRI-guided
prostate
biopsy
Brain surgery
Breast cancer
surgery guidance
Radiation dose
calculations
Model-Guided Deep
Brain Simulation
Diagnosis of Osteoarthritis
Degeneration
Quantitative assessment
of COPD
Clinical
users drive
creation of
technology
Surgical
navigation
- 13 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Interdisciplinary applications
- 15 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
• SlicerAstro: Visualization of hydrogen in galaxies
- 17 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Automatic regression testing
• Automatic tests ensure detecting regression errors
• Results published on web dashboard every night
– For 3D Slicer main application
– For every extension
– For each supported platform
- 18 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Slicer is extensible
The Slicer Extension Manager offers the possibility to the user to download and install additional Slicer modules
- 19 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Image-guided therapy
Nephrostomy tutor: draining the urine from the kidney in a training phantom with tracking and navigation
hardware
OpenIGTLink
Registration Segmentation ...3D Slicer
Calibration, synchronization, pre-processing, simulation, record&replay
Visualization
SlicerIGT
Pivot calibration
PLUS
software
SlicerRT
DICOM-RT import
VTK, ITK, CTK, QT, DCMTK, …
... ...
ProstateNav
Device interface
LumpNav Gel dosimetry
...
...Slicelets
(custom Slicer applets)...
Tool watchdog
Model registration
Dose volume histogram
Dose comparison
Breach warningSlicer
extensions
DICOMOpenIGTLink
Quantification
hundreds of devices (imaging, position tracking, various sensors, and manipulators)
Ultrasoundscanners
Robotic devices, manipulators
Navigationsystems
Surgical microscopes, endoscopes
MRI, CT, PET scanners
Image-guided therapy in 3D Slicer
- 20 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
External beam plan.
Film dosimetry
Building on a platform
BrainsTools ext1.3%
Qt29.2%
VTK27.6%
ITK13.1%
Python9.9%
Numpy6.8%
DCMTK5.2%
3DSlicer core3.5%
CTK1.7%
Plus toolkit1.4%
SlicerIGT ext0.2%
LumpNav ext0.01%
SlicerIGT1.6%
LINES OF SOURCE CODE - ILLUSTRATED THROUGH LUMPNAV(NAVIGATION SOFTWARE FOR BREAST CANCER SURGERY)
- 21 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
- 22 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
PerkLab contributions to 3D Slicer
Slicer core commits in past 3 years: 726/2912 (25%)
Extension downloads (overall, as of 2017.02.)
• #1: SlicerRT: 10400
• #2: SlicerIGT: 5000
• #3: Matlab Bridge: 4200
• #7: Volume Clip: 3300
• #16: Sequences: 2200
• #27: PerkTutor: 1800
• #34: GelDosimetry: 1700
• Average extension: 1KView from SlicerRT
- 23 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2016
Example: Use custom MATLAB algorithm in 3D Slicer
• Make MATLAB algorithms easier to use
• Integrate into workflow
• Rely on the power of SlicerRT and 3D Slicer
• No programming
DICOM-RT
Non-DICOM
- 24 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
SlicerRT used for creating a sample prostate proton treatment plan on an abdominal phantom
Example: External Beam Planning
Example: Gel Dosimetry
25- 25 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Example: MRI/US fusion for biopsy
- 26 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Example: LumpNav (touch optimized)
27
http://www.slicerigt.org/wp/breast-cancer-surgery/
- 27 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
- 28 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Planned Canarie-funded developments
• Surface digitization using high-end and low-cost scanners
• Position tracking using low-cost 3D vision systems
• Augmented and Virtual Reality display
• Sharing of algorithms between Plus toolkit and SlicerIGT
intel.com
Rankin et al. 2017House et al. 2017
- 29 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Thank you!
[email protected] http://slicer.org
- 31 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2013
• One of the 3D Slicer core developer groups
– Leading Image-Guided Therapy development group
• Interdisciplinary: computer science, electrical and mech. engineering, clinical sciences (radiology, radiation oncology, and surgery)
• Information: http://perk.cs.queensu.ca
About our lab
Without an application platform
• Each application is developed from ground up
• Completely new software is developed for each problem/procedure/device
• Significant work is needed to integrate new, advanced algorithms
• Core functionalities are already implemented
• New software modules can be developed for specific needs
• Many new, advanced algorithms are available
• Well-supported with a large user and developer community
Building on an application platform
Quick start.
Huge waste of time, money, and effort overall.
Investment at the beginning: learning.
Minimal wasted efforts.
- 32 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Data import/export• DICOM: 2D/3D/4D volumes,
structure sets, dose volumes, etc. (extensible withoutSlicer core changes)
• Research data formatsfor volumes, meshes, transforms (NRRD, MetaIO, VTK, HDF, etc.)
• Common non-medical data formats (JPEG, TIFF, etc.)
• Save and complete restore of application state
- 33 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Many other modules...
• Image filtering(image noise reduction, MRI bias correction, etc.)
• Surface processing
• Diffusion imaging
• Quantification, statistics
• ...
- 34 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
• Each optimal for
• either storage (A)
• or analysis (C)
• or visualization (B,D)
• Imposed needs
• Conversion
• Simultaneous– Visualization
– Transformation
Various representations
- 35 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Typical representations:A: Contours, B: Surface, C: Image, D: Ribbons
Patient
Patient
• 1 segmentation contains N segments (structures)
– Coherence ✓
• Provides automatic conversions
– Operation ✓
• Each segment contains multiple representations
– Identity ✓
Segmentation “object”
- 36 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Tumor (contour)
Tumor
Brain (contour)
Brain
• “Promoted” representation
Master representation
- 37 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017
Patient
• Conversions use it as source
• When changed, the other representations are cleared
Tumor
image
surface
– And re-converted as needed
• When saving to disk, this representation is written
• Solves Validity ✓